configuring site-to-site

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

configuring site-to-site

anup s
Hi,
   I am trying to perform a site to site configuration.

I am trying to do a GetFile at one site and do a PutFile in another.

What I did:
Configured NiFi in machine A and set the socket input port number and set secure to false in nifi.properties
 - Ran the first NiFi instance in machine A
 - Setup a GetFile with a input dir on machine A
 - Connected it to an output port (named it XYZ)

Configured NiFi in machine B and set the socket input port number to a different number and set secure to false in nifi.properties
 - Ran the second NiFi instance in machine B
 - Setup a PutFile processor with the local dir of machine B
 - Connected it to an input port (named it same, XYZ )

When I run both the nifi instances I do not see the files getting transferred. What am i missing?
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Mark Payne
Anup,

Site-to-site doesn't work by having a port with the same name on both
machines. There have been a few people try that, so I guess that's a
fairly common misconception. Sorry about that!

On Machine A, you don't want to setup an Output Port. Instead, you want
to drag on a Remote Process Group (the 5th icon in the toolbar).

It will ask you for the URL of the instance to send to. This is the URL
of the second instance's UI - so for instance localhost:8080/nifi

Then, you can create a connection from GetFile to that
RemoteProcessGroup. It will ask which port to send data to.

If you drag the connection and it says that there are no input ports
available, you may need to right-click and choose "Refresh flow" and
wait a few seconds for it to refresh the remote instance's contents.
Then try creating the connection again.

Let us know if you still have problems.

Thanks!
-Mark


------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/28/2015 10:11:30 AM
Subject: configuring site-to-site

>Hi,
>    I am trying to perform a site to site configuration.
>
>I am trying to do a GetFile at one site and do a PutFile in another.
>
>What I did:
>Configured NiFi in machine A and set the socket input port number and
>set
>secure to false in nifi.properties
>  - Ran the first NiFi instance in machine A
>  - Setup a GetFile with a input dir on machine A
>  - Connected it to an output port (named it XYZ)
>
>Configured NiFi in machine B and set the socket input port number to a
>different number and set secure to false in nifi.properties
>  - Ran the second NiFi instance in machine B
>  - Setup a PutFile processor with the local dir of machine B
>  - Connected it to an input port (named it same, XYZ )
>
>When I run both the nifi instances I do not see the files getting
>transferred. What am i missing?
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Corey Flowers
Also,

        When you do this, you need to right click and select "remote
ports". You will see a toggle switch (on/off) beside each input port on the
remote system. Make sure this is set to on.

Thanks!
Corey



On Tue, Apr 28, 2015 at 10:36 AM, Mark Payne <[hidden email]> wrote:

> Anup,
>
> Site-to-site doesn't work by having a port with the same name on both
> machines. There have been a few people try that, so I guess that's a fairly
> common misconception. Sorry about that!
>
> On Machine A, you don't want to setup an Output Port. Instead, you want to
> drag on a Remote Process Group (the 5th icon in the toolbar).
>
> It will ask you for the URL of the instance to send to. This is the URL of
> the second instance's UI - so for instance localhost:8080/nifi
>
> Then, you can create a connection from GetFile to that RemoteProcessGroup.
> It will ask which port to send data to.
>
> If you drag the connection and it says that there are no input ports
> available, you may need to right-click and choose "Refresh flow" and wait a
> few seconds for it to refresh the remote instance's contents. Then try
> creating the connection again.
>
> Let us know if you still have problems.
>
> Thanks!
> -Mark
>
>
>
> ------ Original Message ------
> From: "anup s" <[hidden email]>
> To: [hidden email]
> Sent: 4/28/2015 10:11:30 AM
> Subject: configuring site-to-site
>
>  Hi,
>>    I am trying to perform a site to site configuration.
>>
>> I am trying to do a GetFile at one site and do a PutFile in another.
>>
>> What I did:
>> Configured NiFi in machine A and set the socket input port number and set
>> secure to false in nifi.properties
>>  - Ran the first NiFi instance in machine A
>>  - Setup a GetFile with a input dir on machine A
>>  - Connected it to an output port (named it XYZ)
>>
>> Configured NiFi in machine B and set the socket input port number to a
>> different number and set secure to false in nifi.properties
>>  - Ran the second NiFi instance in machine B
>>  - Setup a PutFile processor with the local dir of machine B
>>  - Connected it to an input port (named it same, XYZ )
>>
>> When I run both the nifi instances I do not see the files getting
>> transferred. What am i missing?
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253.html
>> Sent from the Apache NiFi (incubating) Developer List mailing list
>> archive at Nabble.com.
>>
>


--
Corey Flowers
Vice President, Onyx Point, Inc
(410) 541-6699
[hidden email]

-- This account not approved for unencrypted proprietary information --
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

anup s
In reply to this post by Mark Payne
Thanks Mark, I tried that initially connecting the GetFile to the Remote Process Group. But it throws me an error "NiFi Flow does not have any input ports',
I configured the input port, but it doesn't show up any entries when I right - click on the RPG.

Also, I am not able to find the "Refresh Flow" that you mentioned..
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Mark Payne
Anup,

Depending on which version of NiFi you are running the "Refresh flow"
may have been renamed to simply "Refresh". It should be the 4th from the
bottom on the context menu. Also please make sure that you used a Remote
Process Group and not a local Process Group - as obvious as that may
sound, I've done it a few times myself and left myself very confused.

Thanks
-Mark


------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/28/2015 10:40:54 AM
Subject: Re: configuring site-to-site

>Thanks Mark, I tried that initially connecting the GetFile to the
>Remote
>Process Group. But it throws me an error "NiFi Flow does not have any
>input
>ports',
>I configured the input port, but it doesn't show up any entries when I
>right
>- click on the RPG.
>
>Also, I am not able to find the "Refresh Flow" that you mentioned..
>
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1262.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

anup s
In reply to this post by Corey Flowers
Hi Corey,
   I don't see any input ports been shown in Remote Process Group Ports :(

 
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Mark Payne
Anup,

If you right-click on the RPG and click "Go to" does it take you to the
other instance?

------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/28/2015 10:49:09 AM
Subject: Re: configuring site-to-site

>Hi Corey,
>    I don't see any input ports been shown in Remote Process Group Ports
>:(
>
>
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1265.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

RE: configuring site-to-site

anup s

Mark,

  Yes, That works. But I don’t see the ports appearing in RPG ports.

 

From: Mark Payne [via Apache NiFi (incubating) Developer List] [mailto:ml-node+[hidden email]]
Sent: Tuesday, April 28, 2015 8:23 PM
To: Sethuram, Anup
Subject: Re: configuring site-to-site

 

Anup,

If you right-click on the RPG and click "Go to" does it take you to the
other instance?

------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/28/2015 10:49:09 AM
Subject: Re: configuring site-to-site


>Hi Corey,
>    I don't see any input ports been shown in Remote Process Group Ports
>:(
>
>
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1265.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.

 


If you reply to this email, your message will be added to the discussion below:

http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1266.html

To unsubscribe from configuring site-to-site, click here.
NAML



The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Mark Payne
OK. So a couple of more questions then:

Is the NiFi instance that you're trying to connect to secure (i.e., http
or https)? If you are running https then you will need to allow the
other instance to access the port. Will go into that a bit more if
that's the case. If http, then it's not an issue.

The Input Port that you created on the instance that you're trying to
send to - did you create it on the root group, or did you create it
within a sub-group?

Were you able to find the 'refresh' button in the context menu?



------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/28/2015 10:55:29 AM
Subject: RE: configuring site-to-site

>Mark,
>   Yes, That works. But I don't see the ports appearing in RPG ports.
>
>From: Mark Payne [via Apache NiFi (incubating) Developer List]
>[mailto:[hidden email]]
>Sent: Tuesday, April 28, 2015 8:23 PM
>To: Sethuram, Anup
>Subject: Re: configuring site-to-site
>
>Anup,
>
>If you right-click on the RPG and click "Go to" does it take you to the
>other instance?
>
>------ Original Message ------
>From: "anup s" <[hidden
>email]</user/SendEmail.jtp?type=node&node=1266&i=0>>
>To: [hidden email]</user/SendEmail.jtp?type=node&node=1266&i=1>
>Sent: 4/28/2015 10:49:09 AM
>Subject: Re: configuring site-to-site
>
>>Hi Corey,
>>     I don't see any input ports been shown in Remote Process Group
>>Ports
>>:(
>>
>>
>>
>>
>>
>>--
>>View this message in context:
>>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1265.html
>>Sent from the Apache NiFi (incubating) Developer List mailing list
>>archive at Nabble.com.
>
>________________________________
>If you reply to this email, your message will be added to the
>discussion below:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1266.html
>To unsubscribe from configuring site-to-site, click
>here<
>NAML<
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>________________________________
>The information contained in this message may be confidential and
>legally protected under applicable law. The message is intended solely
>for the addressee(s). If you are not the intended recipient, you are
>hereby notified that any use, forwarding, dissemination, or
>reproduction of this message is strictly prohibited and may be
>unlawful. If you are not the intended recipient, please contact the
>sender by return e-mail and destroy all copies of the original message.
>
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1267.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

RE: configuring site-to-site

anup s

Its http, but want to move to https. Actually, the second machine I had mentioned is configured as a cluster and I am providing enabling the secure ports In the NCM.

 

I tried configuring another nifi-instance in a non-clustered mode and called the RPG group. This time it worked. J

 

I probably need to set the input ports for cluster mode and that should work. Please let me know if there are additional things to do.

Also, it would be great if you could let me know the process to make it secure.

 

Thanks again.

 

From: Mark Payne [via Apache NiFi (incubating) Developer List] [mailto:ml-node+[hidden email]]
Sent: Tuesday, April 28, 2015 8:31 PM
To: Sethuram, Anup
Subject: Re: configuring site-to-site

 

OK. So a couple of more questions then:

Is the NiFi instance that you're trying to connect to secure (i.e., http
or https)? If you are running https then you will need to allow the
other instance to access the port. Will go into that a bit more if
that's the case. If http, then it's not an issue.

The Input Port that you created on the instance that you're trying to
send to - did you create it on the root group, or did you create it
within a sub-group?

Were you able to find the 'refresh' button in the context menu?



------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/28/2015 10:55:29 AM
Subject: RE: configuring site-to-site


>Mark,
>   Yes, That works. But I don't see the ports appearing in RPG ports.
>
>From: Mark Payne [via Apache NiFi (incubating) Developer List]
>[mailto:[hidden email]]
>Sent: Tuesday, April 28, 2015 8:23 PM
>To: Sethuram, Anup
>Subject: Re: configuring site-to-site
>
>Anup,
>
>If you right-click on the RPG and click "Go to" does it take you to the
>other instance?
>
>------ Original Message ------
>From: "anup s" <[hidden
>email]</user/SendEmail.jtp?type=node&node=1266&i=0>>
>To: [hidden email]</user/SendEmail.jtp?type=node&node=1266&i=1>
>Sent: 4/28/2015 10:49:09 AM
>Subject: Re: configuring site-to-site
>
>>Hi Corey,
>>     I don't see any input ports been shown in Remote Process Group
>>Ports
>>:(
>>
>>
>>
>>
>>
>>--
>>View this message in context:
>>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1265.html
>>Sent from the Apache NiFi (incubating) Developer List mailing list
>>archive at Nabble.com.
>
>________________________________
>If you reply to this email, your message will be added to the
>discussion below:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1266.html
>To unsubscribe from configuring site-to-site, click
>here<
>NAML<
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>________________________________
>The information contained in this message may be confidential and
>legally protected under applicable law. The message is intended solely
>for the addressee(s). If you are not the intended recipient, you are
>hereby notified that any use, forwarding, dissemination, or
>reproduction of this message is strictly prohibited and may be
>unlawful. If you are not the intended recipient, please contact the
>sender by return e-mail and destroy all copies of the original message.
>
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1267.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.

 


If you reply to this email, your message will be added to the discussion below:

http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1268.html

To unsubscribe from configuring site-to-site, click here.
NAML



The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Mark Payne
Anup,

So whether you are sending to a cluster or a standalone NiFi it should
work exactly the same way. But it is important that your cluster manager
also has the "remote.input.socket.port" property set. Otherwise it
should all work the same.

If you want to run secure, I'd recommend you go through the Admin Guide
to run securely. You can get to that in the help menu of the app.

Beyond that, whenever you try to connect to the secure instance via
site-to-site, it will automatically request access to the flow. Then
you'll go to the Users page and give the requestor the "NiFi" role. Then
you can right-click and configure a port and go to "Access Controls" and
specify which users can interact with that port.

Thanks
-Mark

------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/28/2015 11:25:37 AM
Subject: RE: configuring site-to-site

>Its http, but want to move to https. Actually, the second machine I had
>mentioned is configured as a cluster and I am providing enabling the
>secure ports In the NCM.
>
>I tried configuring another nifi-instance in a non-clustered mode and
>called the RPG group. This time it worked. :)
>
>I probably need to set the input ports for cluster mode and that should
>work. Please let me know if there are additional things to do.
>Also, it would be great if you could let me know the process to make it
>secure.
>
>Thanks again.
>
>From: Mark Payne [via Apache NiFi (incubating) Developer List]
>[mailto:[hidden email]]
>Sent: Tuesday, April 28, 2015 8:31 PM
>To: Sethuram, Anup
>Subject: Re: configuring site-to-site
>
>OK. So a couple of more questions then:
>
>Is the NiFi instance that you're trying to connect to secure (i.e.,
>http
>or https)? If you are running https then you will need to allow the
>other instance to access the port. Will go into that a bit more if
>that's the case. If http, then it's not an issue.
>
>The Input Port that you created on the instance that you're trying to
>send to - did you create it on the root group, or did you create it
>within a sub-group?
>
>Were you able to find the 'refresh' button in the context menu?
>
>
>
>------ Original Message ------
>From: "anup s" <[hidden
>email]</user/SendEmail.jtp?type=node&node=1268&i=0>>
>To: [hidden email]</user/SendEmail.jtp?type=node&node=1268&i=1>
>Sent: 4/28/2015 10:55:29 AM
>Subject: RE: configuring site-to-site
>
>>Mark,
>>    Yes, That works. But I don't see the ports appearing in RPG ports.
>>
>>From: Mark Payne [via Apache NiFi (incubating) Developer List]
>>[mailto:[hidden email]</user/SendEmail.jtp?type=node&node=1268&i=2>]
>>Sent: Tuesday, April 28, 2015 8:23 PM
>>To: Sethuram, Anup
>>Subject: Re: configuring site-to-site
>>
>>Anup,
>>
>>If you right-click on the RPG and click "Go to" does it take you to
>>the
>>other instance?
>>
>>------ Original Message ------
>>From: "anup s" <[hidden
>>email]</user/SendEmail.jtp?type=node&node=1266&i=0>>
>>To: [hidden email]</user/SendEmail.jtp?type=node&node=1266&i=1>
>>Sent: 4/28/2015 10:49:09 AM
>>Subject: Re: configuring site-to-site
>>
>>>Hi Corey,
>>>      I don't see any input ports been shown in Remote Process Group
>>>Ports
>>>:(
>>>
>>>
>>>
>>>
>>>
>>>--
>>>View this message in context:
>>>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1265.html
>>>Sent from the Apache NiFi (incubating) Developer List mailing list
>>>archive at Nabble.com.
>>
>>________________________________
>>If you reply to this email, your message will be added to the
>>discussion below:
>>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1266.html
>>To unsubscribe from configuring site-to-site, click
>>here<
>>NAML<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>>________________________________
>>The information contained in this message may be confidential and
>>legally protected under applicable law. The message is intended solely
>>for the addressee(s). If you are not the intended recipient, you are
>>hereby notified that any use, forwarding, dissemination, or
>>reproduction of this message is strictly prohibited and may be
>>unlawful. If you are not the intended recipient, please contact the
>>sender by return e-mail and destroy all copies of the original
>>message.
>>
>>
>>
>>
>>--
>>View this message in context:
>>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1267.html
>>Sent from the Apache NiFi (incubating) Developer List mailing list
>>archive at Nabble.com.
>
>________________________________
>If you reply to this email, your message will be added to the
>discussion below:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1268.html
>To unsubscribe from configuring site-to-site, click
>here<
>NAML<
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
>________________________________
>The information contained in this message may be confidential and
>legally protected under applicable law. The message is intended solely
>for the addressee(s). If you are not the intended recipient, you are
>hereby notified that any use, forwarding, dissemination, or
>reproduction of this message is strictly prohibited and may be
>unlawful. If you are not the intended recipient, please contact the
>sender by return e-mail and destroy all copies of the original message.
>
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1271.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Joe Witt
This thread, others before, and some of my own experiences lead me to
believe we need to make site-to-site status information more obvious.
It isn't easy to know whether site to site is configured for a given
node/cluster and whether it is actually working and whether it is
secure.  Perhaps we should make a JIRA to add a status page showing
the status of key items like this so users can get a nice 'Green
checkbox' that it is on and good or a 'red x' that is off or not
configured.

On Tue, Apr 28, 2015 at 11:58 AM, Mark Payne <[hidden email]> wrote:

> Anup,
>
> So whether you are sending to a cluster or a standalone NiFi it should work
> exactly the same way. But it is important that your cluster manager also has
> the "remote.input.socket.port" property set. Otherwise it should all work
> the same.
>
> If you want to run secure, I'd recommend you go through the Admin Guide to
> run securely. You can get to that in the help menu of the app.
>
> Beyond that, whenever you try to connect to the secure instance via
> site-to-site, it will automatically request access to the flow. Then you'll
> go to the Users page and give the requestor the "NiFi" role. Then you can
> right-click and configure a port and go to "Access Controls" and specify
> which users can interact with that port.
>
> Thanks
> -Mark
>
> ------ Original Message ------
> From: "anup s" <[hidden email]>
> To: [hidden email]
> Sent: 4/28/2015 11:25:37 AM
> Subject: RE: configuring site-to-site
>
>> Its http, but want to move to https. Actually, the second machine I had
>> mentioned is configured as a cluster and I am providing enabling the secure
>> ports In the NCM.
>>
>> I tried configuring another nifi-instance in a non-clustered mode and
>> called the RPG group. This time it worked. :)
>>
>> I probably need to set the input ports for cluster mode and that should
>> work. Please let me know if there are additional things to do.
>> Also, it would be great if you could let me know the process to make it
>> secure.
>>
>> Thanks again.
>>
>> From: Mark Payne [via Apache NiFi (incubating) Developer List]
>> [mailto:[hidden email]]
>> Sent: Tuesday, April 28, 2015 8:31 PM
>> To: Sethuram, Anup
>> Subject: Re: configuring site-to-site
>>
>> OK. So a couple of more questions then:
>>
>> Is the NiFi instance that you're trying to connect to secure (i.e., http
>> or https)? If you are running https then you will need to allow the
>> other instance to access the port. Will go into that a bit more if
>> that's the case. If http, then it's not an issue.
>>
>> The Input Port that you created on the instance that you're trying to
>> send to - did you create it on the root group, or did you create it
>> within a sub-group?
>>
>> Were you able to find the 'refresh' button in the context menu?
>>
>>
>>
>> ------ Original Message ------
>> From: "anup s" <[hidden
>> email]</user/SendEmail.jtp?type=node&node=1268&i=0>>
>> To: [hidden email]</user/SendEmail.jtp?type=node&node=1268&i=1>
>> Sent: 4/28/2015 10:55:29 AM
>> Subject: RE: configuring site-to-site
>>
>>> Mark,
>>>    Yes, That works. But I don't see the ports appearing in RPG ports.
>>>
>>> From: Mark Payne [via Apache NiFi (incubating) Developer List]
>>> [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=1268&i=2>]
>>> Sent: Tuesday, April 28, 2015 8:23 PM
>>> To: Sethuram, Anup
>>> Subject: Re: configuring site-to-site
>>>
>>> Anup,
>>>
>>> If you right-click on the RPG and click "Go to" does it take you to the
>>> other instance?
>>>
>>> ------ Original Message ------
>>> From: "anup s" <[hidden
>>> email]</user/SendEmail.jtp?type=node&node=1266&i=0>>
>>> To: [hidden email]</user/SendEmail.jtp?type=node&node=1266&i=1>
>>> Sent: 4/28/2015 10:49:09 AM
>>> Subject: Re: configuring site-to-site
>>>
>>>> Hi Corey,
>>>>      I don't see any input ports been shown in Remote Process Group
>>>> Ports
>>>> :(
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>>
>>>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1265.html
>>>> Sent from the Apache NiFi (incubating) Developer List mailing list
>>>> archive at Nabble.com.
>>>
>>>
>>> ________________________________
>>> If you reply to this email, your message will be added to the
>>> discussion below:
>>>
>>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1266.html
>>> To unsubscribe from configuring site-to-site, click
>>> here<
>>>
>>> NAML<http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>>
>>> ________________________________
>>> The information contained in this message may be confidential and
>>> legally protected under applicable law. The message is intended solely
>>> for the addressee(s). If you are not the intended recipient, you are
>>> hereby notified that any use, forwarding, dissemination, or
>>> reproduction of this message is strictly prohibited and may be
>>> unlawful. If you are not the intended recipient, please contact the
>>> sender by return e-mail and destroy all copies of the original message.
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>>
>>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1267.html
>>> Sent from the Apache NiFi (incubating) Developer List mailing list
>>> archive at Nabble.com.
>>
>>
>> ________________________________
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1268.html
>> To unsubscribe from configuring site-to-site, click
>> here<
>>
>> NAML<
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>> ________________________________
>> The information contained in this message may be confidential and legally
>> protected under applicable law. The message is intended solely for the
>> addressee(s). If you are not the intended recipient, you are hereby notified
>> that any use, forwarding, dissemination, or reproduction of this message is
>> strictly prohibited and may be unlawful. If you are not the intended
>> recipient, please contact the sender by return e-mail and destroy all copies
>> of the original message.
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1271.html
>> Sent from the Apache NiFi (incubating) Developer List mailing list archive
>> at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

anup s
In reply to this post by Mark Payne
Hi Mark,
    The Secure Configuration section isn't updated yet. Could you point me to some other location.

Regards,
anup
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Matt Gilman
Anup,

I replied earlier today to your email on the users mailing list but it
doesn't seem to have gone through correctly. I am including that response
below...

That section is still incomplete unfortunately. We are definitely pushing
the documentation at the moment. Personally, I am working through getting
our REST endpoints documented. I know another committer has been working on
the contribution guide as well some introduction to NiFi quick start
guides. I can provide some quick points here in the meantime.

In the section for web properties you'll want to configure the 'https'
properties instead of the 'http' properties.

nifi.web.http.host=
nifi.web.http.port=
nifi.web.https.host=
nifi.web.https.port=

The further down you'll need to configure the security properties.

nifi.security.keystore=
nifi.security.keystoreType=
nifi.security.keystorePasswd=
nifi.security.keyPasswd=
nifi.security.truststore=
nifi.security.truststoreType=
nifi.security.truststorePasswd=
nifi.security.needClientAuth=

These will define the certificates that are used by the web server (and
cluster and site to site communications). You will need to configure all
the keystore properties and truststore properties (if keyPasswd is not
configured the keystorePasswd will be tried as the keyPasswd). If you set
needClientAuth to false, clients will be required to trust the keystore
configured here. User access will still be anonymous however. If you set
needClientAuth to true, clients will need to have certificates loaded in
their browser that are trusted by the truststore configured here. User
access will be considered using the DN from their certificate and the
authorization provider.

NiFi supports pluggable authorization which is only necessary if
needClientAuth is set to true. By default its configured with a file based
solution.

nifi.security.user.authority.provider=file-provider

Details on setting up this file and controlling the level of access have
started being discussed here [1].

Hope this helps while we get more detailed documentation written up. Thanks.

Matt

[1]
https://nifi.incubator.apache.org/docs/nifi-docs/administration-guide.html#controlling-levels-of-access

On Wed, Apr 29, 2015 at 8:18 AM, anup s <[hidden email]> wrote:

> Hi Mark,
>     The Secure Configuration section isn't updated yet. Could you point me
> to some other location.
>
> Regards,
> anup
>
>
>
> --
> View this message in context:
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1280.html
> Sent from the Apache NiFi (incubating) Developer List mailing list archive
> at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Mark Payne
In reply to this post by anup s
Anup,

Sorry, I didn't realize that hasn't been put up yet. I'll tr to get that
up soon.

Thanks
-Mark

------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/29/2015 8:18:41 AM
Subject: Re: configuring site-to-site

>Hi Mark,
>     The Secure Configuration section isn't updated yet. Could you point
>me
>to some other location.
>
>Regards,
>anup
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p1280.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

RE: configuring site-to-site

anup s
Thanks Mark

-----Original Message-----
From: Mark Payne [mailto:[hidden email]]
Sent: Friday, May 01, 2015 12:25 AM
To: [hidden email]
Subject: Re: configuring site-to-site

Anup,

Sorry, I didn't realize that hasn't been put up yet. I'll tr to get that up soon.

Thanks
-Mark

------ Original Message ------
From: "anup s" <[hidden email]>
To: [hidden email]
Sent: 4/29/2015 8:18:41 AM
Subject: Re: configuring site-to-site

>Hi Mark,
>     The Secure Configuration section isn't updated yet. Could you
>point me to some other location.
>
>Regards,
>anup
>
>
>
>--
>View this message in context:
>http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/config
>uring-site-to-site-tp1253p1280.html
>Sent from the Apache NiFi (incubating) Developer List mailing list
>archive at Nabble.com.

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Heping Shang
In reply to this post by anup s
I use the latest NiFi (downloaded yesterday)
I have one NiFi on PC and one on Solaris
I want to send data from my PC to Solaris

On PC, I created a GenerateFlowFile and a remote group to Solaris (I use http, not https)
On Solaris, I created an inputPort and a LogAttribute to sink data.

The problem I have is, on my PC, the the remote group port has nothing. None for the PC and none for the Solaris. The remote connection by itself did not complain anything. I assume it is okay.

Why cannot I see any remote ports?

Thank you for your helps.

             ------- Heping Shang.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Mark Payne
Heping,

On the Remote Process Group, in the lower-right-hand corner, does it indicate the time when it was last updated,
or does it say that the remote flow is not current? If the remote flow is not current (or if it's been a long time since
the last update), you can try Right-clicking on the Remote Process Group and choosing Refresh from the context
menu. It will generally take about a minute to refresh.

It's also important to ensure that on the remote node that the "remote.socket.input.port" property is set in the
conf/nifi.properties file.

I would also recommend you look into the logs/nifi-app.log file on the sending system and see if there are any log
messages about it.

Thanks
-Mark


> On Oct 22, 2015, at 1:04 PM, Heping Shang <[hidden email]> wrote:
>
> I use the latest NiFi (downloaded yesterday)
> I have one NiFi on PC and one on Solaris
> I want to send data from my PC to Solaris
>
> On PC, I created a GenerateFlowFile and a remote group to Solaris (I use
> http, not https)
> On Solaris, I created an inputPort and a LogAttribute to sink data.
>
> The problem I have is, on my PC, the the remote group port has nothing. None
> for the PC and none for the Solaris. The remote connection by itself did not
> complain anything. I assume it is okay.
>
> Why cannot I see any remote ports?
>
> Thank you for your helps.
>
>             ------- Heping Shang.
>
>
>
> --
> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p3240.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Heping Shang
Thank you Mark.

I did refresh the remote process group multiple times. It shows the latest time.

remote.socket.input.port in my conf/nifi.properties is empty. What value I must set? On both machines? I thought it will be default to 8080. Do I get something wrong?

Last few entries in logs/nifi-app.log are:
2015-10-22 13:23:51,260 INFO [pool-20-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2015-10-22 13:23:54,105 INFO [pool-20-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@52c46334 checkpointed with 2 Records and 0 Swap Files in 2845 milliseconds (Stop-the-world time = 1160 milliseconds, Clear Edit Logs time = 1652 millis), max Transaction ID 1485147
2015-10-22 13:23:54,105 INFO [pool-20-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 2845 milliseconds
2015-10-22 13:25:54,101 INFO [pool-20-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile Repository
2015-10-22 13:25:56,797 INFO [pool-20-thread-1] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@52c46334 checkpointed with 2 Records and 0 Swap Files in 2696 milliseconds (Stop-the-world time = 1170 milliseconds, Clear Edit Logs time = 1504 millis), max Transaction ID 1485147
2015-10-22 13:25:56,797 INFO [pool-20-thread-1] o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile Repository with 2 records in 2696 milliseconds

Anything else should I do?
Thank you.

                ----------- Heping Shang.
Reply | Threaded
Open this post in threaded view
|

Re: configuring site-to-site

Matthew Clarke
You must supply an available unused port for remote.socket.input.port. you
will need it set in both machine is you intent on sending data in both
directions. Make sure these ports are open in your firewalls.

Thanks,
Matt
On Oct 22, 2015 4:33 PM, "Heping Shang" <[hidden email]> wrote:

> Thank you Mark.
>
> I did refresh the remote process group multiple times. It shows the latest
> time.
>
> remote.socket.input.port in my conf/nifi.properties is empty. What value I
> must set? On both machines? I thought it will be default to 8080. Do I get
> something wrong?
>
> Last few entries in logs/nifi-app.log are:
> 2015-10-22 13:23:51,260 INFO [pool-20-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
> Repository
> 2015-10-22 13:23:54,105 INFO [pool-20-thread-1]
> org.wali.MinimalLockingWriteAheadLog
> org.wali.MinimalLockingWriteAheadLog@52c46334 checkpointed with 2 Records
> and 0 Swap Files in 2845 milliseconds (Stop-the-world time = 1160
> milliseconds, Clear Edit Logs time = 1652 millis), max Transaction ID
> 1485147
> 2015-10-22 13:23:54,105 INFO [pool-20-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
> Repository with 2 records in 2845 milliseconds
> 2015-10-22 13:25:54,101 INFO [pool-20-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Initiating checkpoint of FlowFile
> Repository
> 2015-10-22 13:25:56,797 INFO [pool-20-thread-1]
> org.wali.MinimalLockingWriteAheadLog
> org.wali.MinimalLockingWriteAheadLog@52c46334 checkpointed with 2 Records
> and 0 Swap Files in 2696 milliseconds (Stop-the-world time = 1170
> milliseconds, Clear Edit Logs time = 1504 millis), max Transaction ID
> 1485147
> 2015-10-22 13:25:56,797 INFO [pool-20-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
> Repository with 2 records in 2696 milliseconds
>
> Anything else should I do?
> Thank you.
>
>                 ----------- Heping Shang.
>
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/configuring-site-to-site-tp1253p3245.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
12