Installing NIFI through ambari.

classic Classic list List threaded Threaded
7 messages Options
pd
Reply | Threaded
Open this post in threaded view
|

Installing NIFI through ambari.

pd
I'm trying to install NIFI on active hadoop cluster (2.3.2) through Ambari (2.1.2). When I go thorough the installation screen, I have an option to selct master node, but once I select that it moves to the "Customize Services" screen skipping the "Assign Slaves and Clients" selection screen
Does anyone know the fix/workaround?

Ambari node:
/var/lib/ambari-server/resources/stacks/HDP/2.3/services/NIFI
root@g1:NIFI> ls -ltr
total 72
-rwxr-xr-x 1 root 14940 Feb  6 14:24 widgets.json*
-rwxr-xr-x 1 root 17908 Feb  6 14:24 README.md*
-rwxr-xr-x 1 root  2141 Feb  6 14:24 nifi-bootstrap.xml*
-rwxr-xr-x 1 root  4249 Feb  6 14:24 metrics.json*
-rwxr-xr-x 1 root  1834 Feb  6 14:24 metainfo.xml*
-rwxr-xr-x 1 root   221 Feb  6 14:24 kerberos.json*
drwxr-xr-x 3 root  4096 Feb  6 14:24 package/
drwxr-xr-x 2 root  4096 Feb  6 14:24 screenshots/
drwxr-xr-x 2 root  4096 Feb  6 14:24 demofiles/
drwxr-xr-x 2 root  4096 Feb  7 08:22 configuration/


nifi-install-screen
Reply | Threaded
Open this post in threaded view
|

Re: Installing NIFI through ambari.

Bryan Bende
Hello,

The Apache NiFi community only provides an example service for viewing
metrics in Ambari [1]. This example does not actually install and manage
NiFi through Ambari, it only acts as placeholder to view  metrics.

-Bryan

[1] https://cwiki.apache.org/confluence/display/NIFI/Ambari+Metrics




On Sun, Feb 7, 2016 at 9:03 AM, pd <[hidden email]> wrote:

> I'm trying to install NIFI on active hadoop cluster (2.3.2) through Ambari
> (2.1.2). When I go thorough the installation screen, I have an option to
> selct master node, but once I select that it moves to the "Customize
> Services" screen skipping the "Assign Slaves and Clients" selection screen
> Does anyone know the fix/workaround?
>
> Ambari node:
> /var/lib/ambari-server/resources/stacks/HDP/2.3/services/NIFI
> root@g1:NIFI> ls -ltr
> total 72
> -rwxr-xr-x 1 root 14940 Feb  6 14:24 widgets.json*
> -rwxr-xr-x 1 root 17908 Feb  6 14:24 README.md*
> -rwxr-xr-x 1 root  2141 Feb  6 14:24 nifi-bootstrap.xml*
> -rwxr-xr-x 1 root  4249 Feb  6 14:24 metrics.json*
> -rwxr-xr-x 1 root  1834 Feb  6 14:24 metainfo.xml*
> -rwxr-xr-x 1 root   221 Feb  6 14:24 kerberos.json*
> drwxr-xr-x 3 root  4096 Feb  6 14:24 package/
> drwxr-xr-x 2 root  4096 Feb  6 14:24 screenshots/
> drwxr-xr-x 2 root  4096 Feb  6 14:24 demofiles/
> drwxr-xr-x 2 root  4096 Feb  7 08:22 configuration/
>
>
> <
> http://apache-nifi-developer-list.39713.n7.nabble.com/file/n7194/nifi-issue-screen.png
> >
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Installing-NIFI-through-ambari-tp7194.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: Installing NIFI through ambari.

shweta
If we enable the Ambari Reporting Tasks through Nifi  we should be able to see the metrics on Ambari. Guys please feel free to correct me if I'm wrong, since I haven't tried this before.

Thanks,
Shweta
Reply | Threaded
Open this post in threaded view
|

Re: Installing NIFI through ambari.

Bryan Bende
That is correct.

-Bryan

On Sun, Feb 7, 2016 at 11:13 AM, shweta <[hidden email]> wrote:

> If we enable the Ambari Reporting Tasks through Nifi  we should be able to
> see the metrics on Ambari. Guys please feel free to correct me if I'm
> wrong,
> since I haven't tried this before.
>
> Thanks,
> Shweta
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Installing-NIFI-through-ambari-tp7194p7196.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
pd
Reply | Threaded
Open this post in threaded view
|

Re: Installing NIFI through ambari.

pd
Here is what I've done so far.
1) I've setup a Master node and 3 slave nodes. NiFi url is working and I can see 3 connected nodes.

2) based on the URL I've configured Ambari Metrics. But when I try to start NiFi server (step 5) it fails (see attached log file and screenshot).
But now I've 2 nifi dir under /opt on NCM (nifi-0.4.1 & nifi-1.1.1.0-12/).
https://cwiki.apache.org/confluence/display/NIFI/Ambari+Metrics

3) On Nifi, I've configured "Configured Reporting Task" to point to Metrics Collector server.


nifi-service-failed-ambari

nifi-app.log
Reply | Threaded
Open this post in threaded view
|

Re: Installing NIFI through ambari.

Bryan Bende
It may have something to do with the APP_ID in the metrics being
'nificluster'. It needs to line up with the APP_ID in the service. I would
recommend leaving it as the default value and see if that works.

On Friday, February 26, 2016, davi <[hidden email]> wrote:

> I'm have similar situation . I have installed a NIFI cluster (1 NCM /3
> Nodes)with ambari .NIFI is running fine and data are sent to ambari metrics
> collector . I verified with the phoenix query , but still not showing on
> the
> ambari web page: Result of Phoenix query :   SELECT
> METRIC_NAME,SERVER_TIME,APP_ID FROM METRIC_RECORD WHERE
> APP_ID='nificluster'
> ORDER BY SERVER_TIME LIMIT
>
> 20;+------------------------------------------+-----------------------------------+|
> METRIC_NAME                |               SERVER_TIME
>
> |+------------------------------------------+-----------------------------------+|
> jvm.daemon_thread_count                  | 1456484338751
> || jvm.gc.runs.PSMarkSweep                  | 1456484338751
> || jvm.gc.runs.PSScavenge                   | 1456484338751
> || jvm.heap_usage                           | 1456484338751
> || jvm.gc.time.PSMarkSweep                  | 1456484338751
> || jvm.gc.time.PSScavenge                   | 1456484338751
> || jvm.heap_used                            | 1456484338751
> || jvm.uptime                               | 1456484338751
> || jvm.thread_states.blocked                | 1456484338751
> || jvm.thread_states.runnable               | 1456484338751
> || jvm.file_descriptor_usage                | 1456484338751
> || jvm.thread_states.timed_waiting          | 1456484338751
> || jvm.thread_states.terminated             | 1456484338751
> || jvm.non_heap_usage                       | 1456484338751
> || jvm.thread_count                         | 1456484338751
> || ActiveThreads                            | 1456484338751
> || BytesReadLast5Minutes                    | 1456484338751
> || BytesReceivedLast5Minutes                | 1456484338751
> || FlowFilesQueued                          | 1456484338751
> || BytesSentLast5Minutes                    | 1456484338751
>
> |+------------------------------------------+-----------------------------------+and
> ambari view :
> <
> http://apache-nifi-developer-list.39713.n7.nabble.com/file/n7644/nifiservice.png
> >
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/Installing-NIFI-through-ambari-tp7194p7644.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.



--
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: Installing NIFI through ambari.

davi
I did the deployment with the standard name with not more success. I did also change with upper and lower cases ,without any success.