Need Information - Apache NiFi

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Need Information - Apache NiFi

Mothi86
Hi Apache NiFi team,

Greetings. I did my hands on experience with multiple tutorials around
sentimental analysis integrating social media like twitter and facebook and
also log analysis. Could you help me understand in what features Apache
NiFi would act as an alternative to Splunk.

I do have couple of technical clarifications in Apache NiFi - would it be
possible to have a initial technical call to clarify my questions.

Warm Regards,
M. Mothilal
Cell : +1(647)641-0910.
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Joe Witt
Hello,

I think as a community we're all quite happy to help address questions
or ideas you might have for Apache NiFi.  Please feel free to ask your
questions here and if they're more from the perspective of a user
please use the [hidden email] list.

For the technical questions it is probably best to align them with
some portion of the overview, administration, user, best practice, or
developer guide so we can use that as a basis for further discussion.
You can find those here: https://nifi.apache.org/docs.html

Thanks
Joe

On Tue, Nov 8, 2016 at 1:25 PM, Mothilal M <[hidden email]> wrote:

> Hi Apache NiFi team,
>
> Greetings. I did my hands on experience with multiple tutorials around
> sentimental analysis integrating social media like twitter and facebook and
> also log analysis. Could you help me understand in what features Apache
> NiFi would act as an alternative to Splunk.
>
> I do have couple of technical clarifications in Apache NiFi - would it be
> possible to have a initial technical call to clarify my questions.
>
> Warm Regards,
> M. Mothilal
> Cell : +1(647)641-0910.
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Mothi86
In reply to this post by Mothi86
Greetings Joe,

Glad to connect with you from your reply in Apache NiFi community. I am
integrating NiFi with social media - twitter and Facebook to achieve
analysis but I am struck at a point. Request your assistance in solving the
same.

getTwitter processor - i am not able to get tweets filtered by twitter IDs.
(IDs to follow field value is not working as expected. I did get the
numeric value of twitter IDs and input in the value as comma separated but
no results). Please advice as I look to filter my tweets on particular IDs
and also by particular location by cities like Toronto, calgary.

getHTTP processor for facebook integration works fine but i am unable to
fetch data from my personal facebook ID. I tried accessing the Graph API
and it mentions user permissions are not granted but I did provide the
essential permissions in app settings. Please advice if you could help out
in setting the facebook app permissions to ensure I can fetch user_status,
user_location fields. On success, I would need to venture out few options
further.

General question - How could Apache NiFi be useful in replacing Splunk. I
understand most of the current companies have a paid version of Splunk and
NiFi is an open-source but please let me know features which are key in
replacing Splunk.

I appreciate your time and would be more helpful if you could let me know
your comfortable timing for a technical call discussion. I am available and
reachable on +1(647)641-0910. Look forwardly for your reply.

Warm Regards,
M. Mothilal

------------------------------------------------------------------------------------------

Hello,

I think as a community we're all quite happy to help address questions
or ideas you might have for Apache NiFi.  Please feel free to ask your
questions here and if they're more from the perspective of a user
please use the [hidden email] list.

For the technical questions it is probably best to align them with
some portion of the overview, administration, user, best practice, or
developer guide so we can use that as a basis for further discussion.
You can find those here: https://nifi.apache.org/docs.html

Thanks
Joe


---------- Forwarded message ----------
From: Mothilal M <[hidden email]>
Date: Tue, Nov 8, 2016 at 1:25 PM
Subject: Need Information - Apache NiFi
To: [hidden email]


Hi Apache NiFi team,

Greetings. I did my hands on experience with multiple tutorials around
sentimental analysis integrating social media like twitter and facebook and
also log analysis. Could you help me understand in what features Apache
NiFi would act as an alternative to Splunk.

I do have couple of technical clarifications in Apache NiFi - would it be
possible to have a initial technical call to clarify my questions.

Warm Regards,
M. Mothilal
Cell : +1(647)641-0910.
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Joe Witt
Regarding comparison of NiFi and Splunk this is probably not the best
place to make those comparisons or claims and I'm not sure the two are
even in comparable space.  What we can easily do though is help you in
talking through your use case and providing ideas on how Apache NiFi
can play a role in addressing it.

I'm hopeful others more knowledgable with those specific processors
and APIs (twitter, facebook) can assist.

Thanks
Joe

On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <[hidden email]> wrote:

> Greetings Joe,
>
> Glad to connect with you from your reply in Apache NiFi community. I am
> integrating NiFi with social media - twitter and Facebook to achieve
> analysis but I am struck at a point. Request your assistance in solving the
> same.
>
> getTwitter processor - i am not able to get tweets filtered by twitter IDs.
> (IDs to follow field value is not working as expected. I did get the numeric
> value of twitter IDs and input in the value as comma separated but no
> results). Please advice as I look to filter my tweets on particular IDs and
> also by particular location by cities like Toronto, calgary.
>
> getHTTP processor for facebook integration works fine but i am unable to
> fetch data from my personal facebook ID. I tried accessing the Graph API and
> it mentions user permissions are not granted but I did provide the essential
> permissions in app settings. Please advice if you could help out in setting
> the facebook app permissions to ensure I can fetch user_status,
> user_location fields. On success, I would need to venture out few options
> further.
>
> General question - How could Apache NiFi be useful in replacing Splunk. I
> understand most of the current companies have a paid version of Splunk and
> NiFi is an open-source but please let me know features which are key in
> replacing Splunk.
>
> I appreciate your time and would be more helpful if you could let me know
> your comfortable timing for a technical call discussion. I am available and
> reachable on +1(647)641-0910. Look forwardly for your reply.
>
> Warm Regards,
> M. Mothilal
>
> ------------------------------------------------------------------------------------------
>
> Hello,
>
> I think as a community we're all quite happy to help address questions
> or ideas you might have for Apache NiFi.  Please feel free to ask your
> questions here and if they're more from the perspective of a user
> please use the [hidden email] list.
>
> For the technical questions it is probably best to align them with
> some portion of the overview, administration, user, best practice, or
> developer guide so we can use that as a basis for further discussion.
> You can find those here: https://nifi.apache.org/docs.html
>
> Thanks
> Joe
>
>
> ---------- Forwarded message ----------
> From: Mothilal M <[hidden email]>
> Date: Tue, Nov 8, 2016 at 1:25 PM
> Subject: Need Information - Apache NiFi
> To: [hidden email]
>
>
> Hi Apache NiFi team,
>
> Greetings. I did my hands on experience with multiple tutorials around
> sentimental analysis integrating social media like twitter and facebook and
> also log analysis. Could you help me understand in what features Apache NiFi
> would act as an alternative to Splunk.
>
> I do have couple of technical clarifications in Apache NiFi - would it be
> possible to have a initial technical call to clarify my questions.
>
> Warm Regards,
> M. Mothilal
> Cell : +1(647)641-0910.
>
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Mothi86
In reply to this post by Mothi86
Hi Joe,

Greetings and appreciate your reply. Totally agree with your opinion on
comparisons.

Look forward for other developers to comment on suggestions of getTwitter
and getHTTP processor issues for twitter and facebook integration.

Warm Regards,
M. Mothilal

On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <[hidden email]> wrote:

> Greetings Joe,
>
> Glad to connect with you from your reply in Apache NiFi community. I am
> integrating NiFi with social media - twitter and Facebook to achieve
> analysis but I am struck at a point. Request your assistance in solving the
> same.
>
> getTwitter processor - i am not able to get tweets filtered by twitter
> IDs. (IDs to follow field value is not working as expected. I did get the
> numeric value of twitter IDs and input in the value as comma separated but
> no results). Please advice as I look to filter my tweets on particular IDs
> and also by particular location by cities like Toronto, calgary.
>
> getHTTP processor for facebook integration works fine but i am unable to
> fetch data from my personal facebook ID. I tried accessing the Graph API
> and it mentions user permissions are not granted but I did provide the
> essential permissions in app settings. Please advice if you could help out
> in setting the facebook app permissions to ensure I can fetch user_status,
> user_location fields. On success, I would need to venture out few options
> further.
>
> General question - How could Apache NiFi be useful in replacing Splunk. I
> understand most of the current companies have a paid version of Splunk and
> NiFi is an open-source but please let me know features which are key in
> replacing Splunk.
>
> I appreciate your time and would be more helpful if you could let me know
> your comfortable timing for a technical call discussion. I am available and
> reachable on +1(647)641-0910. Look forwardly for your reply.
>
> Warm Regards,
> M. Mothilal
>
> ------------------------------------------------------------
> ------------------------------
>
> Hello,
>
> I think as a community we're all quite happy to help address questions
> or ideas you might have for Apache NiFi.  Please feel free to ask your
> questions here and if they're more from the perspective of a user
> please use the [hidden email] list.
>
> For the technical questions it is probably best to align them with
> some portion of the overview, administration, user, best practice, or
> developer guide so we can use that as a basis for further discussion.
> You can find those here: https://nifi.apache.org/docs.html
>
> Thanks
> Joe
>
>
> ---------- Forwarded message ----------
> From: Mothilal M <[hidden email]>
> Date: Tue, Nov 8, 2016 at 1:25 PM
> Subject: Need Information - Apache NiFi
> To: [hidden email]
>
>
> Hi Apache NiFi team,
>
> Greetings. I did my hands on experience with multiple tutorials around
> sentimental analysis integrating social media like twitter and facebook and
> also log analysis. Could you help me understand in what features Apache
> NiFi would act as an alternative to Splunk.
>
> I do have couple of technical clarifications in Apache NiFi - would it be
> possible to have a initial technical call to clarify my questions.
>
> Warm Regards,
> M. Mothilal
> Cell : +1(647)641-0910.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Bryan Bende
For GetTwitter, I can't really tell if the followers field is supposed to
be the numeric id, or just the twitter username that you would see in
tweets.

I believe it would be this from their API:
https://dev.twitter.com/streaming/overview/request-parameters#follow

It does say that IDs to Follow is only used when using the Filter
end-point, did you try changing the end-point to Filter instead of Sample?

-Bryan


On Tue, Nov 8, 2016 at 2:45 PM, Mothilal M <[hidden email]> wrote:

> Hi Joe,
>
> Greetings and appreciate your reply. Totally agree with your opinion on
> comparisons.
>
> Look forward for other developers to comment on suggestions of getTwitter
> and getHTTP processor issues for twitter and facebook integration.
>
> Warm Regards,
> M. Mothilal
>
> On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <[hidden email]> wrote:
>
> > Greetings Joe,
> >
> > Glad to connect with you from your reply in Apache NiFi community. I am
> > integrating NiFi with social media - twitter and Facebook to achieve
> > analysis but I am struck at a point. Request your assistance in solving
> the
> > same.
> >
> > getTwitter processor - i am not able to get tweets filtered by twitter
> > IDs. (IDs to follow field value is not working as expected. I did get the
> > numeric value of twitter IDs and input in the value as comma separated
> but
> > no results). Please advice as I look to filter my tweets on particular
> IDs
> > and also by particular location by cities like Toronto, calgary.
> >
> > getHTTP processor for facebook integration works fine but i am unable to
> > fetch data from my personal facebook ID. I tried accessing the Graph API
> > and it mentions user permissions are not granted but I did provide the
> > essential permissions in app settings. Please advice if you could help
> out
> > in setting the facebook app permissions to ensure I can fetch
> user_status,
> > user_location fields. On success, I would need to venture out few options
> > further.
> >
> > General question - How could Apache NiFi be useful in replacing Splunk. I
> > understand most of the current companies have a paid version of Splunk
> and
> > NiFi is an open-source but please let me know features which are key in
> > replacing Splunk.
> >
> > I appreciate your time and would be more helpful if you could let me know
> > your comfortable timing for a technical call discussion. I am available
> and
> > reachable on +1(647)641-0910. Look forwardly for your reply.
> >
> > Warm Regards,
> > M. Mothilal
> >
> > ------------------------------------------------------------
> > ------------------------------
> >
> > Hello,
> >
> > I think as a community we're all quite happy to help address questions
> > or ideas you might have for Apache NiFi.  Please feel free to ask your
> > questions here and if they're more from the perspective of a user
> > please use the [hidden email] list.
> >
> > For the technical questions it is probably best to align them with
> > some portion of the overview, administration, user, best practice, or
> > developer guide so we can use that as a basis for further discussion.
> > You can find those here: https://nifi.apache.org/docs.html
> >
> > Thanks
> > Joe
> >
> >
> > ---------- Forwarded message ----------
> > From: Mothilal M <[hidden email]>
> > Date: Tue, Nov 8, 2016 at 1:25 PM
> > Subject: Need Information - Apache NiFi
> > To: [hidden email]
> >
> >
> > Hi Apache NiFi team,
> >
> > Greetings. I did my hands on experience with multiple tutorials around
> > sentimental analysis integrating social media like twitter and facebook
> and
> > also log analysis. Could you help me understand in what features Apache
> > NiFi would act as an alternative to Splunk.
> >
> > I do have couple of technical clarifications in Apache NiFi - would it be
> > possible to have a initial technical call to clarify my questions.
> >
> > Warm Regards,
> > M. Mothilal
> > Cell : +1(647)641-0910.
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Mothi86
In reply to this post by Mothi86
Hi Bryan Bende,

Greetings. I did try filter endpoint and it certainly needs numeric ID and
not twitter ID. Processor shows error if twitter ID - so I had converted it
to respective numeric. Please advice if there is any sample worked on this
use case - filtered tweets based on specific IDs.

Warm Regards,
M. Mothilal

On Tue, Nov 8, 2016 at 2:45 PM, Mothilal M <[hidden email]> wrote:

> Hi Joe,
>
> Greetings and appreciate your reply. Totally agree with your opinion on
> comparisons.
>
> Look forward for other developers to comment on suggestions of getTwitter
> and getHTTP processor issues for twitter and facebook integration.
>
> Warm Regards,
> M. Mothilal
>
> On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <[hidden email]> wrote:
>
>> Greetings Joe,
>>
>> Glad to connect with you from your reply in Apache NiFi community. I am
>> integrating NiFi with social media - twitter and Facebook to achieve
>> analysis but I am struck at a point. Request your assistance in solving the
>> same.
>>
>> getTwitter processor - i am not able to get tweets filtered by twitter
>> IDs. (IDs to follow field value is not working as expected. I did get the
>> numeric value of twitter IDs and input in the value as comma separated but
>> no results). Please advice as I look to filter my tweets on particular IDs
>> and also by particular location by cities like Toronto, calgary.
>>
>> getHTTP processor for facebook integration works fine but i am unable to
>> fetch data from my personal facebook ID. I tried accessing the Graph API
>> and it mentions user permissions are not granted but I did provide the
>> essential permissions in app settings. Please advice if you could help out
>> in setting the facebook app permissions to ensure I can fetch user_status,
>> user_location fields. On success, I would need to venture out few options
>> further.
>>
>> General question - How could Apache NiFi be useful in replacing Splunk. I
>> understand most of the current companies have a paid version of Splunk and
>> NiFi is an open-source but please let me know features which are key in
>> replacing Splunk.
>>
>> I appreciate your time and would be more helpful if you could let me know
>> your comfortable timing for a technical call discussion. I am available and
>> reachable on +1(647)641-0910. Look forwardly for your reply.
>>
>> Warm Regards,
>> M. Mothilal
>>
>> ------------------------------------------------------------
>> ------------------------------
>>
>> Hello,
>>
>> I think as a community we're all quite happy to help address questions
>> or ideas you might have for Apache NiFi.  Please feel free to ask your
>> questions here and if they're more from the perspective of a user
>> please use the [hidden email] list.
>>
>> For the technical questions it is probably best to align them with
>> some portion of the overview, administration, user, best practice, or
>> developer guide so we can use that as a basis for further discussion.
>> You can find those here: https://nifi.apache.org/docs.html
>>
>> Thanks
>> Joe
>>
>>
>> ---------- Forwarded message ----------
>> From: Mothilal M <[hidden email]>
>> Date: Tue, Nov 8, 2016 at 1:25 PM
>> Subject: Need Information - Apache NiFi
>> To: [hidden email]
>>
>>
>> Hi Apache NiFi team,
>>
>> Greetings. I did my hands on experience with multiple tutorials around
>> sentimental analysis integrating social media like twitter and facebook and
>> also log analysis. Could you help me understand in what features Apache
>> NiFi would act as an alternative to Splunk.
>>
>> I do have couple of technical clarifications in Apache NiFi - would it be
>> possible to have a initial technical call to clarify my questions.
>>
>> Warm Regards,
>> M. Mothilal
>> Cell : +1(647)641-0910.
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Andrew Psaltis
Hi M. Mothilal,
Attached is a template that filters based on a Twitter ID (my id, which I retrieved from [1]). As I am sure you already know, you need to change the tokens and keys to match your application. After setting up the configuration, I went to Twitter and just re-tweeted a tweet and moments later it appeared in my NiFi flow. The different actions that should cause data to appear are identified here [2].

Hopefully this helps.




Thanks,
Andrew

On Tue, Nov 8, 2016 at 6:59 PM, Mothilal M <[hidden email]> wrote:
Hi Bryan Bende,

Greetings. I did try filter endpoint and it certainly needs numeric ID and
not twitter ID. Processor shows error if twitter ID - so I had converted it
to respective numeric. Please advice if there is any sample worked on this
use case - filtered tweets based on specific IDs.

Warm Regards,
M. Mothilal

On Tue, Nov 8, 2016 at 2:45 PM, Mothilal M <[hidden email]> wrote:

> Hi Joe,
>
> Greetings and appreciate your reply. Totally agree with your opinion on
> comparisons.
>
> Look forward for other developers to comment on suggestions of getTwitter
> and getHTTP processor issues for twitter and facebook integration.
>
> Warm Regards,
> M. Mothilal
>
> On Tue, Nov 8, 2016 at 2:21 PM, Mothilal M <[hidden email]> wrote:
>
>> Greetings Joe,
>>
>> Glad to connect with you from your reply in Apache NiFi community. I am
>> integrating NiFi with social media - twitter and Facebook to achieve
>> analysis but I am struck at a point. Request your assistance in solving the
>> same.
>>
>> getTwitter processor - i am not able to get tweets filtered by twitter
>> IDs. (IDs to follow field value is not working as expected. I did get the
>> numeric value of twitter IDs and input in the value as comma separated but
>> no results). Please advice as I look to filter my tweets on particular IDs
>> and also by particular location by cities like Toronto, calgary.
>>
>> getHTTP processor for facebook integration works fine but i am unable to
>> fetch data from my personal facebook ID. I tried accessing the Graph API
>> and it mentions user permissions are not granted but I did provide the
>> essential permissions in app settings. Please advice if you could help out
>> in setting the facebook app permissions to ensure I can fetch user_status,
>> user_location fields. On success, I would need to venture out few options
>> further.
>>
>> General question - How could Apache NiFi be useful in replacing Splunk. I
>> understand most of the current companies have a paid version of Splunk and
>> NiFi is an open-source but please let me know features which are key in
>> replacing Splunk.
>>
>> I appreciate your time and would be more helpful if you could let me know
>> your comfortable timing for a technical call discussion. I am available and
>> reachable on <a href="tel:%2B1%28647%29641-0910" value="+16476410910">+1(647)641-0910. Look forwardly for your reply.
>>
>> Warm Regards,
>> M. Mothilal
>>
>> ------------------------------------------------------------
>> ------------------------------
>>
>> Hello,
>>
>> I think as a community we're all quite happy to help address questions
>> or ideas you might have for Apache NiFi.  Please feel free to ask your
>> questions here and if they're more from the perspective of a user
>> please use the [hidden email] list.
>>
>> For the technical questions it is probably best to align them with
>> some portion of the overview, administration, user, best practice, or
>> developer guide so we can use that as a basis for further discussion.
>> You can find those here: https://nifi.apache.org/docs.html
>>
>> Thanks
>> Joe
>>
>>
>> ---------- Forwarded message ----------
>> From: Mothilal M <[hidden email]>
>> Date: Tue, Nov 8, 2016 at 1:25 PM
>> Subject: Need Information - Apache NiFi
>> To: [hidden email]
>>
>>
>> Hi Apache NiFi team,
>>
>> Greetings. I did my hands on experience with multiple tutorials around
>> sentimental analysis integrating social media like twitter and facebook and
>> also log analysis. Could you help me understand in what features Apache
>> NiFi would act as an alternative to Splunk.
>>
>> I do have couple of technical clarifications in Apache NiFi - would it be
>> possible to have a initial technical call to clarify my questions.
>>
>> Warm Regards,
>> M. Mothilal
>> Cell : <a href="tel:%2B1%28647%29641-0910" value="+16476410910">+1(647)641-0910.
>>
>>
>



--
Thanks,
Andrew

Subscribe to my book: Streaming Data

twiiter: @itmdata

FilterTwitterByID.xml (34K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Need Information - Apache NiFi

Mothi86
Thanks @JoeWitt @BryanBende @AndrewPsaltis for your valuable comments.

Getting back active on this thread connections after a long time. I was able to succeed long back in retrieving the tweets filtered by users (IDs needs to be the numeric values and can be fetched from twitter or http://gettwitterid.com/).