Possibility of running NiFi embedded in a test

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Possibility of running NiFi embedded in a test

Esteban Aliverti
Hello,
My name is Esteban Aliverti and I'm currently trying to use NiFi in one of
my company's projects. The idea is to use NiFi behind a java API.
What I was wondering is if there's any way to run NiFi embedded so I can
easily build unit tests without having to start a separate NiFi server
before I run the tests. Another important thing is that each test should be
self-contained and thus I would like them to always run in a "clean" NiFi
instance.
What I was thinking was a way to programatically configure and start NiFi
and maybe import a workflow from a template definition I can create
beforehand.
Is this currently possible? If not, any suggestion?

Regards,

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Mark Payne
Esteban,


NiFi does not allow you to run it embedded. However, it does provide a “Mock Framework” that we use very heavily for doing unit testing of NiFi components.


Can you explain your use case a bit further? What exactly are you trying to unit test?


Thanks

-Mark






From: Esteban Aliverti
Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
To: [hidden email]





Hello,
My name is Esteban Aliverti and I'm currently trying to use NiFi in one of
my company's projects. The idea is to use NiFi behind a java API.
What I was wondering is if there's any way to run NiFi embedded so I can
easily build unit tests without having to start a separate NiFi server
before I run the tests. Another important thing is that each test should be
self-contained and thus I would like them to always run in a "clean" NiFi
instance.
What I was thinking was a way to programatically configure and start NiFi
and maybe import a workflow from a template definition I can create
beforehand.
Is this currently possible? If not, any suggestion?

Regards,

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Esteban Aliverti
Thanks for the prompt response Mark.
My idea was to have a NiFi workflow defined and to be able to run some
automated tests on it. I thought that maybe there was a way to start an
embedded version of NiFi programatically and to interact with it.
I did see the TestRunner class and I even used it for some of the unit
tests of some custom processors I've implemented. TestRunner seems to be
exactly what I'm looking for, but instead of being focused in a single
processor, I need something that allows me to run multiple processors that
compose a flow.
I do agree that each individual processor must be tested using a
TestRunner, but I was planning on doing something more like an integration
test of an entire flow.

Regards,




XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com

On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]> wrote:

> Esteban,
>
>
> NiFi does not allow you to run it embedded. However, it does provide a
> “Mock Framework” that we use very heavily for doing unit testing of NiFi
> components.
>
>
> Can you explain your use case a bit further? What exactly are you trying
> to unit test?
>
>
> Thanks
>
> -Mark
>
>
>
>
>
>
> From: Esteban Aliverti
> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
> To: [hidden email]
>
>
>
>
>
> Hello,
> My name is Esteban Aliverti and I'm currently trying to use NiFi in one of
> my company's projects. The idea is to use NiFi behind a java API.
> What I was wondering is if there's any way to run NiFi embedded so I can
> easily build unit tests without having to start a separate NiFi server
> before I run the tests. Another important thing is that each test should be
> self-contained and thus I would like them to always run in a "clean" NiFi
> instance.
> What I was thinking was a way to programatically configure and start NiFi
> and maybe import a workflow from a template definition I can create
> beforehand.
> Is this currently possible? If not, any suggestion?
>
> Regards,
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Mark Payne
In reply to this post by Esteban Aliverti
Esteban,




OK, thanks for clarifying. This notion of how to test your flow has starting coming up a lot in the last week or so on the dev list. I’ll take this to be a good thing, meaning that people are really starting to think about how they can use it! 😊




We have actually intentionally avoided creating anything to allow developers to create unit/integration tests for flows. Creating these sorts of tests are very common for most ETL types of applications. However, NiFi was designed specifically so that there was no need to do so. The real-time command and control that NiFi affords you is very different than that provided by typical ETL applications. Because of this real-time command and control, we push operators to build a “test" flow on the production instance and run the live data through the test flow at the same time that it is processed by the operational flow. In the test flow, though, the operator will likely terminate the data at the end, rather than sending to any other system. This allows the operator to verify that everything is in good shape with the test flow. Once it has been determined that everything is in good shape, the operator should then enable sending the data to the appropriate destinations. This tends to provide much faster time-to-market of flow changes.


Over the past several days there has been another thread discussing the notion of testing a flow. This thread can be viewed at http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811


If you have specific questions/concerns, I’d be happy to discuss in more detail.


Thanks

-Mark









From: Esteban Aliverti
Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎10‎:‎52‎ ‎AM
To: [hidden email]





Thanks for the prompt response Mark.
My idea was to have a NiFi workflow defined and to be able to run some
automated tests on it. I thought that maybe there was a way to start an
embedded version of NiFi programatically and to interact with it.
I did see the TestRunner class and I even used it for some of the unit
tests of some custom processors I've implemented. TestRunner seems to be
exactly what I'm looking for, but instead of being focused in a single
processor, I need something that allows me to run multiple processors that
compose a flow.
I do agree that each individual processor must be tested using a
TestRunner, but I was planning on doing something more like an integration
test of an entire flow.

Regards,




XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com

On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]> wrote:

> Esteban,
>
>
> NiFi does not allow you to run it embedded. However, it does provide a
> “Mock Framework” that we use very heavily for doing unit testing of NiFi
> components.
>
>
> Can you explain your use case a bit further? What exactly are you trying
> to unit test?
>
>
> Thanks
>
> -Mark
>
>
>
>
>
>
> From: Esteban Aliverti
> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
> To: [hidden email]
>
>
>
>
>
> Hello,
> My name is Esteban Aliverti and I'm currently trying to use NiFi in one of
> my company's projects. The idea is to use NiFi behind a java API.
> What I was wondering is if there's any way to run NiFi embedded so I can
> easily build unit tests without having to start a separate NiFi server
> before I run the tests. Another important thing is that each test should be
> self-contained and thus I would like them to always run in a "clean" NiFi
> instance.
> What I was thinking was a way to programatically configure and start NiFi
> and maybe import a workflow from a template definition I can create
> beforehand.
> Is this currently possible? If not, any suggestion?
>
> Regards,
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Esteban Aliverti
Thanks for the clarification. I'll try to see what other options do I have.
My scenario is not a simple ETL where everything gets resolved using local
data. The flow interacts with a bunch of external systems to both, retrieve
and send data. I was planning to use mocked services in those cases.

Thanks for your time!

Regards,


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com

On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[hidden email]> wrote:

> Esteban,
>
>
>
>
> OK, thanks for clarifying. This notion of how to test your flow has
> starting coming up a lot in the last week or so on the dev list. I’ll take
> this to be a good thing, meaning that people are really starting to think
> about how they can use it! 😊
>
>
>
>
> We have actually intentionally avoided creating anything to allow
> developers to create unit/integration tests for flows. Creating these sorts
> of tests are very common for most ETL types of applications. However, NiFi
> was designed specifically so that there was no need to do so. The real-time
> command and control that NiFi affords you is very different than that
> provided by typical ETL applications. Because of this real-time command and
> control, we push operators to build a “test" flow on the production
> instance and run the live data through the test flow at the same time that
> it is processed by the operational flow. In the test flow, though, the
> operator will likely terminate the data at the end, rather than sending to
> any other system. This allows the operator to verify that everything is in
> good shape with the test flow. Once it has been determined that everything
> is in good shape, the operator should then enable sending the data to the
> appropriate destinations. This tends to provide much faster time-to-market
> of flow changes.
>
>
> Over the past several days there has been another thread discussing the
> notion of testing a flow. This thread can be viewed at
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811
>
>
> If you have specific questions/concerns, I’d be happy to discuss in more
> detail.
>
>
> Thanks
>
> -Mark
>
>
>
>
>
>
>
>
>
> From: Esteban Aliverti
> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎10‎:‎52‎ ‎AM
> To: [hidden email]
>
>
>
>
>
> Thanks for the prompt response Mark.
> My idea was to have a NiFi workflow defined and to be able to run some
> automated tests on it. I thought that maybe there was a way to start an
> embedded version of NiFi programatically and to interact with it.
> I did see the TestRunner class and I even used it for some of the unit
> tests of some custom processors I've implemented. TestRunner seems to be
> exactly what I'm looking for, but instead of being focused in a single
> processor, I need something that allows me to run multiple processors that
> compose a flow.
> I do agree that each individual processor must be tested using a
> TestRunner, but I was planning on doing something more like an integration
> test of an entire flow.
>
> Regards,
>
>
>
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
> On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]> wrote:
>
> > Esteban,
> >
> >
> > NiFi does not allow you to run it embedded. However, it does provide a
> > “Mock Framework” that we use very heavily for doing unit testing of NiFi
> > components.
> >
> >
> > Can you explain your use case a bit further? What exactly are you trying
> > to unit test?
> >
> >
> > Thanks
> >
> > -Mark
> >
> >
> >
> >
> >
> >
> > From: Esteban Aliverti
> > Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
> > To: [hidden email]
> >
> >
> >
> >
> >
> > Hello,
> > My name is Esteban Aliverti and I'm currently trying to use NiFi in one
> of
> > my company's projects. The idea is to use NiFi behind a java API.
> > What I was wondering is if there's any way to run NiFi embedded so I can
> > easily build unit tests without having to start a separate NiFi server
> > before I run the tests. Another important thing is that each test should
> be
> > self-contained and thus I would like them to always run in a "clean" NiFi
> > instance.
> > What I was thinking was a way to programatically configure and start NiFi
> > and maybe import a workflow from a template definition I can create
> > beforehand.
> > Is this currently possible? If not, any suggestion?
> >
> > Regards,
> >
> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> >
> > Esteban Aliverti
> > - Blog @ http://ilesteban.wordpress.com
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Mark Payne
In reply to this post by Esteban Aliverti
Esteban,


I totally understand. In this case, I believe it makes my description of the NiFi use case much stronger. As the flow becomes more and more complex, the more and more difficult it becomes to keep an integration test up-to-date (and to build it!) and the easier it becomes to configure via the UI.


But if I’m missing something please let me know.


Thanks

-Mark








From: Esteban Aliverti
Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎11‎:‎15‎ ‎AM
To: [hidden email]





Thanks for the clarification. I'll try to see what other options do I have.
My scenario is not a simple ETL where everything gets resolved using local
data. The flow interacts with a bunch of external systems to both, retrieve
and send data. I was planning to use mocked services in those cases.

Thanks for your time!

Regards,


XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com

On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[hidden email]> wrote:

> Esteban,
>
>
>
>
> OK, thanks for clarifying. This notion of how to test your flow has
> starting coming up a lot in the last week or so on the dev list. I’ll take
> this to be a good thing, meaning that people are really starting to think
> about how they can use it! 😊
>
>
>
>
> We have actually intentionally avoided creating anything to allow
> developers to create unit/integration tests for flows. Creating these sorts
> of tests are very common for most ETL types of applications. However, NiFi
> was designed specifically so that there was no need to do so. The real-time
> command and control that NiFi affords you is very different than that
> provided by typical ETL applications. Because of this real-time command and
> control, we push operators to build a “test" flow on the production
> instance and run the live data through the test flow at the same time that
> it is processed by the operational flow. In the test flow, though, the
> operator will likely terminate the data at the end, rather than sending to
> any other system. This allows the operator to verify that everything is in
> good shape with the test flow. Once it has been determined that everything
> is in good shape, the operator should then enable sending the data to the
> appropriate destinations. This tends to provide much faster time-to-market
> of flow changes.
>
>
> Over the past several days there has been another thread discussing the
> notion of testing a flow. This thread can be viewed at
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811
>
>
> If you have specific questions/concerns, I’d be happy to discuss in more
> detail.
>
>
> Thanks
>
> -Mark
>
>
>
>
>
>
>
>
>
> From: Esteban Aliverti
> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎10‎:‎52‎ ‎AM
> To: [hidden email]
>
>
>
>
>
> Thanks for the prompt response Mark.
> My idea was to have a NiFi workflow defined and to be able to run some
> automated tests on it. I thought that maybe there was a way to start an
> embedded version of NiFi programatically and to interact with it.
> I did see the TestRunner class and I even used it for some of the unit
> tests of some custom processors I've implemented. TestRunner seems to be
> exactly what I'm looking for, but instead of being focused in a single
> processor, I need something that allows me to run multiple processors that
> compose a flow.
> I do agree that each individual processor must be tested using a
> TestRunner, but I was planning on doing something more like an integration
> test of an entire flow.
>
> Regards,
>
>
>
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
> On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]> wrote:
>
> > Esteban,
> >
> >
> > NiFi does not allow you to run it embedded. However, it does provide a
> > “Mock Framework” that we use very heavily for doing unit testing of NiFi
> > components.
> >
> >
> > Can you explain your use case a bit further? What exactly are you trying
> > to unit test?
> >
> >
> > Thanks
> >
> > -Mark
> >
> >
> >
> >
> >
> >
> > From: Esteban Aliverti
> > Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
> > To: [hidden email]
> >
> >
> >
> >
> >
> > Hello,
> > My name is Esteban Aliverti and I'm currently trying to use NiFi in one
> of
> > my company's projects. The idea is to use NiFi behind a java API.
> > What I was wondering is if there's any way to run NiFi embedded so I can
> > easily build unit tests without having to start a separate NiFi server
> > before I run the tests. Another important thing is that each test should
> be
> > self-contained and thus I would like them to always run in a "clean" NiFi
> > instance.
> > What I was thinking was a way to programatically configure and start NiFi
> > and maybe import a workflow from a template definition I can create
> > beforehand.
> > Is this currently possible? If not, any suggestion?
> >
> > Regards,
> >
> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> >
> > Esteban Aliverti
> > - Blog @ http://ilesteban.wordpress.com
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Joe Witt
In reply to this post by Esteban Aliverti
Esteban,

Understood.  This is really a key strength of a flow-based programming
approach like NiFi takes.  Each component is a cohesive and loosely
coupled entity.  You should be able to produce very effective unit
tests of those components against their well defined interface of
inputs, outputs.  Then in an actual dataflow one can freely compose
those together in new and interesting ways provided it adheres to the
purpose of those components.

When creating unit tests that hit external services it is definitely a
good idea to mock them out.  The same is true when building any
processor.  You want to essentially mock input data and output data
which adheres to or violates your preconditions and validate the
output is as expected.

Integration testing is still of value though of course.  But I'd think
the integration tests would be having your process actually hit a real
instance of the external service.  I think there is likely less value
in an integration test of a preconfigured series of processors.  At
that point your testing the flow and not the processes themselves.  If
you do indeed want to test the behavior of a configured flow itself
that i think is best done on a live instance.

Having said all that I wonder if there is value in us enabling a
developer to programatically generate templates and then providing an
automated testing infrastructure around those templates.  Will have to
think about that.

Thanks
Joe

On Tue, Feb 24, 2015 at 11:13 AM, Esteban Aliverti
<[hidden email]> wrote:

> Thanks for the clarification. I'll try to see what other options do I have.
> My scenario is not a simple ETL where everything gets resolved using local
> data. The flow interacts with a bunch of external systems to both, retrieve
> and send data. I was planning to use mocked services in those cases.
>
> Thanks for your time!
>
> Regards,
>
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
> On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[hidden email]> wrote:
>
>> Esteban,
>>
>>
>>
>>
>> OK, thanks for clarifying. This notion of how to test your flow has
>> starting coming up a lot in the last week or so on the dev list. I’ll take
>> this to be a good thing, meaning that people are really starting to think
>> about how they can use it! 😊
>>
>>
>>
>>
>> We have actually intentionally avoided creating anything to allow
>> developers to create unit/integration tests for flows. Creating these sorts
>> of tests are very common for most ETL types of applications. However, NiFi
>> was designed specifically so that there was no need to do so. The real-time
>> command and control that NiFi affords you is very different than that
>> provided by typical ETL applications. Because of this real-time command and
>> control, we push operators to build a “test" flow on the production
>> instance and run the live data through the test flow at the same time that
>> it is processed by the operational flow. In the test flow, though, the
>> operator will likely terminate the data at the end, rather than sending to
>> any other system. This allows the operator to verify that everything is in
>> good shape with the test flow. Once it has been determined that everything
>> is in good shape, the operator should then enable sending the data to the
>> appropriate destinations. This tends to provide much faster time-to-market
>> of flow changes.
>>
>>
>> Over the past several days there has been another thread discussing the
>> notion of testing a flow. This thread can be viewed at
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811
>>
>>
>> If you have specific questions/concerns, I’d be happy to discuss in more
>> detail.
>>
>>
>> Thanks
>>
>> -Mark
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> From: Esteban Aliverti
>> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎10‎:‎52‎ ‎AM
>> To: [hidden email]
>>
>>
>>
>>
>>
>> Thanks for the prompt response Mark.
>> My idea was to have a NiFi workflow defined and to be able to run some
>> automated tests on it. I thought that maybe there was a way to start an
>> embedded version of NiFi programatically and to interact with it.
>> I did see the TestRunner class and I even used it for some of the unit
>> tests of some custom processors I've implemented. TestRunner seems to be
>> exactly what I'm looking for, but instead of being focused in a single
>> processor, I need something that allows me to run multiple processors that
>> compose a flow.
>> I do agree that each individual processor must be tested using a
>> TestRunner, but I was planning on doing something more like an integration
>> test of an entire flow.
>>
>> Regards,
>>
>>
>>
>>
>> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>>
>> Esteban Aliverti
>> - Blog @ http://ilesteban.wordpress.com
>>
>> On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]> wrote:
>>
>> > Esteban,
>> >
>> >
>> > NiFi does not allow you to run it embedded. However, it does provide a
>> > “Mock Framework” that we use very heavily for doing unit testing of NiFi
>> > components.
>> >
>> >
>> > Can you explain your use case a bit further? What exactly are you trying
>> > to unit test?
>> >
>> >
>> > Thanks
>> >
>> > -Mark
>> >
>> >
>> >
>> >
>> >
>> >
>> > From: Esteban Aliverti
>> > Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
>> > To: [hidden email]
>> >
>> >
>> >
>> >
>> >
>> > Hello,
>> > My name is Esteban Aliverti and I'm currently trying to use NiFi in one
>> of
>> > my company's projects. The idea is to use NiFi behind a java API.
>> > What I was wondering is if there's any way to run NiFi embedded so I can
>> > easily build unit tests without having to start a separate NiFi server
>> > before I run the tests. Another important thing is that each test should
>> be
>> > self-contained and thus I would like them to always run in a "clean" NiFi
>> > instance.
>> > What I was thinking was a way to programatically configure and start NiFi
>> > and maybe import a workflow from a template definition I can create
>> > beforehand.
>> > Is this currently possible? If not, any suggestion?
>> >
>> > Regards,
>> >
>> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>> >
>> > Esteban Aliverti
>> > - Blog @ http://ilesteban.wordpress.com
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Mark Payne
In reply to this post by Esteban Aliverti
Joe,


Regarding the idea of a testing infrastructure that runs against a template, we also have to remember that if that template communicates with an external service, you will then have test failures (and build failures) when that external service is not available. This is probably not a good thing.


Thanks

-Mark






From: Joe Witt
Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎11‎:‎27‎ ‎AM
To: [hidden email]





Esteban,

Understood.  This is really a key strength of a flow-based programming
approach like NiFi takes.  Each component is a cohesive and loosely
coupled entity.  You should be able to produce very effective unit
tests of those components against their well defined interface of
inputs, outputs.  Then in an actual dataflow one can freely compose
those together in new and interesting ways provided it adheres to the
purpose of those components.

When creating unit tests that hit external services it is definitely a
good idea to mock them out.  The same is true when building any
processor.  You want to essentially mock input data and output data
which adheres to or violates your preconditions and validate the
output is as expected.

Integration testing is still of value though of course.  But I'd think
the integration tests would be having your process actually hit a real
instance of the external service.  I think there is likely less value
in an integration test of a preconfigured series of processors.  At
that point your testing the flow and not the processes themselves.  If
you do indeed want to test the behavior of a configured flow itself
that i think is best done on a live instance.

Having said all that I wonder if there is value in us enabling a
developer to programatically generate templates and then providing an
automated testing infrastructure around those templates.  Will have to
think about that.

Thanks
Joe

On Tue, Feb 24, 2015 at 11:13 AM, Esteban Aliverti
<[hidden email]> wrote:

> Thanks for the clarification. I'll try to see what other options do I have.
> My scenario is not a simple ETL where everything gets resolved using local
> data. The flow interacts with a bunch of external systems to both, retrieve
> and send data. I was planning to use mocked services in those cases.
>
> Thanks for your time!
>
> Regards,
>
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> Esteban Aliverti
> - Blog @ http://ilesteban.wordpress.com
>
> On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[hidden email]> wrote:
>
>> Esteban,
>>
>>
>>
>>
>> OK, thanks for clarifying. This notion of how to test your flow has
>> starting coming up a lot in the last week or so on the dev list. I’ll take
>> this to be a good thing, meaning that people are really starting to think
>> about how they can use it! 😊
>>
>>
>>
>>
>> We have actually intentionally avoided creating anything to allow
>> developers to create unit/integration tests for flows. Creating these sorts
>> of tests are very common for most ETL types of applications. However, NiFi
>> was designed specifically so that there was no need to do so. The real-time
>> command and control that NiFi affords you is very different than that
>> provided by typical ETL applications. Because of this real-time command and
>> control, we push operators to build a “test" flow on the production
>> instance and run the live data through the test flow at the same time that
>> it is processed by the operational flow. In the test flow, though, the
>> operator will likely terminate the data at the end, rather than sending to
>> any other system. This allows the operator to verify that everything is in
>> good shape with the test flow. Once it has been determined that everything
>> is in good shape, the operator should then enable sending the data to the
>> appropriate destinations. This tends to provide much faster time-to-market
>> of flow changes.
>>
>>
>> Over the past several days there has been another thread discussing the
>> notion of testing a flow. This thread can be viewed at
>> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811
>>
>>
>> If you have specific questions/concerns, I’d be happy to discuss in more
>> detail.
>>
>>
>> Thanks
>>
>> -Mark
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> From: Esteban Aliverti
>> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎10‎:‎52‎ ‎AM
>> To: [hidden email]
>>
>>
>>
>>
>>
>> Thanks for the prompt response Mark.
>> My idea was to have a NiFi workflow defined and to be able to run some
>> automated tests on it. I thought that maybe there was a way to start an
>> embedded version of NiFi programatically and to interact with it.
>> I did see the TestRunner class and I even used it for some of the unit
>> tests of some custom processors I've implemented. TestRunner seems to be
>> exactly what I'm looking for, but instead of being focused in a single
>> processor, I need something that allows me to run multiple processors that
>> compose a flow.
>> I do agree that each individual processor must be tested using a
>> TestRunner, but I was planning on doing something more like an integration
>> test of an entire flow.
>>
>> Regards,
>>
>>
>>
>>
>> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>>
>> Esteban Aliverti
>> - Blog @ http://ilesteban.wordpress.com
>>
>> On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]> wrote:
>>
>> > Esteban,
>> >
>> >
>> > NiFi does not allow you to run it embedded. However, it does provide a
>> > “Mock Framework” that we use very heavily for doing unit testing of NiFi
>> > components.
>> >
>> >
>> > Can you explain your use case a bit further? What exactly are you trying
>> > to unit test?
>> >
>> >
>> > Thanks
>> >
>> > -Mark
>> >
>> >
>> >
>> >
>> >
>> >
>> > From: Esteban Aliverti
>> > Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
>> > To: [hidden email]
>> >
>> >
>> >
>> >
>> >
>> > Hello,
>> > My name is Esteban Aliverti and I'm currently trying to use NiFi in one
>> of
>> > my company's projects. The idea is to use NiFi behind a java API.
>> > What I was wondering is if there's any way to run NiFi embedded so I can
>> > easily build unit tests without having to start a separate NiFi server
>> > before I run the tests. Another important thing is that each test should
>> be
>> > self-contained and thus I would like them to always run in a "clean" NiFi
>> > instance.
>> > What I was thinking was a way to programatically configure and start NiFi
>> > and maybe import a workflow from a template definition I can create
>> > beforehand.
>> > Is this currently possible? If not, any suggestion?
>> >
>> > Regards,
>> >
>> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>> >
>> > Esteban Aliverti
>> > - Blog @ http://ilesteban.wordpress.com
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Joe Gresock
For what it's worth, our team has developed an extensive set of integration
tests against a live instance using an "integration.test" flow file
attribute to route data to local disk in addition to the actual endpoints.
We're able to retrieve files from these directories and verify some basic
things about their filenames, contents, etc.

This has proved invaluable and has caught many, many flow configuration
regression errors, as well as detected problems occurring in the external
systems.  With Nifi as it currently is, I'd strongly recommend an approach
like this.

Incidentally, the Nifi mock framework is quite powerful for simple
processor chaining in unit tests if you need it; the output of one
processor can be captured in a MockFlowFile, which can be the input to
another processor.

Joe

On Tue, Feb 24, 2015 at 11:31 AM, Mark Payne <[hidden email]> wrote:

> Joe,
>
>
> Regarding the idea of a testing infrastructure that runs against a
> template, we also have to remember that if that template communicates with
> an external service, you will then have test failures (and build failures)
> when that external service is not available. This is probably not a good
> thing.
>
>
> Thanks
>
> -Mark
>
>
>
>
>
>
> From: Joe Witt
> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎11‎:‎27‎ ‎AM
> To: [hidden email]
>
>
>
>
>
> Esteban,
>
> Understood.  This is really a key strength of a flow-based programming
> approach like NiFi takes.  Each component is a cohesive and loosely
> coupled entity.  You should be able to produce very effective unit
> tests of those components against their well defined interface of
> inputs, outputs.  Then in an actual dataflow one can freely compose
> those together in new and interesting ways provided it adheres to the
> purpose of those components.
>
> When creating unit tests that hit external services it is definitely a
> good idea to mock them out.  The same is true when building any
> processor.  You want to essentially mock input data and output data
> which adheres to or violates your preconditions and validate the
> output is as expected.
>
> Integration testing is still of value though of course.  But I'd think
> the integration tests would be having your process actually hit a real
> instance of the external service.  I think there is likely less value
> in an integration test of a preconfigured series of processors.  At
> that point your testing the flow and not the processes themselves.  If
> you do indeed want to test the behavior of a configured flow itself
> that i think is best done on a live instance.
>
> Having said all that I wonder if there is value in us enabling a
> developer to programatically generate templates and then providing an
> automated testing infrastructure around those templates.  Will have to
> think about that.
>
> Thanks
> Joe
>
> On Tue, Feb 24, 2015 at 11:13 AM, Esteban Aliverti
> <[hidden email]> wrote:
> > Thanks for the clarification. I'll try to see what other options do I
> have.
> > My scenario is not a simple ETL where everything gets resolved using
> local
> > data. The flow interacts with a bunch of external systems to both,
> retrieve
> > and send data. I was planning to use mocked services in those cases.
> >
> > Thanks for your time!
> >
> > Regards,
> >
> >
> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> >
> > Esteban Aliverti
> > - Blog @ http://ilesteban.wordpress.com
> >
> > On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[hidden email]>
> wrote:
> >
> >> Esteban,
> >>
> >>
> >>
> >>
> >> OK, thanks for clarifying. This notion of how to test your flow has
> >> starting coming up a lot in the last week or so on the dev list. I’ll
> take
> >> this to be a good thing, meaning that people are really starting to
> think
> >> about how they can use it! 😊
> >>
> >>
> >>
> >>
> >> We have actually intentionally avoided creating anything to allow
> >> developers to create unit/integration tests for flows. Creating these
> sorts
> >> of tests are very common for most ETL types of applications. However,
> NiFi
> >> was designed specifically so that there was no need to do so. The
> real-time
> >> command and control that NiFi affords you is very different than that
> >> provided by typical ETL applications. Because of this real-time command
> and
> >> control, we push operators to build a “test" flow on the production
> >> instance and run the live data through the test flow at the same time
> that
> >> it is processed by the operational flow. In the test flow, though, the
> >> operator will likely terminate the data at the end, rather than sending
> to
> >> any other system. This allows the operator to verify that everything is
> in
> >> good shape with the test flow. Once it has been determined that
> everything
> >> is in good shape, the operator should then enable sending the data to
> the
> >> appropriate destinations. This tends to provide much faster
> time-to-market
> >> of flow changes.
> >>
> >>
> >> Over the past several days there has been another thread discussing the
> >> notion of testing a flow. This thread can be viewed at
> >>
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811
> >>
> >>
> >> If you have specific questions/concerns, I’d be happy to discuss in more
> >> detail.
> >>
> >>
> >> Thanks
> >>
> >> -Mark
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> From: Esteban Aliverti
> >> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎10‎:‎52‎ ‎AM
> >> To: [hidden email]
> >>
> >>
> >>
> >>
> >>
> >> Thanks for the prompt response Mark.
> >> My idea was to have a NiFi workflow defined and to be able to run some
> >> automated tests on it. I thought that maybe there was a way to start an
> >> embedded version of NiFi programatically and to interact with it.
> >> I did see the TestRunner class and I even used it for some of the unit
> >> tests of some custom processors I've implemented. TestRunner seems to be
> >> exactly what I'm looking for, but instead of being focused in a single
> >> processor, I need something that allows me to run multiple processors
> that
> >> compose a flow.
> >> I do agree that each individual processor must be tested using a
> >> TestRunner, but I was planning on doing something more like an
> integration
> >> test of an entire flow.
> >>
> >> Regards,
> >>
> >>
> >>
> >>
> >> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> >>
> >> Esteban Aliverti
> >> - Blog @ http://ilesteban.wordpress.com
> >>
> >> On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]>
> wrote:
> >>
> >> > Esteban,
> >> >
> >> >
> >> > NiFi does not allow you to run it embedded. However, it does provide a
> >> > “Mock Framework” that we use very heavily for doing unit testing of
> NiFi
> >> > components.
> >> >
> >> >
> >> > Can you explain your use case a bit further? What exactly are you
> trying
> >> > to unit test?
> >> >
> >> >
> >> > Thanks
> >> >
> >> > -Mark
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > From: Esteban Aliverti
> >> > Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
> >> > To: [hidden email]
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > Hello,
> >> > My name is Esteban Aliverti and I'm currently trying to use NiFi in
> one
> >> of
> >> > my company's projects. The idea is to use NiFi behind a java API.
> >> > What I was wondering is if there's any way to run NiFi embedded so I
> can
> >> > easily build unit tests without having to start a separate NiFi server
> >> > before I run the tests. Another important thing is that each test
> should
> >> be
> >> > self-contained and thus I would like them to always run in a "clean"
> NiFi
> >> > instance.
> >> > What I was thinking was a way to programatically configure and start
> NiFi
> >> > and maybe import a workflow from a template definition I can create
> >> > beforehand.
> >> > Is this currently possible? If not, any suggestion?
> >> >
> >> > Regards,
> >> >
> >> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> >> >
> >> > Esteban Aliverti
> >> > - Blog @ http://ilesteban.wordpress.com
> >> >
> >>
>



--
I know what it is to be in need, and I know what it is to have plenty.  I
have learned the secret of being content in any and every situation,
whether well fed or hungry, whether living in plenty or in want.  I can do
all this through him who gives me strength.    *-Philippians 4:12-13*
Reply | Threaded
Open this post in threaded view
|

Re: Possibility of running NiFi embedded in a test

Esteban Aliverti
@Mark, "Regarding the idea of a testing infrastructure that runs against a
template, we also have to remember that if that template communicates with
an external service, you will then have test failures (and build failures)
when that external service is not available. This is probably not a good
thing.". -> True, what I was planning to do was to mock those services. The
template will not be pointing to the production endpoints, but to some
mocked ones. Most of the interaction with external systems in my case are
done through HTTP, so that is something easy to mock for my unit tests. I'm
already doing something similar when testing a custom HTTP Post Processor
I've implemented with TestRunner.

@Joe, thanks for the advice! I'll have it in mind.

Regards,




XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Esteban Aliverti
- Blog @ http://ilesteban.wordpress.com

On Wed, Feb 25, 2015 at 2:58 AM, Joe Gresock <[hidden email]> wrote:

> For what it's worth, our team has developed an extensive set of integration
> tests against a live instance using an "integration.test" flow file
> attribute to route data to local disk in addition to the actual endpoints.
> We're able to retrieve files from these directories and verify some basic
> things about their filenames, contents, etc.
>
> This has proved invaluable and has caught many, many flow configuration
> regression errors, as well as detected problems occurring in the external
> systems.  With Nifi as it currently is, I'd strongly recommend an approach
> like this.
>
> Incidentally, the Nifi mock framework is quite powerful for simple
> processor chaining in unit tests if you need it; the output of one
> processor can be captured in a MockFlowFile, which can be the input to
> another processor.
>
> Joe
>
> On Tue, Feb 24, 2015 at 11:31 AM, Mark Payne <[hidden email]> wrote:
>
> > Joe,
> >
> >
> > Regarding the idea of a testing infrastructure that runs against a
> > template, we also have to remember that if that template communicates
> with
> > an external service, you will then have test failures (and build
> failures)
> > when that external service is not available. This is probably not a good
> > thing.
> >
> >
> > Thanks
> >
> > -Mark
> >
> >
> >
> >
> >
> >
> > From: Joe Witt
> > Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎11‎:‎27‎ ‎AM
> > To: [hidden email]
> >
> >
> >
> >
> >
> > Esteban,
> >
> > Understood.  This is really a key strength of a flow-based programming
> > approach like NiFi takes.  Each component is a cohesive and loosely
> > coupled entity.  You should be able to produce very effective unit
> > tests of those components against their well defined interface of
> > inputs, outputs.  Then in an actual dataflow one can freely compose
> > those together in new and interesting ways provided it adheres to the
> > purpose of those components.
> >
> > When creating unit tests that hit external services it is definitely a
> > good idea to mock them out.  The same is true when building any
> > processor.  You want to essentially mock input data and output data
> > which adheres to or violates your preconditions and validate the
> > output is as expected.
> >
> > Integration testing is still of value though of course.  But I'd think
> > the integration tests would be having your process actually hit a real
> > instance of the external service.  I think there is likely less value
> > in an integration test of a preconfigured series of processors.  At
> > that point your testing the flow and not the processes themselves.  If
> > you do indeed want to test the behavior of a configured flow itself
> > that i think is best done on a live instance.
> >
> > Having said all that I wonder if there is value in us enabling a
> > developer to programatically generate templates and then providing an
> > automated testing infrastructure around those templates.  Will have to
> > think about that.
> >
> > Thanks
> > Joe
> >
> > On Tue, Feb 24, 2015 at 11:13 AM, Esteban Aliverti
> > <[hidden email]> wrote:
> > > Thanks for the clarification. I'll try to see what other options do I
> > have.
> > > My scenario is not a simple ETL where everything gets resolved using
> > local
> > > data. The flow interacts with a bunch of external systems to both,
> > retrieve
> > > and send data. I was planning to use mocked services in those cases.
> > >
> > > Thanks for your time!
> > >
> > > Regards,
> > >
> > >
> > > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> > >
> > > Esteban Aliverti
> > > - Blog @ http://ilesteban.wordpress.com
> > >
> > > On Tue, Feb 24, 2015 at 4:54 PM, Mark Payne <[hidden email]>
> > wrote:
> > >
> > >> Esteban,
> > >>
> > >>
> > >>
> > >>
> > >> OK, thanks for clarifying. This notion of how to test your flow has
> > >> starting coming up a lot in the last week or so on the dev list. I’ll
> > take
> > >> this to be a good thing, meaning that people are really starting to
> > think
> > >> about how they can use it! 😊
> > >>
> > >>
> > >>
> > >>
> > >> We have actually intentionally avoided creating anything to allow
> > >> developers to create unit/integration tests for flows. Creating these
> > sorts
> > >> of tests are very common for most ETL types of applications. However,
> > NiFi
> > >> was designed specifically so that there was no need to do so. The
> > real-time
> > >> command and control that NiFi affords you is very different than that
> > >> provided by typical ETL applications. Because of this real-time
> command
> > and
> > >> control, we push operators to build a “test" flow on the production
> > >> instance and run the live data through the test flow at the same time
> > that
> > >> it is processed by the operational flow. In the test flow, though, the
> > >> operator will likely terminate the data at the end, rather than
> sending
> > to
> > >> any other system. This allows the operator to verify that everything
> is
> > in
> > >> good shape with the test flow. Once it has been determined that
> > everything
> > >> is in good shape, the operator should then enable sending the data to
> > the
> > >> appropriate destinations. This tends to provide much faster
> > time-to-market
> > >> of flow changes.
> > >>
> > >>
> > >> Over the past several days there has been another thread discussing
> the
> > >> notion of testing a flow. This thread can be viewed at
> > >>
> >
> http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/Great-question-on-nifi-IRC-room-today-NiFi-BPM-sharing-configuration-td787.html#a811
> > >>
> > >>
> > >> If you have specific questions/concerns, I’d be happy to discuss in
> more
> > >> detail.
> > >>
> > >>
> > >> Thanks
> > >>
> > >> -Mark
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> From: Esteban Aliverti
> > >> Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎10‎:‎52‎ ‎AM
> > >> To: [hidden email]
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Thanks for the prompt response Mark.
> > >> My idea was to have a NiFi workflow defined and to be able to run some
> > >> automated tests on it. I thought that maybe there was a way to start
> an
> > >> embedded version of NiFi programatically and to interact with it.
> > >> I did see the TestRunner class and I even used it for some of the unit
> > >> tests of some custom processors I've implemented. TestRunner seems to
> be
> > >> exactly what I'm looking for, but instead of being focused in a single
> > >> processor, I need something that allows me to run multiple processors
> > that
> > >> compose a flow.
> > >> I do agree that each individual processor must be tested using a
> > >> TestRunner, but I was planning on doing something more like an
> > integration
> > >> test of an entire flow.
> > >>
> > >> Regards,
> > >>
> > >>
> > >>
> > >>
> > >> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> > >>
> > >> Esteban Aliverti
> > >> - Blog @ http://ilesteban.wordpress.com
> > >>
> > >> On Tue, Feb 24, 2015 at 4:42 PM, Mark Payne <[hidden email]>
> > wrote:
> > >>
> > >> > Esteban,
> > >> >
> > >> >
> > >> > NiFi does not allow you to run it embedded. However, it does
> provide a
> > >> > “Mock Framework” that we use very heavily for doing unit testing of
> > NiFi
> > >> > components.
> > >> >
> > >> >
> > >> > Can you explain your use case a bit further? What exactly are you
> > trying
> > >> > to unit test?
> > >> >
> > >> >
> > >> > Thanks
> > >> >
> > >> > -Mark
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > From: Esteban Aliverti
> > >> > Sent: ‎Tuesday‎, ‎February‎ ‎24‎, ‎2015 ‎8‎:‎34‎ ‎AM
> > >> > To: [hidden email]
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > Hello,
> > >> > My name is Esteban Aliverti and I'm currently trying to use NiFi in
> > one
> > >> of
> > >> > my company's projects. The idea is to use NiFi behind a java API.
> > >> > What I was wondering is if there's any way to run NiFi embedded so I
> > can
> > >> > easily build unit tests without having to start a separate NiFi
> server
> > >> > before I run the tests. Another important thing is that each test
> > should
> > >> be
> > >> > self-contained and thus I would like them to always run in a "clean"
> > NiFi
> > >> > instance.
> > >> > What I was thinking was a way to programatically configure and start
> > NiFi
> > >> > and maybe import a workflow from a template definition I can create
> > >> > beforehand.
> > >> > Is this currently possible? If not, any suggestion?
> > >> >
> > >> > Regards,
> > >> >
> > >> > XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> > >> >
> > >> > Esteban Aliverti
> > >> > - Blog @ http://ilesteban.wordpress.com
> > >> >
> > >>
> >
>
>
>
> --
> I know what it is to be in need, and I know what it is to have plenty.  I
> have learned the secret of being content in any and every situation,
> whether well fed or hungry, whether living in plenty or in want.  I can do
> all this through him who gives me strength.    *-Philippians 4:12-13*
>