nar plugin

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

nar plugin

Benson Margulies
I'm a tiny bit active in the nar plugin on github. Do you have a fork of
it? Are you interested in selling your fork back to the current maintainers?
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Karl Heinz Marbaise
Hi Benson,

I thought also about the nar-maven-plugin (github) just related by the
name... but i think this plugin goes a different direction but you might
better know than i do...

The nar-maven-plugin (github) is intended to support building
(C/C++/Fortran) etc. with packaging the header files etc. into a archive
(NAR ;-)..which will be unpacked...(you know better than i do)...


On 12/14/14 8:41 PM, Benson Margulies wrote:
> I'm a tiny bit active in the nar plugin on github. Do you have a fork of
> it? Are you interested in selling your fork back to the current maintainers?
>

 From my point of view it does not look like this cause the code is at
the moment very rudimentary.......

May be others in the project can say more about the nar-maven-plugin
which intention is behind it?


Kind regards
Karl Heinz marbaise
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

trkurc
Administrator
They're the same in name only. The N of the NiFi nars is for "NiFi" versus
"native". I don't have a canned concise description, perhaps someone else
will weigh in, but they are for packing together code with dependent
libraries, and allowing different bundles to have different dependencies.
They are similar in ilk to the concept of wars. Two wars may have different
versions, potentially conflicting versions of libraries. However, tomcat,
for example, can load the two web applications just fine.

On Sun, Dec 14, 2014 at 2:58 PM, Karl Heinz Marbaise <[hidden email]>
wrote:

>
> Hi Benson,
>
> I thought also about the nar-maven-plugin (github) just related by the
> name... but i think this plugin goes a different direction but you might
> better know than i do...
>
> The nar-maven-plugin (github) is intended to support building
> (C/C++/Fortran) etc. with packaging the header files etc. into a archive
> (NAR ;-)..which will be unpacked...(you know better than i do)...
>
>
> On 12/14/14 8:41 PM, Benson Margulies wrote:
>
>> I'm a tiny bit active in the nar plugin on github. Do you have a fork of
>> it? Are you interested in selling your fork back to the current
>> maintainers?
>>
>>
> From my point of view it does not look like this cause the code is at the
> moment very rudimentary.......
>
> May be others in the project can say more about the nar-maven-plugin which
> intention is behind it?
>
>
> Kind regards
> Karl Heinz marbaise
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Karl Heinz Marbaise
Hi Tony,

sounds like a job for maven-shade-plugin or maven-assembly-plugin...

kind regards
Karl Heinz Marbaisea
On 12/14/14 9:23 PM, Tony Kurc wrote:

> They're the same in name only. The N of the NiFi nars is for "NiFi" versus
> "native". I don't have a canned concise description, perhaps someone else
> will weigh in, but they are for packing together code with dependent
> libraries, and allowing different bundles to have different dependencies.
> They are similar in ilk to the concept of wars. Two wars may have different
> versions, potentially conflicting versions of libraries. However, tomcat,
> for example, can load the two web applications just fine.
>
> On Sun, Dec 14, 2014 at 2:58 PM, Karl Heinz Marbaise <[hidden email]>
> wrote:
>>
>> Hi Benson,
>>
>> I thought also about the nar-maven-plugin (github) just related by the
>> name... but i think this plugin goes a different direction but you might
>> better know than i do...
>>
>> The nar-maven-plugin (github) is intended to support building
>> (C/C++/Fortran) etc. with packaging the header files etc. into a archive
>> (NAR ;-)..which will be unpacked...(you know better than i do)...
>>
>>
>> On 12/14/14 8:41 PM, Benson Margulies wrote:
>>
>>> I'm a tiny bit active in the nar plugin on github. Do you have a fork of
>>> it? Are you interested in selling your fork back to the current
>>> maintainers?
>>>
>>>
>>  From my point of view it does not look like this cause the code is at the
>> moment very rudimentary.......
>>
>> May be others in the project can say more about the nar-maven-plugin which
>> intention is behind it?
>>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Benson Margulies
(begin semi-joke) or OSGi (end semi-joke)

On Sun, Dec 14, 2014 at 4:46 PM, Karl Heinz Marbaise <[hidden email]>
wrote:

> Hi Tony,
>
> sounds like a job for maven-shade-plugin or maven-assembly-plugin...
>
> kind regards
> Karl Heinz Marbaisea
>
> On 12/14/14 9:23 PM, Tony Kurc wrote:
>
>> They're the same in name only. The N of the NiFi nars is for "NiFi" versus
>> "native". I don't have a canned concise description, perhaps someone else
>> will weigh in, but they are for packing together code with dependent
>> libraries, and allowing different bundles to have different dependencies.
>> They are similar in ilk to the concept of wars. Two wars may have
>> different
>> versions, potentially conflicting versions of libraries. However, tomcat,
>> for example, can load the two web applications just fine.
>>
>> On Sun, Dec 14, 2014 at 2:58 PM, Karl Heinz Marbaise <[hidden email]>
>> wrote:
>>
>>>
>>> Hi Benson,
>>>
>>> I thought also about the nar-maven-plugin (github) just related by the
>>> name... but i think this plugin goes a different direction but you might
>>> better know than i do...
>>>
>>> The nar-maven-plugin (github) is intended to support building
>>> (C/C++/Fortran) etc. with packaging the header files etc. into a archive
>>> (NAR ;-)..which will be unpacked...(you know better than i do)...
>>>
>>>
>>> On 12/14/14 8:41 PM, Benson Margulies wrote:
>>>
>>>  I'm a tiny bit active in the nar plugin on github. Do you have a fork of
>>>> it? Are you interested in selling your fork back to the current
>>>> maintainers?
>>>>
>>>>
>>>>   From my point of view it does not look like this cause the code is at
>>> the
>>> moment very rudimentary.......
>>>
>>> May be others in the project can say more about the nar-maven-plugin
>>> which
>>> intention is behind it?
>>>
>>>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Joe Witt
OSGi was the most obvious choice at the time but we stayed away due to the
tooling/cost/benefit.

Our simple goal was for those building extensions to easily produce bundles
of their code and associated dependencies.

Building uber jars was generally considered non-desirable because we did
not want to change the structure of the extension or dependent jars.

The assembly plugin might in fact be totally sufficient for our purposes.
It requires some extra steps for the client but those are probably
completely automated by maven archetypes.

In any event we recognize the current nar concept is problematic because
nobody is going to want us publishing nar artifact bundles to maven repos.
So we're looking at what we should do.

Ideas on how best to achieve this in a Maven, Apache, and NiFi manner are
certainly welcome.



On Sun, Dec 14, 2014 at 4:53 PM, Benson Margulies <[hidden email]>
wrote:

>
> (begin semi-joke) or OSGi (end semi-joke)
>
> On Sun, Dec 14, 2014 at 4:46 PM, Karl Heinz Marbaise <[hidden email]>
> wrote:
>
> > Hi Tony,
> >
> > sounds like a job for maven-shade-plugin or maven-assembly-plugin...
> >
> > kind regards
> > Karl Heinz Marbaisea
> >
> > On 12/14/14 9:23 PM, Tony Kurc wrote:
> >
> >> They're the same in name only. The N of the NiFi nars is for "NiFi"
> versus
> >> "native". I don't have a canned concise description, perhaps someone
> else
> >> will weigh in, but they are for packing together code with dependent
> >> libraries, and allowing different bundles to have different
> dependencies.
> >> They are similar in ilk to the concept of wars. Two wars may have
> >> different
> >> versions, potentially conflicting versions of libraries. However,
> tomcat,
> >> for example, can load the two web applications just fine.
> >>
> >> On Sun, Dec 14, 2014 at 2:58 PM, Karl Heinz Marbaise <[hidden email]
> >
> >> wrote:
> >>
> >>>
> >>> Hi Benson,
> >>>
> >>> I thought also about the nar-maven-plugin (github) just related by the
> >>> name... but i think this plugin goes a different direction but you
> might
> >>> better know than i do...
> >>>
> >>> The nar-maven-plugin (github) is intended to support building
> >>> (C/C++/Fortran) etc. with packaging the header files etc. into a
> archive
> >>> (NAR ;-)..which will be unpacked...(you know better than i do)...
> >>>
> >>>
> >>> On 12/14/14 8:41 PM, Benson Margulies wrote:
> >>>
> >>>  I'm a tiny bit active in the nar plugin on github. Do you have a fork
> of
> >>>> it? Are you interested in selling your fork back to the current
> >>>> maintainers?
> >>>>
> >>>>
> >>>>   From my point of view it does not look like this cause the code is
> at
> >>> the
> >>> moment very rudimentary.......
> >>>
> >>> May be others in the project can say more about the nar-maven-plugin
> >>> which
> >>> intention is behind it?
> >>>
> >>>
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

trkurc
Administrator
why not host bundles off the download page?

On Sun, Dec 14, 2014 at 5:24 PM, Joe Witt <[hidden email]> wrote:

>
> OSGi was the most obvious choice at the time but we stayed away due to the
> tooling/cost/benefit.
>
> Our simple goal was for those building extensions to easily produce bundles
> of their code and associated dependencies.
>
> Building uber jars was generally considered non-desirable because we did
> not want to change the structure of the extension or dependent jars.
>
> The assembly plugin might in fact be totally sufficient for our purposes.
> It requires some extra steps for the client but those are probably
> completely automated by maven archetypes.
>
> In any event we recognize the current nar concept is problematic because
> nobody is going to want us publishing nar artifact bundles to maven repos.
> So we're looking at what we should do.
>
> Ideas on how best to achieve this in a Maven, Apache, and NiFi manner are
> certainly welcome.
>
>
>
> On Sun, Dec 14, 2014 at 4:53 PM, Benson Margulies <[hidden email]>
> wrote:
> >
> > (begin semi-joke) or OSGi (end semi-joke)
> >
> > On Sun, Dec 14, 2014 at 4:46 PM, Karl Heinz Marbaise <[hidden email]>
> > wrote:
> >
> > > Hi Tony,
> > >
> > > sounds like a job for maven-shade-plugin or maven-assembly-plugin...
> > >
> > > kind regards
> > > Karl Heinz Marbaisea
> > >
> > > On 12/14/14 9:23 PM, Tony Kurc wrote:
> > >
> > >> They're the same in name only. The N of the NiFi nars is for "NiFi"
> > versus
> > >> "native". I don't have a canned concise description, perhaps someone
> > else
> > >> will weigh in, but they are for packing together code with dependent
> > >> libraries, and allowing different bundles to have different
> > dependencies.
> > >> They are similar in ilk to the concept of wars. Two wars may have
> > >> different
> > >> versions, potentially conflicting versions of libraries. However,
> > tomcat,
> > >> for example, can load the two web applications just fine.
> > >>
> > >> On Sun, Dec 14, 2014 at 2:58 PM, Karl Heinz Marbaise <
> [hidden email]
> > >
> > >> wrote:
> > >>
> > >>>
> > >>> Hi Benson,
> > >>>
> > >>> I thought also about the nar-maven-plugin (github) just related by
> the
> > >>> name... but i think this plugin goes a different direction but you
> > might
> > >>> better know than i do...
> > >>>
> > >>> The nar-maven-plugin (github) is intended to support building
> > >>> (C/C++/Fortran) etc. with packaging the header files etc. into a
> > archive
> > >>> (NAR ;-)..which will be unpacked...(you know better than i do)...
> > >>>
> > >>>
> > >>> On 12/14/14 8:41 PM, Benson Margulies wrote:
> > >>>
> > >>>  I'm a tiny bit active in the nar plugin on github. Do you have a
> fork
> > of
> > >>>> it? Are you interested in selling your fork back to the current
> > >>>> maintainers?
> > >>>>
> > >>>>
> > >>>>   From my point of view it does not look like this cause the code is
> > at
> > >>> the
> > >>> moment very rudimentary.......
> > >>>
> > >>> May be others in the project can say more about the nar-maven-plugin
> > >>> which
> > >>> intention is behind it?
> > >>>
> > >>>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Sean Busbey
In reply to this post by Joe Witt
On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
>
> OSGi was the most obvious choice at the time but we stayed away due to the
> tooling/cost/benefit.
>

Might be worth revisiting. I know the specter of OSGi is raised every time
classpath woes surface in Hadoop-land.

> Our simple goal was for those building extensions to easily produce
bundles
> of their code and associated dependencies.
>
> Building uber jars was generally considered non-desirable because we did
> not want to change the structure of the extension or dependent jars.
>

+1 for continuing to avoid uber jars.

> The assembly plugin might in fact be totally sufficient for our purposes.
> It requires some extra steps for the client but those are probably
> completely automated by maven archetypes.
>

Reworking things so that nar becomes a kind of assembly is an interesting
idea. Do we have docs on the expectations the runtime has for a nar?

> In any event we recognize the current nar concept is problematic because
> nobody is going to want us publishing nar artifact bundles to maven repos.
> So we're looking at what we should do.
>

I don't see why publishing nars would be a problem. Other projects publish
tarballs  (source and binary) and things go well.
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

trkurc
Administrator
there seem to be plenty of wars which include dependencies in
repository.apache.org

On Sun, Dec 14, 2014 at 5:59 PM, Sean Busbey <[hidden email]> wrote:

>
> On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
> >
> > OSGi was the most obvious choice at the time but we stayed away due to
> the
> > tooling/cost/benefit.
> >
>
> Might be worth revisiting. I know the specter of OSGi is raised every time
> classpath woes surface in Hadoop-land.
>
> > Our simple goal was for those building extensions to easily produce
> bundles
> > of their code and associated dependencies.
> >
> > Building uber jars was generally considered non-desirable because we did
> > not want to change the structure of the extension or dependent jars.
> >
>
> +1 for continuing to avoid uber jars.
>
> > The assembly plugin might in fact be totally sufficient for our purposes.
> > It requires some extra steps for the client but those are probably
> > completely automated by maven archetypes.
> >
>
> Reworking things so that nar becomes a kind of assembly is an interesting
> idea. Do we have docs on the expectations the runtime has for a nar?
>
> > In any event we recognize the current nar concept is problematic because
> > nobody is going to want us publishing nar artifact bundles to maven
> repos.
> > So we're looking at what we should do.
> >
>
> I don't see why publishing nars would be a problem. Other projects publish
> tarballs  (source and binary) and things go well.
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Benson Margulies
I wouldn't reject Maven Central off-hand.

And I wouldn't reject keeping your own maven plugin and packaging, even if
it's a nearly complete fork of the maven-war-plugin.

I think that Maven's handling of 'packaging' and such is less than delux,
and leads in this direction. You'd like to be able to take advantage of the
underlying function of the war plugin (or the assembly plugin) and still
get DRY by avoiding having your devs need to copy a complex assembly
descriptor or do something funny-looking with the war plugin, but I don't
know a way. So if you want a nice, clean, pom for 'build me a NiFi plugin',
you end up, I suspect, with .. a maven plugin.

I've recently stuck my toes into the OSGi waters at work; I have a theory
of how to have an OSGi system while still cleaving closely to the Maven
dependency universe so as to avoid ending up adrift in OBR. The theory is
neither completely worked out nor proven in production, but I could write
it up such as it is. I'm still digesting some ideas from the Felix user
list.

Basically, I think of OSGi as a generalization of 'war', and then I am
trying to work out a low-labor way to keep track of all the dependencies.


On Sun, Dec 14, 2014 at 6:30 PM, Tony Kurc <[hidden email]> wrote:

> there seem to be plenty of wars which include dependencies in
> repository.apache.org
>
> On Sun, Dec 14, 2014 at 5:59 PM, Sean Busbey <[hidden email]> wrote:
> >
> > On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
> > >
> > > OSGi was the most obvious choice at the time but we stayed away due to
> > the
> > > tooling/cost/benefit.
> > >
> >
> > Might be worth revisiting. I know the specter of OSGi is raised every
> time
> > classpath woes surface in Hadoop-land.
> >
> > > Our simple goal was for those building extensions to easily produce
> > bundles
> > > of their code and associated dependencies.
> > >
> > > Building uber jars was generally considered non-desirable because we
> did
> > > not want to change the structure of the extension or dependent jars.
> > >
> >
> > +1 for continuing to avoid uber jars.
> >
> > > The assembly plugin might in fact be totally sufficient for our
> purposes.
> > > It requires some extra steps for the client but those are probably
> > > completely automated by maven archetypes.
> > >
> >
> > Reworking things so that nar becomes a kind of assembly is an interesting
> > idea. Do we have docs on the expectations the runtime has for a nar?
> >
> > > In any event we recognize the current nar concept is problematic
> because
> > > nobody is going to want us publishing nar artifact bundles to maven
> > repos.
> > > So we're looking at what we should do.
> > >
> >
> > I don't see why publishing nars would be a problem. Other projects
> publish
> > tarballs  (source and binary) and things go well.
> >
>
Reply | Threaded
Open this post in threaded view
|

RE: nar plugin

Mark Payne
I have very little experience with OSGi (I toyed with it at home when we discussed the introduction of NARs several years ago but that's about it). But I have quite a lot of knowledge about NARs, so I thought I'd point out the key features/design points. OSGi may or may not cover all of these concepts, but if we are thinking about OSGi as a possible alternative (or a parallel), I think it's good to go ahead and call out these concepts.
NARs map directly to a Java ClassLoader. Whatever your NAR depends on in its pom is what will be loaded into the Java ClassLoader. This makes understanding ClassLoading issues much easier, I think.
A NAR can declare a "Parent NAR." That means that the parent of its ClassLoader will be the ClassLoader of the parent NAR.
NARs can also provide a 'native' folder that houses native libraries. This provides native library isolation as well as class loading isolation.
You are able to define a Controller Service interface in a 'service-api' nar and then have that NAR be the parent of say my-processor-nar and my-service-nar. The Processor can then call methods in the Controller Service via the interface. Even though the implementation lives in my-service-nar, we can cross that NAR boundary safely since we only interact with the interface (which is defined in the common parent).

> Date: Sun, 14 Dec 2014 18:44:05 -0500
> Subject: Re: nar plugin
> From: [hidden email]
> To: [hidden email]
>
> I wouldn't reject Maven Central off-hand.
>
> And I wouldn't reject keeping your own maven plugin and packaging, even if
> it's a nearly complete fork of the maven-war-plugin.
>
> I think that Maven's handling of 'packaging' and such is less than delux,
> and leads in this direction. You'd like to be able to take advantage of the
> underlying function of the war plugin (or the assembly plugin) and still
> get DRY by avoiding having your devs need to copy a complex assembly
> descriptor or do something funny-looking with the war plugin, but I don't
> know a way. So if you want a nice, clean, pom for 'build me a NiFi plugin',
> you end up, I suspect, with .. a maven plugin.
>
> I've recently stuck my toes into the OSGi waters at work; I have a theory
> of how to have an OSGi system while still cleaving closely to the Maven
> dependency universe so as to avoid ending up adrift in OBR. The theory is
> neither completely worked out nor proven in production, but I could write
> it up such as it is. I'm still digesting some ideas from the Felix user
> list.
>
> Basically, I think of OSGi as a generalization of 'war', and then I am
> trying to work out a low-labor way to keep track of all the dependencies.
>
>
> On Sun, Dec 14, 2014 at 6:30 PM, Tony Kurc <[hidden email]> wrote:
>
> > there seem to be plenty of wars which include dependencies in
> > repository.apache.org
> >
> > On Sun, Dec 14, 2014 at 5:59 PM, Sean Busbey <[hidden email]> wrote:
> > >
> > > On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
> > > >
> > > > OSGi was the most obvious choice at the time but we stayed away due to
> > > the
> > > > tooling/cost/benefit.
> > > >
> > >
> > > Might be worth revisiting. I know the specter of OSGi is raised every
> > time
> > > classpath woes surface in Hadoop-land.
> > >
> > > > Our simple goal was for those building extensions to easily produce
> > > bundles
> > > > of their code and associated dependencies.
> > > >
> > > > Building uber jars was generally considered non-desirable because we
> > did
> > > > not want to change the structure of the extension or dependent jars.
> > > >
> > >
> > > +1 for continuing to avoid uber jars.
> > >
> > > > The assembly plugin might in fact be totally sufficient for our
> > purposes.
> > > > It requires some extra steps for the client but those are probably
> > > > completely automated by maven archetypes.
> > > >
> > >
> > > Reworking things so that nar becomes a kind of assembly is an interesting
> > > idea. Do we have docs on the expectations the runtime has for a nar?
> > >
> > > > In any event we recognize the current nar concept is problematic
> > because
> > > > nobody is going to want us publishing nar artifact bundles to maven
> > > repos.
> > > > So we're looking at what we should do.
> > > >
> > >
> > > I don't see why publishing nars would be a problem. Other projects
> > publish
> > > tarballs  (source and binary) and things go well.
> > >
> >
     
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Joe Witt
I think ultimately with Nar or OSGi you can achieve the same ends (most
likely).  Where the differences come in would be how easily a developer can
build, access, and reason over a bundle and the associated overhead/baggage
that comes with it at runtime.

Nars (the nifi kind) are purpose built for NiFi and so the overhead and
class loading model is very well understood and battle tested.  But the
truth is our plugin reflects our novice (copy and paste) level of knowledge
of the maven internals specifically as it relates to building plugins.

Nars did, and perhaps at the moment still do, have the right tradeoff.  But
OSGi by design should be the right choice and so we need to gain
familiarity or better yet do a good job of solving our other priorities and
attracting folks with expertise here who can help us get it right (though
Benson's notes are encouraging too).

I'm glad to hear folks aren't balking at the notion of deploying nar
bundles to maven repos.  That means we can prioritize other issues higher
(like the docs and having a build process that is more common and
approachable).

Thanks
Joe

On Sun, Dec 14, 2014 at 7:34 PM, Mark Payne <[hidden email]> wrote:

>
> I have very little experience with OSGi (I toyed with it at home when we
> discussed the introduction of NARs several years ago but that's about it).
> But I have quite a lot of knowledge about NARs, so I thought I'd point out
> the key features/design points. OSGi may or may not cover all of these
> concepts, but if we are thinking about OSGi as a possible alternative (or a
> parallel), I think it's good to go ahead and call out these concepts.
> NARs map directly to a Java ClassLoader. Whatever your NAR depends on in
> its pom is what will be loaded into the Java ClassLoader. This makes
> understanding ClassLoading issues much easier, I think.
> A NAR can declare a "Parent NAR." That means that the parent of its
> ClassLoader will be the ClassLoader of the parent NAR.
> NARs can also provide a 'native' folder that houses native libraries. This
> provides native library isolation as well as class loading isolation.
> You are able to define a Controller Service interface in a 'service-api'
> nar and then have that NAR be the parent of say my-processor-nar and
> my-service-nar. The Processor can then call methods in the Controller
> Service via the interface. Even though the implementation lives in
> my-service-nar, we can cross that NAR boundary safely since we only
> interact with the interface (which is defined in the common parent).
>
> > Date: Sun, 14 Dec 2014 18:44:05 -0500
> > Subject: Re: nar plugin
> > From: [hidden email]
> > To: [hidden email]
> >
> > I wouldn't reject Maven Central off-hand.
> >
> > And I wouldn't reject keeping your own maven plugin and packaging, even
> if
> > it's a nearly complete fork of the maven-war-plugin.
> >
> > I think that Maven's handling of 'packaging' and such is less than delux,
> > and leads in this direction. You'd like to be able to take advantage of
> the
> > underlying function of the war plugin (or the assembly plugin) and still
> > get DRY by avoiding having your devs need to copy a complex assembly
> > descriptor or do something funny-looking with the war plugin, but I don't
> > know a way. So if you want a nice, clean, pom for 'build me a NiFi
> plugin',
> > you end up, I suspect, with .. a maven plugin.
> >
> > I've recently stuck my toes into the OSGi waters at work; I have a theory
> > of how to have an OSGi system while still cleaving closely to the Maven
> > dependency universe so as to avoid ending up adrift in OBR. The theory is
> > neither completely worked out nor proven in production, but I could write
> > it up such as it is. I'm still digesting some ideas from the Felix user
> > list.
> >
> > Basically, I think of OSGi as a generalization of 'war', and then I am
> > trying to work out a low-labor way to keep track of all the dependencies.
> >
> >
> > On Sun, Dec 14, 2014 at 6:30 PM, Tony Kurc <[hidden email]> wrote:
> >
> > > there seem to be plenty of wars which include dependencies in
> > > repository.apache.org
> > >
> > > On Sun, Dec 14, 2014 at 5:59 PM, Sean Busbey <[hidden email]>
> wrote:
> > > >
> > > > On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
> > > > >
> > > > > OSGi was the most obvious choice at the time but we stayed away
> due to
> > > > the
> > > > > tooling/cost/benefit.
> > > > >
> > > >
> > > > Might be worth revisiting. I know the specter of OSGi is raised every
> > > time
> > > > classpath woes surface in Hadoop-land.
> > > >
> > > > > Our simple goal was for those building extensions to easily produce
> > > > bundles
> > > > > of their code and associated dependencies.
> > > > >
> > > > > Building uber jars was generally considered non-desirable because
> we
> > > did
> > > > > not want to change the structure of the extension or dependent
> jars.
> > > > >
> > > >
> > > > +1 for continuing to avoid uber jars.
> > > >
> > > > > The assembly plugin might in fact be totally sufficient for our
> > > purposes.
> > > > > It requires some extra steps for the client but those are probably
> > > > > completely automated by maven archetypes.
> > > > >
> > > >
> > > > Reworking things so that nar becomes a kind of assembly is an
> interesting
> > > > idea. Do we have docs on the expectations the runtime has for a nar?
> > > >
> > > > > In any event we recognize the current nar concept is problematic
> > > because
> > > > > nobody is going to want us publishing nar artifact bundles to maven
> > > > repos.
> > > > > So we're looking at what we should do.
> > > > >
> > > >
> > > > I don't see why publishing nars would be a problem. Other projects
> > > publish
> > > > tarballs  (source and binary) and things go well.
> > > >
> > >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Karl Heinz Marbaise
In reply to this post by Benson Margulies
Hi,

On 12/15/14 12:44 AM, Benson Margulies wrote:

> I wouldn't reject Maven Central off-hand.
>
> And I wouldn't reject keeping your own maven plugin and packaging, even if
> it's a nearly complete fork of the maven-war-plugin.
>
> I think that Maven's handling of 'packaging' and such is less than delux,
> and leads in this direction. You'd like to be able to take advantage of the
> underlying function of the war plugin (or the assembly plugin) and still
> get DRY by avoiding having your devs need to copy a complex assembly
> descriptor or do something funny-looking with the war plugin, but I don't
> know a way. So if you want a nice, clean, pom for 'build me a NiFi plugin',
> you end up, I suspect, with .. a maven plugin.

If it fit the needs of the project better than other ways so of
course..go for it...I can offer my help.....

BTW: Something about the descriptor comes into my mind. I'm not sure if
this could be solution. Just define an complex descriptor and use it as
a remote-resource or as a predefined descriptor (works in
maven-assembly-plugin if it's the right choice). So only once you have
to write up such a complex descriptor and all other use it...but it
might not that comfortable in comparsion to a full fledged plugin...


>
> I've recently stuck my toes into the OSGi waters at work; I have a theory
> of how to have an OSGi system while still cleaving closely to the Maven
> dependency universe so as to avoid ending up adrift in OBR. The theory is
> neither completely worked out nor proven in production, but I could write
> it up such as it is. I'm still digesting some ideas from the Felix user
> list.
>
> Basically, I think of OSGi as a generalization of 'war', and then I am
> trying to work out a low-labor way to keep track of all the dependencies.
>
>
> On Sun, Dec 14, 2014 at 6:30 PM, Tony Kurc <[hidden email]> wrote:
>
>> there seem to be plenty of wars which include dependencies in
>> repository.apache.org
>>
>> On Sun, Dec 14, 2014 at 5:59 PM, Sean Busbey <[hidden email]> wrote:
>>>
>>> On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
>>>>
>>>> OSGi was the most obvious choice at the time but we stayed away due to
>>> the
>>>> tooling/cost/benefit.
>>>>
>>>
>>> Might be worth revisiting. I know the specter of OSGi is raised every
>> time
>>> classpath woes surface in Hadoop-land.
>>>
>>>> Our simple goal was for those building extensions to easily produce
>>> bundles
>>>> of their code and associated dependencies.
>>>>
>>>> Building uber jars was generally considered non-desirable because we
>> did
>>>> not want to change the structure of the extension or dependent jars.
>>>>
>>>
>>> +1 for continuing to avoid uber jars.
>>>
>>>> The assembly plugin might in fact be totally sufficient for our
>> purposes.
>>>> It requires some extra steps for the client but those are probably
>>>> completely automated by maven archetypes.
>>>>
>>>
>>> Reworking things so that nar becomes a kind of assembly is an interesting
>>> idea. Do we have docs on the expectations the runtime has for a nar?
>>>
>>>> In any event we recognize the current nar concept is problematic
>> because
>>>> nobody is going to want us publishing nar artifact bundles to maven
>>> repos.
>>>> So we're looking at what we should do.
>>>>
>>>
>>> I don't see why publishing nars would be a problem. Other projects
>> publish
>>> tarballs  (source and binary) and things go well.

Yes that's absolutely true...


Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Adam Taft
In reply to this post by Joe Witt
I definitely agree that we can use guidance for any consideration or
implementation of OSGi. I am definitely no OSGi expert, but at least for
the past couple of years, I have spent a reasonable amount of time
exploring the use of OSGi within/around the existing NiFi framework. Here
are some observations (hopefully as apple-to-apple as possible), for what
they're worth:

-  Both NiFi and OSGi are modular application frameworks, defining a plugin
component model realized by the NAR (NiFi Archive) format and the OSGi
bundle format, respectively.

-  Both frameworks are able to find, load and run components at application
startup.  OSGi can additionally support dynamic loading and unloading at
any point during runtime.

-  Both provide module independence via classloader isolation.

-  Both implement a service layer architecture, allowing modules to
interact outside of their isolation container. OSGi has an extremely mature
service registration, discovery and binding architecture, given this was
one of its primary target areas.  NiFi's service architecture approach is
limited and immature in comparison; several opportunities for growth can be
easily recognized.

-  The NAR format allows for a single parentage, effectively required (but
limiting) for service discovery.  OSGi does not define any restriction
between modules and services, since any registered service can be
discovered by any module (many-to-many).

-  The NAR format, being very similar to WAR, includes all dependencies
inside of the archive.  Multiple copies of the same base library (for
example, any Apache Commons jar) are loaded into each isolated classloader
space; NAR and WAR are very analogous in this way.

In contrast, OSGi bundles do not traditionally include dependencies as part
of the bundle archive.  Instead with OSGi, only a single copy of any common
library is maintained for the entire application framework.  i.e. one and
only one copy of any commons jar is ever loaded.  Additionally, OSGi
provides version conflict resolution, somewhat similar to maven conflict
resolution.  OSGi benefits from reduced permgen usage loading due to these
features.

-  NiFi requires all NAR modules to be available locally at startup.  OSGi
containers generally support dynamic loading of modules and services from
remote repositories (OBR or OSGi Bundle Repository). OSGi bundles can be
easily added or updated from an OBR, including all transitive dependencies.

Note that usage of an OBR for dependency resolution, however, can be an
impediment in environments without network connectivity to such external
repositories.  Manual download and local install would still likely be
required on any restricted network environment. (Nexus can act as an OBR,
though.)

-  Both provide module lifecycle support.  However, OSGi lifecycle support
is geared more towards module or service loading/unloading; whereas, NAR
lifecycle support in NiFi is geared to processor level lifecycle events.
Limited lifecycle control is provided at the NAR level itself.

-  OSGi defines dependencies and service interfaces via bundle declarations
in the MANIFEST.MF file.  NAR modules are self-contained and require no
META-INF information.  NARs are arguably simpler to build, though OSGi
related maven plugins can help.

In summary, the NiFi framework has a lot of features that OSGi provides,
particularly around application startup, module loading, classloader
isolation, service discovery, etc.  OSGi gets "right" many of the concepts
that NiFi has grown into.

While I don't believe any near-term release of NiFi could utilize OSGi, it
is something that could likely be considered for a major (definitely
non-backwards compatible) release in the future.  It won't come without
significant investment in time, as many parts in the "guts" of NiFi will be
affected.  Allowing OSGi compatible NARs (and the processors contained) to
be loaded/unloaded at anytime during runtime needs to be thought through
carefully.

Obviously, NiFi should refrain from "reinventing the wheel" and "not
invented here" in places where a well recognized platform is already
available. Therefore it's possible that OSGi could be used for many of the
problems which NiFi addresses today.  Most of the NiFi concepts mentioned
above can be implemented on top of OSGi. This would free NiFi to instead
focus on its true value proposition, namely data flow management.

Food for thought.  Hope this was useful.

Adam





On Sun, Dec 14, 2014 at 6:08 PM, Joe Witt <[hidden email]> wrote:

>
> I think ultimately with Nar or OSGi you can achieve the same ends (most
> likely).  Where the differences come in would be how easily a developer can
> build, access, and reason over a bundle and the associated overhead/baggage
> that comes with it at runtime.
>
> Nars (the nifi kind) are purpose built for NiFi and so the overhead and
> class loading model is very well understood and battle tested.  But the
> truth is our plugin reflects our novice (copy and paste) level of knowledge
> of the maven internals specifically as it relates to building plugins.
>
> Nars did, and perhaps at the moment still do, have the right tradeoff.  But
> OSGi by design should be the right choice and so we need to gain
> familiarity or better yet do a good job of solving our other priorities and
> attracting folks with expertise here who can help us get it right (though
> Benson's notes are encouraging too).
>
> I'm glad to hear folks aren't balking at the notion of deploying nar
> bundles to maven repos.  That means we can prioritize other issues higher
> (like the docs and having a build process that is more common and
> approachable).
>
> Thanks
> Joe
>
> On Sun, Dec 14, 2014 at 7:34 PM, Mark Payne <[hidden email]> wrote:
> >
> > I have very little experience with OSGi (I toyed with it at home when we
> > discussed the introduction of NARs several years ago but that's about
> it).
> > But I have quite a lot of knowledge about NARs, so I thought I'd point
> out
> > the key features/design points. OSGi may or may not cover all of these
> > concepts, but if we are thinking about OSGi as a possible alternative
> (or a
> > parallel), I think it's good to go ahead and call out these concepts.
> > NARs map directly to a Java ClassLoader. Whatever your NAR depends on in
> > its pom is what will be loaded into the Java ClassLoader. This makes
> > understanding ClassLoading issues much easier, I think.
> > A NAR can declare a "Parent NAR." That means that the parent of its
> > ClassLoader will be the ClassLoader of the parent NAR.
> > NARs can also provide a 'native' folder that houses native libraries.
> This
> > provides native library isolation as well as class loading isolation.
> > You are able to define a Controller Service interface in a 'service-api'
> > nar and then have that NAR be the parent of say my-processor-nar and
> > my-service-nar. The Processor can then call methods in the Controller
> > Service via the interface. Even though the implementation lives in
> > my-service-nar, we can cross that NAR boundary safely since we only
> > interact with the interface (which is defined in the common parent).
> >
> > > Date: Sun, 14 Dec 2014 18:44:05 -0500
> > > Subject: Re: nar plugin
> > > From: [hidden email]
> > > To: [hidden email]
> > >
> > > I wouldn't reject Maven Central off-hand.
> > >
> > > And I wouldn't reject keeping your own maven plugin and packaging, even
> > if
> > > it's a nearly complete fork of the maven-war-plugin.
> > >
> > > I think that Maven's handling of 'packaging' and such is less than
> delux,
> > > and leads in this direction. You'd like to be able to take advantage of
> > the
> > > underlying function of the war plugin (or the assembly plugin) and
> still
> > > get DRY by avoiding having your devs need to copy a complex assembly
> > > descriptor or do something funny-looking with the war plugin, but I
> don't
> > > know a way. So if you want a nice, clean, pom for 'build me a NiFi
> > plugin',
> > > you end up, I suspect, with .. a maven plugin.
> > >
> > > I've recently stuck my toes into the OSGi waters at work; I have a
> theory
> > > of how to have an OSGi system while still cleaving closely to the Maven
> > > dependency universe so as to avoid ending up adrift in OBR. The theory
> is
> > > neither completely worked out nor proven in production, but I could
> write
> > > it up such as it is. I'm still digesting some ideas from the Felix user
> > > list.
> > >
> > > Basically, I think of OSGi as a generalization of 'war', and then I am
> > > trying to work out a low-labor way to keep track of all the
> dependencies.
> > >
> > >
> > > On Sun, Dec 14, 2014 at 6:30 PM, Tony Kurc <[hidden email]> wrote:
> > >
> > > > there seem to be plenty of wars which include dependencies in
> > > > repository.apache.org
> > > >
> > > > On Sun, Dec 14, 2014 at 5:59 PM, Sean Busbey <[hidden email]>
> > wrote:
> > > > >
> > > > > On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
> > > > > >
> > > > > > OSGi was the most obvious choice at the time but we stayed away
> > due to
> > > > > the
> > > > > > tooling/cost/benefit.
> > > > > >
> > > > >
> > > > > Might be worth revisiting. I know the specter of OSGi is raised
> every
> > > > time
> > > > > classpath woes surface in Hadoop-land.
> > > > >
> > > > > > Our simple goal was for those building extensions to easily
> produce
> > > > > bundles
> > > > > > of their code and associated dependencies.
> > > > > >
> > > > > > Building uber jars was generally considered non-desirable because
> > we
> > > > did
> > > > > > not want to change the structure of the extension or dependent
> > jars.
> > > > > >
> > > > >
> > > > > +1 for continuing to avoid uber jars.
> > > > >
> > > > > > The assembly plugin might in fact be totally sufficient for our
> > > > purposes.
> > > > > > It requires some extra steps for the client but those are
> probably
> > > > > > completely automated by maven archetypes.
> > > > > >
> > > > >
> > > > > Reworking things so that nar becomes a kind of assembly is an
> > interesting
> > > > > idea. Do we have docs on the expectations the runtime has for a
> nar?
> > > > >
> > > > > > In any event we recognize the current nar concept is problematic
> > > > because
> > > > > > nobody is going to want us publishing nar artifact bundles to
> maven
> > > > > repos.
> > > > > > So we're looking at what we should do.
> > > > > >
> > > > >
> > > > > I don't see why publishing nars would be a problem. Other projects
> > > > publish
> > > > > tarballs  (source and binary) and things go well.
> > > > >
> > > >
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Benson Margulies
On Mon, Dec 15, 2014 at 2:42 AM, Adam Taft <[hidden email]> wrote:

> I definitely agree that we can use guidance for any consideration or
> implementation of OSGi. I am definitely no OSGi expert, but at least for
> the past couple of years, I have spent a reasonable amount of time
> exploring the use of OSGi within/around the existing NiFi framework. Here
> are some observations (hopefully as apple-to-apple as possible), for what
> they're worth:
>
> -  Both NiFi and OSGi are modular application frameworks, defining a plugin
> component model realized by the NAR (NiFi Archive) format and the OSGi
> bundle format, respectively.
>
> -  Both frameworks are able to find, load and run components at application
> startup.  OSGi can additionally support dynamic loading and unloading at
> any point during runtime.
>
> -  Both provide module independence via classloader isolation.
>
> -  Both implement a service layer architecture, allowing modules to
> interact outside of their isolation container. OSGi has an extremely mature
> service registration, discovery and binding architecture, given this was
> one of its primary target areas.  NiFi's service architecture approach is
> limited and immature in comparison; several opportunities for growth can be
> easily recognized.
>
> -  The NAR format allows for a single parentage, effectively required (but
> limiting) for service discovery.  OSGi does not define any restriction
> between modules and services, since any registered service can be
> discovered by any module (many-to-many).
>
> -  The NAR format, being very similar to WAR, includes all dependencies
> inside of the archive.  Multiple copies of the same base library (for
> example, any Apache Commons jar) are loaded into each isolated classloader
> space; NAR and WAR are very analogous in this way.
>
> In contrast, OSGi bundles do not traditionally include dependencies as part
> of the bundle archive.  Instead with OSGi, only a single copy of any common
> library is maintained for the entire application framework.  i.e. one and
> only one copy of any commons jar is ever loaded.  Additionally, OSGi
> provides version conflict resolution, somewhat similar to maven conflict
> resolution.  OSGi benefits from reduced permgen usage loading due to these
> features.
>
> -  NiFi requires all NAR modules to be available locally at startup.  OSGi
> containers generally support dynamic loading of modules and services from
> remote repositories (OBR or OSGi Bundle Repository). OSGi bundles can be
> easily added or updated from an OBR, including all transitive dependencies.
>
> Note that usage of an OBR for dependency resolution, however, can be an
> impediment in environments without network connectivity to such external
> repositories.  Manual download and local install would still likely be
> required on any restricted network environment. (Nexus can act as an OBR,
> though.)
>
> -  Both provide module lifecycle support.  However, OSGi lifecycle support
> is geared more towards module or service loading/unloading; whereas, NAR
> lifecycle support in NiFi is geared to processor level lifecycle events.
> Limited lifecycle control is provided at the NAR level itself.
>
> -  OSGi defines dependencies and service interfaces via bundle declarations
> in the MANIFEST.MF file.  NAR modules are self-contained and require no
> META-INF information.  NARs are arguably simpler to build, though OSGi
> related maven plugins can help.
>
> In summary, the NiFi framework has a lot of features that OSGi provides,
> particularly around application startup, module loading, classloader
> isolation, service discovery, etc.  OSGi gets "right" many of the concepts
> that NiFi has grown into.
>
> While I don't believe any near-term release of NiFi could utilize OSGi, it
> is something that could likely be considered for a major (definitely
> non-backwards compatible) release in the future.  It won't come without
> significant investment in time, as many parts in the "guts" of NiFi will be
> affected.  Allowing OSGi compatible NARs (and the processors contained) to
> be loaded/unloaded at anytime during runtime needs to be thought through
> carefully.
>
> Obviously, NiFi should refrain from "reinventing the wheel" and "not
> invented here" in places where a well recognized platform is already
> available. Therefore it's possible that OSGi could be used for many of the
> problems which NiFi addresses today.  Most of the NiFi concepts mentioned
> above can be implemented on top of OSGi. This would free NiFi to instead
> focus on its true value proposition, namely data flow management.
>
> Food for thought.  Hope this was useful.
>
> Adam
>
> I'm somewhat sorry that I started this rabbit running. I think Adam's
summary is quite comprehensive. I am not on a mission to convince this
group to trade in NAR for OSGi; this started as a discussion of howcome
there's a maven plugin, and I think that question is now fully answered. If
those of you close to the requirements want to explore some of the
alternative trade-offs enabled by OSGi, I'd be happy to help, but I think
that this is a rather large 'if'.


>
>
>
>
> On Sun, Dec 14, 2014 at 6:08 PM, Joe Witt <[hidden email]> wrote:
> >
> > I think ultimately with Nar or OSGi you can achieve the same ends (most
> > likely).  Where the differences come in would be how easily a developer
> can
> > build, access, and reason over a bundle and the associated
> overhead/baggage
> > that comes with it at runtime.
> >
> > Nars (the nifi kind) are purpose built for NiFi and so the overhead and
> > class loading model is very well understood and battle tested.  But the
> > truth is our plugin reflects our novice (copy and paste) level of
> knowledge
> > of the maven internals specifically as it relates to building plugins.
> >
> > Nars did, and perhaps at the moment still do, have the right tradeoff.
> But
> > OSGi by design should be the right choice and so we need to gain
> > familiarity or better yet do a good job of solving our other priorities
> and
> > attracting folks with expertise here who can help us get it right (though
> > Benson's notes are encouraging too).
> >
> > I'm glad to hear folks aren't balking at the notion of deploying nar
> > bundles to maven repos.  That means we can prioritize other issues higher
> > (like the docs and having a build process that is more common and
> > approachable).
> >
> > Thanks
> > Joe
> >
> > On Sun, Dec 14, 2014 at 7:34 PM, Mark Payne <[hidden email]>
> wrote:
> > >
> > > I have very little experience with OSGi (I toyed with it at home when
> we
> > > discussed the introduction of NARs several years ago but that's about
> > it).
> > > But I have quite a lot of knowledge about NARs, so I thought I'd point
> > out
> > > the key features/design points. OSGi may or may not cover all of these
> > > concepts, but if we are thinking about OSGi as a possible alternative
> > (or a
> > > parallel), I think it's good to go ahead and call out these concepts.
> > > NARs map directly to a Java ClassLoader. Whatever your NAR depends on
> in
> > > its pom is what will be loaded into the Java ClassLoader. This makes
> > > understanding ClassLoading issues much easier, I think.
> > > A NAR can declare a "Parent NAR." That means that the parent of its
> > > ClassLoader will be the ClassLoader of the parent NAR.
> > > NARs can also provide a 'native' folder that houses native libraries.
> > This
> > > provides native library isolation as well as class loading isolation.
> > > You are able to define a Controller Service interface in a
> 'service-api'
> > > nar and then have that NAR be the parent of say my-processor-nar and
> > > my-service-nar. The Processor can then call methods in the Controller
> > > Service via the interface. Even though the implementation lives in
> > > my-service-nar, we can cross that NAR boundary safely since we only
> > > interact with the interface (which is defined in the common parent).
> > >
> > > > Date: Sun, 14 Dec 2014 18:44:05 -0500
> > > > Subject: Re: nar plugin
> > > > From: [hidden email]
> > > > To: [hidden email]
> > > >
> > > > I wouldn't reject Maven Central off-hand.
> > > >
> > > > And I wouldn't reject keeping your own maven plugin and packaging,
> even
> > > if
> > > > it's a nearly complete fork of the maven-war-plugin.
> > > >
> > > > I think that Maven's handling of 'packaging' and such is less than
> > delux,
> > > > and leads in this direction. You'd like to be able to take advantage
> of
> > > the
> > > > underlying function of the war plugin (or the assembly plugin) and
> > still
> > > > get DRY by avoiding having your devs need to copy a complex assembly
> > > > descriptor or do something funny-looking with the war plugin, but I
> > don't
> > > > know a way. So if you want a nice, clean, pom for 'build me a NiFi
> > > plugin',
> > > > you end up, I suspect, with .. a maven plugin.
> > > >
> > > > I've recently stuck my toes into the OSGi waters at work; I have a
> > theory
> > > > of how to have an OSGi system while still cleaving closely to the
> Maven
> > > > dependency universe so as to avoid ending up adrift in OBR. The
> theory
> > is
> > > > neither completely worked out nor proven in production, but I could
> > write
> > > > it up such as it is. I'm still digesting some ideas from the Felix
> user
> > > > list.
> > > >
> > > > Basically, I think of OSGi as a generalization of 'war', and then I
> am
> > > > trying to work out a low-labor way to keep track of all the
> > dependencies.
> > > >
> > > >
> > > > On Sun, Dec 14, 2014 at 6:30 PM, Tony Kurc <[hidden email]> wrote:
> > > >
> > > > > there seem to be plenty of wars which include dependencies in
> > > > > repository.apache.org
> > > > >
> > > > > On Sun, Dec 14, 2014 at 5:59 PM, Sean Busbey <[hidden email]>
> > > wrote:
> > > > > >
> > > > > > On Dec 14, 2014 4:25 PM, "Joe Witt" <[hidden email]> wrote:
> > > > > > >
> > > > > > > OSGi was the most obvious choice at the time but we stayed away
> > > due to
> > > > > > the
> > > > > > > tooling/cost/benefit.
> > > > > > >
> > > > > >
> > > > > > Might be worth revisiting. I know the specter of OSGi is raised
> > every
> > > > > time
> > > > > > classpath woes surface in Hadoop-land.
> > > > > >
> > > > > > > Our simple goal was for those building extensions to easily
> > produce
> > > > > > bundles
> > > > > > > of their code and associated dependencies.
> > > > > > >
> > > > > > > Building uber jars was generally considered non-desirable
> because
> > > we
> > > > > did
> > > > > > > not want to change the structure of the extension or dependent
> > > jars.
> > > > > > >
> > > > > >
> > > > > > +1 for continuing to avoid uber jars.
> > > > > >
> > > > > > > The assembly plugin might in fact be totally sufficient for our
> > > > > purposes.
> > > > > > > It requires some extra steps for the client but those are
> > probably
> > > > > > > completely automated by maven archetypes.
> > > > > > >
> > > > > >
> > > > > > Reworking things so that nar becomes a kind of assembly is an
> > > interesting
> > > > > > idea. Do we have docs on the expectations the runtime has for a
> > nar?
> > > > > >
> > > > > > > In any event we recognize the current nar concept is
> problematic
> > > > > because
> > > > > > > nobody is going to want us publishing nar artifact bundles to
> > maven
> > > > > > repos.
> > > > > > > So we're looking at what we should do.
> > > > > > >
> > > > > >
> > > > > > I don't see why publishing nars would be a problem. Other
> projects
> > > > > publish
> > > > > > tarballs  (source and binary) and things go well.
> > > > > >
> > > > >
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Drew Farris
In reply to this post by Adam Taft
Thanks for the excellent summary Adam. I'm not very familiar with OSGi at a
deep level.

I find the point re dependency management point below pretty interesting.
One of the chief reasons I'd look to using OSGi would be to allow multiple
versions of the same dependency to be used by isolated modules. I don't
always enjoy manually de-conflicting version incompatibilities and would
prefer isolation in some cases.

On Mon, Dec 15, 2014 at 2:42 AM, Adam Taft <[hidden email]> wrote:

>
> In contrast, OSGi bundles do not traditionally include dependencies as part
> of the bundle archive.  Instead with OSGi, only a single copy of any common
> library is maintained for the entire application framework.  i.e. one and
> only one copy of any commons jar is ever loaded.  Additionally, OSGi
> provides version conflict resolution, somewhat similar to maven conflict
> resolution.  OSGi benefits from reduced permgen usage loading due to these
> features.
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: nar plugin

Joe Witt
If there are any particularly expert OSGi folks out there that want to help
rock NiFi's world please let us know.  We have Nars which give us isolation
and help devs focus on building isolated bundles of functional joy but I'd
bet you could convince/motivate a move to OSGi if it can be done nicely.

I think we're good with a bit of growing pains provided the developer
experience to build extensions is awesome.  The bottom line goal is making
it really easy for people to build extensions.

On Mon, Dec 15, 2014 at 3:01 PM, Drew Farris <[hidden email]> wrote:

>
> Thanks for the excellent summary Adam. I'm not very familiar with OSGi at a
> deep level.
>
> I find the point re dependency management point below pretty interesting.
> One of the chief reasons I'd look to using OSGi would be to allow multiple
> versions of the same dependency to be used by isolated modules. I don't
> always enjoy manually de-conflicting version incompatibilities and would
> prefer isolation in some cases.
>
> On Mon, Dec 15, 2014 at 2:42 AM, Adam Taft <[hidden email]> wrote:
>
> >
> > In contrast, OSGi bundles do not traditionally include dependencies as
> part
> > of the bundle archive.  Instead with OSGi, only a single copy of any
> common
> > library is maintained for the entire application framework.  i.e. one and
> > only one copy of any commons jar is ever loaded.  Additionally, OSGi
> > provides version conflict resolution, somewhat similar to maven conflict
> > resolution.  OSGi benefits from reduced permgen usage loading due to
> these
> > features.
> >
> >
> >
>