[Discuss] preparing for release 0.0.1

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[Discuss] preparing for release 0.0.1

Joe Witt
Folks,

Looking at the tickets that remain which are presently tied to 0.0.1 we're
probably 1-2 weeks out from this initial release.  Can you provide some
pointers/references or pointers on how to get this ball rolling and any
rocks we must move before going down this path?

http://incubator.apache.org/guides/releasemanagement.html

That link seems really helpful but has a lot of TODOs for areas which need
more explanation.


Thanks
Joe
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Brock Noland
Hi,

This is a decent guide which can be copied:

http://mrunit.apache.org/pmc/how_to_release.html

Brock

On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt <[hidden email]> wrote:

>
> Folks,
>
> Looking at the tickets that remain which are presently tied to 0.0.1 we're
> probably 1-2 weeks out from this initial release.  Can you provide some
> pointers/references or pointers on how to get this ball rolling and any
> rocks we must move before going down this path?
>
> http://incubator.apache.org/guides/releasemanagement.html
>
> That link seems really helpful but has a lot of TODOs for areas which need
> more explanation.
>
>
> Thanks
> Joe
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joe Witt
Thanks Brock this is very helpful.

On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland <[hidden email]> wrote:

>
> Hi,
>
> This is a decent guide which can be copied:
>
> http://mrunit.apache.org/pmc/how_to_release.html
>
> Brock
>
> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt <[hidden email]> wrote:
> >
> > Folks,
> >
> > Looking at the tickets that remain which are presently tied to 0.0.1
> we're
> > probably 1-2 weeks out from this initial release.  Can you provide some
> > pointers/references or pointers on how to get this ball rolling and any
> > rocks we must move before going down this path?
> >
> > http://incubator.apache.org/guides/releasemanagement.html
> >
> > That link seems really helpful but has a lot of TODOs for areas which
> need
> > more explanation.
> >
> >
> > Thanks
> > Joe
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joe Witt
Hello

Just wanted to stir this one up a bit.  Looks like all tickets pegged as
0.0.1 are resolved (2 remain as of now but seem likely to be resolved
shortly based on their comments).  So working through the release steps
available on apache.org and via the link Brock sent.

Anyone interested in this part of the process or who has advice to help us
avoid landmines we're happy to hear it.

Thanks
Joe

On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt <[hidden email]> wrote:

> Thanks Brock this is very helpful.
>
> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland <[hidden email]> wrote:
>>
>> Hi,
>>
>> This is a decent guide which can be copied:
>>
>> http://mrunit.apache.org/pmc/how_to_release.html
>>
>> Brock
>>
>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt <[hidden email]> wrote:
>> >
>> > Folks,
>> >
>> > Looking at the tickets that remain which are presently tied to 0.0.1
>> we're
>> > probably 1-2 weeks out from this initial release.  Can you provide some
>> > pointers/references or pointers on how to get this ball rolling and any
>> > rocks we must move before going down this path?
>> >
>> > http://incubator.apache.org/guides/releasemanagement.html
>> >
>> > That link seems really helpful but has a lot of TODOs for areas which
>> need
>> > more explanation.
>> >
>> >
>> > Thanks
>> > Joe
>> >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

trkurc
Administrator
I read the guide Joe linked and a lot of the sticky parts are marked "TODO"
and it looks like a work in progress

"Podlings can short circuit this process by starting out with written
release documentation. It is strongly recommended that Podlings invest time
looking at the best practices recommended in this document. By selection
and modification, sections of this document can be used to quickly and
easily bootstrap a release guide. "

Is step one putting together a release guide?



On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt <[hidden email]> wrote:

> Hello
>
> Just wanted to stir this one up a bit.  Looks like all tickets pegged as
> 0.0.1 are resolved (2 remain as of now but seem likely to be resolved
> shortly based on their comments).  So working through the release steps
> available on apache.org and via the link Brock sent.
>
> Anyone interested in this part of the process or who has advice to help us
> avoid landmines we're happy to hear it.
>
> Thanks
> Joe
>
> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt <[hidden email]> wrote:
>
> > Thanks Brock this is very helpful.
> >
> > On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland <[hidden email]>
> wrote:
> >>
> >> Hi,
> >>
> >> This is a decent guide which can be copied:
> >>
> >> http://mrunit.apache.org/pmc/how_to_release.html
> >>
> >> Brock
> >>
> >> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt <[hidden email]> wrote:
> >> >
> >> > Folks,
> >> >
> >> > Looking at the tickets that remain which are presently tied to 0.0.1
> >> we're
> >> > probably 1-2 weeks out from this initial release.  Can you provide
> some
> >> > pointers/references or pointers on how to get this ball rolling and
> any
> >> > rocks we must move before going down this path?
> >> >
> >> > http://incubator.apache.org/guides/releasemanagement.html
> >> >
> >> > That link seems really helpful but has a lot of TODOs for areas which
> >> need
> >> > more explanation.
> >> >
> >> >
> >> > Thanks
> >> > Joe
> >> >
> >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Josh Elser
Regardless of what you call it, writing down exactly what was done to
make a RC is extremely important early on (I know that I sure can't
remember what I did last week, much less last release). I'm not sure if
release guide is formally defined somewhere, but enough information that
any other committer can come by after "you get hit by a train" and make
a release is extremely important. Using the CMS and making a page on the
website for this is extremely easy to do, IMO.

Another concern for how to actually make a release is the type of
packaging that is released. I not talking about the source/binary
release here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a
tarball? Are there jars in Maven? etc. A tarball is pretty easy to make,
and likely the closest to what the build-order.sh script already does
(with Maven's help). I'm not sure if maven-released artifacts are quite
as important at this point -- if it's not, punting on that will help.
If/when you get to that point, look into the apache parent pom[1]. It is
extremely well automated to use the existing infrastructure to do it all
for you.

Without looking at official documentation (which is me being lazy,
sorry), some other things that will need to be thought about:

Start with the releases page [2]

* You *must* include a source artifact. It cannot have binaries. No Java
class files, no jars. A user must be able to download the source
artifact and build it all themselves. Artifacts which include binaries
are for user-convenience only and can optionally be provided as well.

* LICENSE, NOTICE and README must all be included in the top level of
the artifact. The NOTICE is the most painful and must include any
required third-party notices. [3]

* You will need to audit your dependencies to make sure that they are
ASLv2 compatible [4]

* Releases must be cryptographically signed (PGP) [5]. Your public key
should be published to known sites (e.g. pgp.mit.edu) and must be listed
in NiFi's KEYS file [6] (which does not yet exist, probably needs an
infra ticket to create your dist/ directory?).

* Verify that every source file contain the proper ASL header. The
release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool that
can (and should, IMO) be integrated into your build process to prevent
people from accidentally committing new files between releases that do
not have the correct header.

And suddenly, this is really long :). Short answer: decide what the
release should look like (just a tarball?), start vetting your source
code for license headers, and looking into NiFi's dependencies and their
licenses.

[1] http://maven.apache.org/pom/asf/
[2] http://www.apache.org/dev/release.html
[3] http://www.apache.org/legal/src-headers.html
[4] http://www.apache.org/legal/resolved.html#category-x
[5] http://www.apache.org/dev/release-signing.html
[6] http://www.apache.org/dist/incubator/nifi/KEYS
[7] http://creadur.apache.org/rat/

Tony Kurc wrote:

> I read the guide Joe linked and a lot of the sticky parts are marked "TODO"
> and it looks like a work in progress
>
> "Podlings can short circuit this process by starting out with written
> release documentation. It is strongly recommended that Podlings invest time
> looking at the best practices recommended in this document. By selection
> and modification, sections of this document can be used to quickly and
> easily bootstrap a release guide. "
>
> Is step one putting together a release guide?
>
>
>
> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>  wrote:
>
>> Hello
>>
>> Just wanted to stir this one up a bit.  Looks like all tickets pegged as
>> 0.0.1 are resolved (2 remain as of now but seem likely to be resolved
>> shortly based on their comments).  So working through the release steps
>> available on apache.org and via the link Brock sent.
>>
>> Anyone interested in this part of the process or who has advice to help us
>> avoid landmines we're happy to hear it.
>>
>> Thanks
>> Joe
>>
>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>  wrote:
>>
>>> Thanks Brock this is very helpful.
>>>
>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<[hidden email]>
>> wrote:
>>>> Hi,
>>>>
>>>> This is a decent guide which can be copied:
>>>>
>>>> http://mrunit.apache.org/pmc/how_to_release.html
>>>>
>>>> Brock
>>>>
>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>  wrote:
>>>>> Folks,
>>>>>
>>>>> Looking at the tickets that remain which are presently tied to 0.0.1
>>>> we're
>>>>> probably 1-2 weeks out from this initial release.  Can you provide
>> some
>>>>> pointers/references or pointers on how to get this ball rolling and
>> any
>>>>> rocks we must move before going down this path?
>>>>>
>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>
>>>>> That link seems really helpful but has a lot of TODOs for areas which
>>>> need
>>>>> more explanation.
>>>>>
>>>>>
>>>>> Thanks
>>>>> Joe
>>>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joe Witt
Josh

Awesome.

Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER, README.
All dependencies have been reviewed and checked for ASLv2 compatibility.

Will submit the infra ticket now for the dist/ path for keys files and
such.

My guess is that the release artifact should be a tarball of all source.
Could we literally just package up a clean source tree?  Anyone else have
views on this?

Ideally with this release we'd do it all properly including maven
artifacts, sources/javadocs, and so on.  The Maven build does already
operate now off a single command at the root to build everything
(build-order is gone) and inherits from the apache parent.

Will need to incorporate RAT.

Thanks for all that - definitely gives some stuff to work on and look into.

Thanks
Joe

On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <[hidden email]> wrote:

> Regardless of what you call it, writing down exactly what was done to make
> a RC is extremely important early on (I know that I sure can't remember
> what I did last week, much less last release). I'm not sure if release
> guide is formally defined somewhere, but enough information that any other
> committer can come by after "you get hit by a train" and make a release is
> extremely important. Using the CMS and making a page on the website for
> this is extremely easy to do, IMO.
>
> Another concern for how to actually make a release is the type of
> packaging that is released. I not talking about the source/binary release
> here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a tarball?
> Are there jars in Maven? etc. A tarball is pretty easy to make, and likely
> the closest to what the build-order.sh script already does (with Maven's
> help). I'm not sure if maven-released artifacts are quite as important at
> this point -- if it's not, punting on that will help. If/when you get to
> that point, look into the apache parent pom[1]. It is extremely well
> automated to use the existing infrastructure to do it all for you.
>
> Without looking at official documentation (which is me being lazy, sorry),
> some other things that will need to be thought about:
>
> Start with the releases page [2]
>
> * You *must* include a source artifact. It cannot have binaries. No Java
> class files, no jars. A user must be able to download the source artifact
> and build it all themselves. Artifacts which include binaries are for
> user-convenience only and can optionally be provided as well.
>
> * LICENSE, NOTICE and README must all be included in the top level of the
> artifact. The NOTICE is the most painful and must include any required
> third-party notices. [3]
>
> * You will need to audit your dependencies to make sure that they are
> ASLv2 compatible [4]
>
> * Releases must be cryptographically signed (PGP) [5]. Your public key
> should be published to known sites (e.g. pgp.mit.edu) and must be listed
> in NiFi's KEYS file [6] (which does not yet exist, probably needs an infra
> ticket to create your dist/ directory?).
>
> * Verify that every source file contain the proper ASL header. The
> release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool that can
> (and should, IMO) be integrated into your build process to prevent people
> from accidentally committing new files between releases that do not have
> the correct header.
>
> And suddenly, this is really long :). Short answer: decide what the
> release should look like (just a tarball?), start vetting your source code
> for license headers, and looking into NiFi's dependencies and their
> licenses.
>
> [1] http://maven.apache.org/pom/asf/
> [2] http://www.apache.org/dev/release.html
> [3] http://www.apache.org/legal/src-headers.html
> [4] http://www.apache.org/legal/resolved.html#category-x
> [5] http://www.apache.org/dev/release-signing.html
> [6] http://www.apache.org/dist/incubator/nifi/KEYS
> [7] http://creadur.apache.org/rat/
>
>
> Tony Kurc wrote:
>
>> I read the guide Joe linked and a lot of the sticky parts are marked
>> "TODO"
>> and it looks like a work in progress
>>
>> "Podlings can short circuit this process by starting out with written
>> release documentation. It is strongly recommended that Podlings invest
>> time
>> looking at the best practices recommended in this document. By selection
>> and modification, sections of this document can be used to quickly and
>> easily bootstrap a release guide. "
>>
>> Is step one putting together a release guide?
>>
>>
>>
>> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>  wrote:
>>
>>  Hello
>>>
>>> Just wanted to stir this one up a bit.  Looks like all tickets pegged as
>>> 0.0.1 are resolved (2 remain as of now but seem likely to be resolved
>>> shortly based on their comments).  So working through the release steps
>>> available on apache.org and via the link Brock sent.
>>>
>>> Anyone interested in this part of the process or who has advice to help
>>> us
>>> avoid landmines we're happy to hear it.
>>>
>>> Thanks
>>> Joe
>>>
>>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>  wrote:
>>>
>>>  Thanks Brock this is very helpful.
>>>>
>>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<[hidden email]>
>>>>
>>> wrote:
>>>
>>>> Hi,
>>>>>
>>>>> This is a decent guide which can be copied:
>>>>>
>>>>> http://mrunit.apache.org/pmc/how_to_release.html
>>>>>
>>>>> Brock
>>>>>
>>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>  wrote:
>>>>>
>>>>>> Folks,
>>>>>>
>>>>>> Looking at the tickets that remain which are presently tied to 0.0.1
>>>>>>
>>>>> we're
>>>>>
>>>>>> probably 1-2 weeks out from this initial release.  Can you provide
>>>>>>
>>>>> some
>>>
>>>> pointers/references or pointers on how to get this ball rolling and
>>>>>>
>>>>> any
>>>
>>>> rocks we must move before going down this path?
>>>>>>
>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>>
>>>>>> That link seems really helpful but has a lot of TODOs for areas which
>>>>>>
>>>>> need
>>>>>
>>>>>> more explanation.
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Joe
>>>>>>
>>>>>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joe Witt
Took at a stab at the infra ticket:
https://issues.apache.org/jira/browse/INFRA-8992

Didn't quite understand what i was asking for so there is a non-zero chance
of that not working out.

Thanks
Joe

On Wed, Jan 7, 2015 at 10:22 PM, Joe Witt <[hidden email]> wrote:

> Josh
>
> Awesome.
>
> Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER, README.
> All dependencies have been reviewed and checked for ASLv2 compatibility.
>
> Will submit the infra ticket now for the dist/ path for keys files and
> such.
>
> My guess is that the release artifact should be a tarball of all source.
> Could we literally just package up a clean source tree?  Anyone else have
> views on this?
>
> Ideally with this release we'd do it all properly including maven
> artifacts, sources/javadocs, and so on.  The Maven build does already
> operate now off a single command at the root to build everything
> (build-order is gone) and inherits from the apache parent.
>
> Will need to incorporate RAT.
>
> Thanks for all that - definitely gives some stuff to work on and look into.
>
> Thanks
> Joe
>
> On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <[hidden email]> wrote:
>
>> Regardless of what you call it, writing down exactly what was done to
>> make a RC is extremely important early on (I know that I sure can't
>> remember what I did last week, much less last release). I'm not sure if
>> release guide is formally defined somewhere, but enough information that
>> any other committer can come by after "you get hit by a train" and make a
>> release is extremely important. Using the CMS and making a page on the
>> website for this is extremely easy to do, IMO.
>>
>> Another concern for how to actually make a release is the type of
>> packaging that is released. I not talking about the source/binary release
>> here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a tarball?
>> Are there jars in Maven? etc. A tarball is pretty easy to make, and likely
>> the closest to what the build-order.sh script already does (with Maven's
>> help). I'm not sure if maven-released artifacts are quite as important at
>> this point -- if it's not, punting on that will help. If/when you get to
>> that point, look into the apache parent pom[1]. It is extremely well
>> automated to use the existing infrastructure to do it all for you.
>>
>> Without looking at official documentation (which is me being lazy,
>> sorry), some other things that will need to be thought about:
>>
>> Start with the releases page [2]
>>
>> * You *must* include a source artifact. It cannot have binaries. No Java
>> class files, no jars. A user must be able to download the source artifact
>> and build it all themselves. Artifacts which include binaries are for
>> user-convenience only and can optionally be provided as well.
>>
>> * LICENSE, NOTICE and README must all be included in the top level of the
>> artifact. The NOTICE is the most painful and must include any required
>> third-party notices. [3]
>>
>> * You will need to audit your dependencies to make sure that they are
>> ASLv2 compatible [4]
>>
>> * Releases must be cryptographically signed (PGP) [5]. Your public key
>> should be published to known sites (e.g. pgp.mit.edu) and must be listed
>> in NiFi's KEYS file [6] (which does not yet exist, probably needs an infra
>> ticket to create your dist/ directory?).
>>
>> * Verify that every source file contain the proper ASL header. The
>> release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool that can
>> (and should, IMO) be integrated into your build process to prevent people
>> from accidentally committing new files between releases that do not have
>> the correct header.
>>
>> And suddenly, this is really long :). Short answer: decide what the
>> release should look like (just a tarball?), start vetting your source code
>> for license headers, and looking into NiFi's dependencies and their
>> licenses.
>>
>> [1] http://maven.apache.org/pom/asf/
>> [2] http://www.apache.org/dev/release.html
>> [3] http://www.apache.org/legal/src-headers.html
>> [4] http://www.apache.org/legal/resolved.html#category-x
>> [5] http://www.apache.org/dev/release-signing.html
>> [6] http://www.apache.org/dist/incubator/nifi/KEYS
>> [7] http://creadur.apache.org/rat/
>>
>>
>> Tony Kurc wrote:
>>
>>> I read the guide Joe linked and a lot of the sticky parts are marked
>>> "TODO"
>>> and it looks like a work in progress
>>>
>>> "Podlings can short circuit this process by starting out with written
>>> release documentation. It is strongly recommended that Podlings invest
>>> time
>>> looking at the best practices recommended in this document. By selection
>>> and modification, sections of this document can be used to quickly and
>>> easily bootstrap a release guide. "
>>>
>>> Is step one putting together a release guide?
>>>
>>>
>>>
>>> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>  wrote:
>>>
>>>  Hello
>>>>
>>>> Just wanted to stir this one up a bit.  Looks like all tickets pegged as
>>>> 0.0.1 are resolved (2 remain as of now but seem likely to be resolved
>>>> shortly based on their comments).  So working through the release steps
>>>> available on apache.org and via the link Brock sent.
>>>>
>>>> Anyone interested in this part of the process or who has advice to help
>>>> us
>>>> avoid landmines we're happy to hear it.
>>>>
>>>> Thanks
>>>> Joe
>>>>
>>>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>  wrote:
>>>>
>>>>  Thanks Brock this is very helpful.
>>>>>
>>>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<[hidden email]>
>>>>>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>>
>>>>>> This is a decent guide which can be copied:
>>>>>>
>>>>>> http://mrunit.apache.org/pmc/how_to_release.html
>>>>>>
>>>>>> Brock
>>>>>>
>>>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Folks,
>>>>>>>
>>>>>>> Looking at the tickets that remain which are presently tied to 0.0.1
>>>>>>>
>>>>>> we're
>>>>>>
>>>>>>> probably 1-2 weeks out from this initial release.  Can you provide
>>>>>>>
>>>>>> some
>>>>
>>>>> pointers/references or pointers on how to get this ball rolling and
>>>>>>>
>>>>>> any
>>>>
>>>>> rocks we must move before going down this path?
>>>>>>>
>>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>>>
>>>>>>> That link seems really helpful but has a lot of TODOs for areas which
>>>>>>>
>>>>>> need
>>>>>>
>>>>>>> more explanation.
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>> Joe
>>>>>>>
>>>>>>>
>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Josh Elser
It's also possible that a mentor already has the karma to create it (I
have no idea how the permissions are controlled tbh). I made a guess
that it will require infra-support, but I think you gave enough
information :)

Joe Witt wrote:

> Took at a stab at the infra ticket:
> https://issues.apache.org/jira/browse/INFRA-8992
>
> Didn't quite understand what i was asking for so there is a non-zero chance
> of that not working out.
>
> Thanks
> Joe
>
> On Wed, Jan 7, 2015 at 10:22 PM, Joe Witt<[hidden email]>  wrote:
>
>> Josh
>>
>> Awesome.
>>
>> Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER, README.
>> All dependencies have been reviewed and checked for ASLv2 compatibility.
>>
>> Will submit the infra ticket now for the dist/ path for keys files and
>> such.
>>
>> My guess is that the release artifact should be a tarball of all source.
>> Could we literally just package up a clean source tree?  Anyone else have
>> views on this?
>>
>> Ideally with this release we'd do it all properly including maven
>> artifacts, sources/javadocs, and so on.  The Maven build does already
>> operate now off a single command at the root to build everything
>> (build-order is gone) and inherits from the apache parent.
>>
>> Will need to incorporate RAT.
>>
>> Thanks for all that - definitely gives some stuff to work on and look into.
>>
>> Thanks
>> Joe
>>
>> On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser<[hidden email]>  wrote:
>>
>>> Regardless of what you call it, writing down exactly what was done to
>>> make a RC is extremely important early on (I know that I sure can't
>>> remember what I did last week, much less last release). I'm not sure if
>>> release guide is formally defined somewhere, but enough information that
>>> any other committer can come by after "you get hit by a train" and make a
>>> release is extremely important. Using the CMS and making a page on the
>>> website for this is extremely easy to do, IMO.
>>>
>>> Another concern for how to actually make a release is the type of
>>> packaging that is released. I not talking about the source/binary release
>>> here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a tarball?
>>> Are there jars in Maven? etc. A tarball is pretty easy to make, and likely
>>> the closest to what the build-order.sh script already does (with Maven's
>>> help). I'm not sure if maven-released artifacts are quite as important at
>>> this point -- if it's not, punting on that will help. If/when you get to
>>> that point, look into the apache parent pom[1]. It is extremely well
>>> automated to use the existing infrastructure to do it all for you.
>>>
>>> Without looking at official documentation (which is me being lazy,
>>> sorry), some other things that will need to be thought about:
>>>
>>> Start with the releases page [2]
>>>
>>> * You *must* include a source artifact. It cannot have binaries. No Java
>>> class files, no jars. A user must be able to download the source artifact
>>> and build it all themselves. Artifacts which include binaries are for
>>> user-convenience only and can optionally be provided as well.
>>>
>>> * LICENSE, NOTICE and README must all be included in the top level of the
>>> artifact. The NOTICE is the most painful and must include any required
>>> third-party notices. [3]
>>>
>>> * You will need to audit your dependencies to make sure that they are
>>> ASLv2 compatible [4]
>>>
>>> * Releases must be cryptographically signed (PGP) [5]. Your public key
>>> should be published to known sites (e.g. pgp.mit.edu) and must be listed
>>> in NiFi's KEYS file [6] (which does not yet exist, probably needs an infra
>>> ticket to create your dist/ directory?).
>>>
>>> * Verify that every source file contain the proper ASL header. The
>>> release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool that can
>>> (and should, IMO) be integrated into your build process to prevent people
>>> from accidentally committing new files between releases that do not have
>>> the correct header.
>>>
>>> And suddenly, this is really long :). Short answer: decide what the
>>> release should look like (just a tarball?), start vetting your source code
>>> for license headers, and looking into NiFi's dependencies and their
>>> licenses.
>>>
>>> [1] http://maven.apache.org/pom/asf/
>>> [2] http://www.apache.org/dev/release.html
>>> [3] http://www.apache.org/legal/src-headers.html
>>> [4] http://www.apache.org/legal/resolved.html#category-x
>>> [5] http://www.apache.org/dev/release-signing.html
>>> [6] http://www.apache.org/dist/incubator/nifi/KEYS
>>> [7] http://creadur.apache.org/rat/
>>>
>>>
>>> Tony Kurc wrote:
>>>
>>>> I read the guide Joe linked and a lot of the sticky parts are marked
>>>> "TODO"
>>>> and it looks like a work in progress
>>>>
>>>> "Podlings can short circuit this process by starting out with written
>>>> release documentation. It is strongly recommended that Podlings invest
>>>> time
>>>> looking at the best practices recommended in this document. By selection
>>>> and modification, sections of this document can be used to quickly and
>>>> easily bootstrap a release guide. "
>>>>
>>>> Is step one putting together a release guide?
>>>>
>>>>
>>>>
>>>> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>   wrote:
>>>>
>>>>   Hello
>>>>> Just wanted to stir this one up a bit.  Looks like all tickets pegged as
>>>>> 0.0.1 are resolved (2 remain as of now but seem likely to be resolved
>>>>> shortly based on their comments).  So working through the release steps
>>>>> available on apache.org and via the link Brock sent.
>>>>>
>>>>> Anyone interested in this part of the process or who has advice to help
>>>>> us
>>>>> avoid landmines we're happy to hear it.
>>>>>
>>>>> Thanks
>>>>> Joe
>>>>>
>>>>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>   wrote:
>>>>>
>>>>>   Thanks Brock this is very helpful.
>>>>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<[hidden email]>
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>> This is a decent guide which can be copied:
>>>>>>>
>>>>>>> http://mrunit.apache.org/pmc/how_to_release.html
>>>>>>>
>>>>>>> Brock
>>>>>>>
>>>>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Folks,
>>>>>>>>
>>>>>>>> Looking at the tickets that remain which are presently tied to 0.0.1
>>>>>>>>
>>>>>>> we're
>>>>>>>
>>>>>>>> probably 1-2 weeks out from this initial release.  Can you provide
>>>>>>>>
>>>>>>> some
>>>>>> pointers/references or pointers on how to get this ball rolling and
>>>>>>> any
>>>>>> rocks we must move before going down this path?
>>>>>>>> http://incubator.apache.org/guides/releasemanagement.html
>>>>>>>>
>>>>>>>> That link seems really helpful but has a lot of TODOs for areas which
>>>>>>>>
>>>>>>> need
>>>>>>>
>>>>>>>> more explanation.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Joe
>>>>>>>>
>>>>>>>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Mike Drob-2
In reply to this post by Joe Witt
On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]> wrote:

> Josh
>
> Awesome.
>
> Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER, README.
> All dependencies have been reviewed and checked for ASLv2 compatibility.
>
> Will submit the infra ticket now for the dist/ path for keys files and
> such.
>
> My guess is that the release artifact should be a tarball of all source.
> Could we literally just package up a clean source tree?  Anyone else have
> views on this?
>

git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is a perfectly
reasonable thing to do.

>
> Ideally with this release we'd do it all properly including maven
> artifacts, sources/javadocs, and so on.  The Maven build does already
> operate now off a single command at the root to build everything
> (build-order is gone) and inherits from the apache parent.
>
> Will need to incorporate RAT.
>
> Thanks for all that - definitely gives some stuff to work on and look into.
>
> Thanks
> Joe
>
> On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <[hidden email]> wrote:
>
> > Regardless of what you call it, writing down exactly what was done to
> make
> > a RC is extremely important early on (I know that I sure can't remember
> > what I did last week, much less last release). I'm not sure if release
> > guide is formally defined somewhere, but enough information that any
> other
> > committer can come by after "you get hit by a train" and make a release
> is
> > extremely important. Using the CMS and making a page on the website for
> > this is extremely easy to do, IMO.
> >
> > Another concern for how to actually make a release is the type of
> > packaging that is released. I not talking about the source/binary release
> > here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a tarball?
> > Are there jars in Maven? etc. A tarball is pretty easy to make, and
> likely
> > the closest to what the build-order.sh script already does (with Maven's
> > help). I'm not sure if maven-released artifacts are quite as important at
> > this point -- if it's not, punting on that will help. If/when you get to
> > that point, look into the apache parent pom[1]. It is extremely well
> > automated to use the existing infrastructure to do it all for you.
> >
> > Without looking at official documentation (which is me being lazy,
> sorry),
> > some other things that will need to be thought about:
> >
> > Start with the releases page [2]
> >
> > * You *must* include a source artifact. It cannot have binaries. No Java
> > class files, no jars. A user must be able to download the source artifact
> > and build it all themselves. Artifacts which include binaries are for
> > user-convenience only and can optionally be provided as well.
> >
> > * LICENSE, NOTICE and README must all be included in the top level of the
> > artifact. The NOTICE is the most painful and must include any required
> > third-party notices. [3]
> >
> > * You will need to audit your dependencies to make sure that they are
> > ASLv2 compatible [4]
> >
> > * Releases must be cryptographically signed (PGP) [5]. Your public key
> > should be published to known sites (e.g. pgp.mit.edu) and must be listed
> > in NiFi's KEYS file [6] (which does not yet exist, probably needs an
> infra
> > ticket to create your dist/ directory?).
> >
> > * Verify that every source file contain the proper ASL header. The
> > release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool that
> can
> > (and should, IMO) be integrated into your build process to prevent people
> > from accidentally committing new files between releases that do not have
> > the correct header.
> >
> > And suddenly, this is really long :). Short answer: decide what the
> > release should look like (just a tarball?), start vetting your source
> code
> > for license headers, and looking into NiFi's dependencies and their
> > licenses.
> >
> > [1] http://maven.apache.org/pom/asf/
> > [2] http://www.apache.org/dev/release.html
> > [3] http://www.apache.org/legal/src-headers.html
> > [4] http://www.apache.org/legal/resolved.html#category-x
> > [5] http://www.apache.org/dev/release-signing.html
> > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > [7] http://creadur.apache.org/rat/
> >
> >
> > Tony Kurc wrote:
> >
> >> I read the guide Joe linked and a lot of the sticky parts are marked
> >> "TODO"
> >> and it looks like a work in progress
> >>
> >> "Podlings can short circuit this process by starting out with written
> >> release documentation. It is strongly recommended that Podlings invest
> >> time
> >> looking at the best practices recommended in this document. By selection
> >> and modification, sections of this document can be used to quickly and
> >> easily bootstrap a release guide. "
> >>
> >> Is step one putting together a release guide?
> >>
> >>
> >>
> >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>  wrote:
> >>
> >>  Hello
> >>>
> >>> Just wanted to stir this one up a bit.  Looks like all tickets pegged
> as
> >>> 0.0.1 are resolved (2 remain as of now but seem likely to be resolved
> >>> shortly based on their comments).  So working through the release steps
> >>> available on apache.org and via the link Brock sent.
> >>>
> >>> Anyone interested in this part of the process or who has advice to help
> >>> us
> >>> avoid landmines we're happy to hear it.
> >>>
> >>> Thanks
> >>> Joe
> >>>
> >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>  wrote:
> >>>
> >>>  Thanks Brock this is very helpful.
> >>>>
> >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<[hidden email]>
> >>>>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>>>
> >>>>> This is a decent guide which can be copied:
> >>>>>
> >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> >>>>>
> >>>>> Brock
> >>>>>
> >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>
> wrote:
> >>>>>
> >>>>>> Folks,
> >>>>>>
> >>>>>> Looking at the tickets that remain which are presently tied to 0.0.1
> >>>>>>
> >>>>> we're
> >>>>>
> >>>>>> probably 1-2 weeks out from this initial release.  Can you provide
> >>>>>>
> >>>>> some
> >>>
> >>>> pointers/references or pointers on how to get this ball rolling and
> >>>>>>
> >>>>> any
> >>>
> >>>> rocks we must move before going down this path?
> >>>>>>
> >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> >>>>>>
> >>>>>> That link seems really helpful but has a lot of TODOs for areas
> which
> >>>>>>
> >>>>> need
> >>>>>
> >>>>>> more explanation.
> >>>>>>
> >>>>>>
> >>>>>> Thanks
> >>>>>> Joe
> >>>>>>
> >>>>>>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Benson Margulies
Typically people set the maven assembly plugin for this and include it in
the build. Would you like me to pitch this in?


On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]> wrote:

> On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]> wrote:
>
> > Josh
> >
> > Awesome.
> >
> > Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER, README.
> > All dependencies have been reviewed and checked for ASLv2 compatibility.
> >
> > Will submit the infra ticket now for the dist/ path for keys files and
> > such.
> >
> > My guess is that the release artifact should be a tarball of all source.
> > Could we literally just package up a clean source tree?  Anyone else have
> > views on this?
> >
>
> git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is a perfectly
> reasonable thing to do.
>
> >
> > Ideally with this release we'd do it all properly including maven
> > artifacts, sources/javadocs, and so on.  The Maven build does already
> > operate now off a single command at the root to build everything
> > (build-order is gone) and inherits from the apache parent.
> >
> > Will need to incorporate RAT.
> >
> > Thanks for all that - definitely gives some stuff to work on and look
> into.
> >
> > Thanks
> > Joe
> >
> > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <[hidden email]>
> wrote:
> >
> > > Regardless of what you call it, writing down exactly what was done to
> > make
> > > a RC is extremely important early on (I know that I sure can't remember
> > > what I did last week, much less last release). I'm not sure if release
> > > guide is formally defined somewhere, but enough information that any
> > other
> > > committer can come by after "you get hit by a train" and make a release
> > is
> > > extremely important. Using the CMS and making a page on the website for
> > > this is extremely easy to do, IMO.
> > >
> > > Another concern for how to actually make a release is the type of
> > > packaging that is released. I not talking about the source/binary
> release
> > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a
> tarball?
> > > Are there jars in Maven? etc. A tarball is pretty easy to make, and
> > likely
> > > the closest to what the build-order.sh script already does (with
> Maven's
> > > help). I'm not sure if maven-released artifacts are quite as important
> at
> > > this point -- if it's not, punting on that will help. If/when you get
> to
> > > that point, look into the apache parent pom[1]. It is extremely well
> > > automated to use the existing infrastructure to do it all for you.
> > >
> > > Without looking at official documentation (which is me being lazy,
> > sorry),
> > > some other things that will need to be thought about:
> > >
> > > Start with the releases page [2]
> > >
> > > * You *must* include a source artifact. It cannot have binaries. No
> Java
> > > class files, no jars. A user must be able to download the source
> artifact
> > > and build it all themselves. Artifacts which include binaries are for
> > > user-convenience only and can optionally be provided as well.
> > >
> > > * LICENSE, NOTICE and README must all be included in the top level of
> the
> > > artifact. The NOTICE is the most painful and must include any required
> > > third-party notices. [3]
> > >
> > > * You will need to audit your dependencies to make sure that they are
> > > ASLv2 compatible [4]
> > >
> > > * Releases must be cryptographically signed (PGP) [5]. Your public key
> > > should be published to known sites (e.g. pgp.mit.edu) and must be
> listed
> > > in NiFi's KEYS file [6] (which does not yet exist, probably needs an
> > infra
> > > ticket to create your dist/ directory?).
> > >
> > > * Verify that every source file contain the proper ASL header. The
> > > release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool that
> > can
> > > (and should, IMO) be integrated into your build process to prevent
> people
> > > from accidentally committing new files between releases that do not
> have
> > > the correct header.
> > >
> > > And suddenly, this is really long :). Short answer: decide what the
> > > release should look like (just a tarball?), start vetting your source
> > code
> > > for license headers, and looking into NiFi's dependencies and their
> > > licenses.
> > >
> > > [1] http://maven.apache.org/pom/asf/
> > > [2] http://www.apache.org/dev/release.html
> > > [3] http://www.apache.org/legal/src-headers.html
> > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > [5] http://www.apache.org/dev/release-signing.html
> > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > [7] http://creadur.apache.org/rat/
> > >
> > >
> > > Tony Kurc wrote:
> > >
> > >> I read the guide Joe linked and a lot of the sticky parts are marked
> > >> "TODO"
> > >> and it looks like a work in progress
> > >>
> > >> "Podlings can short circuit this process by starting out with written
> > >> release documentation. It is strongly recommended that Podlings invest
> > >> time
> > >> looking at the best practices recommended in this document. By
> selection
> > >> and modification, sections of this document can be used to quickly and
> > >> easily bootstrap a release guide. "
> > >>
> > >> Is step one putting together a release guide?
> > >>
> > >>
> > >>
> > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>  wrote:
> > >>
> > >>  Hello
> > >>>
> > >>> Just wanted to stir this one up a bit.  Looks like all tickets pegged
> > as
> > >>> 0.0.1 are resolved (2 remain as of now but seem likely to be resolved
> > >>> shortly based on their comments).  So working through the release
> steps
> > >>> available on apache.org and via the link Brock sent.
> > >>>
> > >>> Anyone interested in this part of the process or who has advice to
> help
> > >>> us
> > >>> avoid landmines we're happy to hear it.
> > >>>
> > >>> Thanks
> > >>> Joe
> > >>>
> > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>
> wrote:
> > >>>
> > >>>  Thanks Brock this is very helpful.
> > >>>>
> > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<[hidden email]>
> > >>>>
> > >>> wrote:
> > >>>
> > >>>> Hi,
> > >>>>>
> > >>>>> This is a decent guide which can be copied:
> > >>>>>
> > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > >>>>>
> > >>>>> Brock
> > >>>>>
> > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>
> > wrote:
> > >>>>>
> > >>>>>> Folks,
> > >>>>>>
> > >>>>>> Looking at the tickets that remain which are presently tied to
> 0.0.1
> > >>>>>>
> > >>>>> we're
> > >>>>>
> > >>>>>> probably 1-2 weeks out from this initial release.  Can you provide
> > >>>>>>
> > >>>>> some
> > >>>
> > >>>> pointers/references or pointers on how to get this ball rolling and
> > >>>>>>
> > >>>>> any
> > >>>
> > >>>> rocks we must move before going down this path?
> > >>>>>>
> > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > >>>>>>
> > >>>>>> That link seems really helpful but has a lot of TODOs for areas
> > which
> > >>>>>>
> > >>>>> need
> > >>>>>
> > >>>>>> more explanation.
> > >>>>>>
> > >>>>>>
> > >>>>>> Thanks
> > >>>>>> Joe
> > >>>>>>
> > >>>>>>
> > >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joe Witt
Benson,

Yes that would be much appreciated.

Here is a rough gamplan of what I *think* will be needed to do the release:

- Branch from develop against a ticket for the overall release process
- Update the version of all the artifacts and dependencies to
0.0.1-RC1-incubating or something like that
- Create the tar.gz of the clean source tree.  Sign that and place both the
signed file and the tar.gz into my people's dir (if i have one of those)
- Notify folks that this is available so they can pull it and attempt to
build from that and validate the signature
- Once folks agree it is good to merge that to master
- Create a tar.gz of that and a signed file.  Call a vote within the team.
If that is good call a vote within IPMC.
- IF NO - fix whatever is wrong.
- If all good then do a maven deploy of the binary artifacts, source
bundles, and javadocs
- Upload the tar.gz of source and the signed file to our dist dir.  And
upload convenience binary build tar.gz and zip with sigs to our dist
folder.  Then create links to these from our downloads page.
- Update JIRA that the release is closed.
- Wash/Rinse/Repeat

<note this all is with the understanding that we've done all the necessary
review of dependencies, licenses, properly documented them, have our
ECCN/encryption stuff properly filed>

Might be missing a detail or two but does that sound roughly on the right
rack?

I had though the maven release plugin would be useful but looks like we'll
just be able to use the versions plugin to update them and can do the other
simple steps manually.

Thanks
Joe

On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <[hidden email]>
wrote:

> Typically people set the maven assembly plugin for this and include it in
> the build. Would you like me to pitch this in?
>
>
> On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]> wrote:
>
> > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]> wrote:
> >
> > > Josh
> > >
> > > Awesome.
> > >
> > > Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER,
> README.
> > > All dependencies have been reviewed and checked for ASLv2
> compatibility.
> > >
> > > Will submit the infra ticket now for the dist/ path for keys files and
> > > such.
> > >
> > > My guess is that the release artifact should be a tarball of all
> source.
> > > Could we literally just package up a clean source tree?  Anyone else
> have
> > > views on this?
> > >
> >
> > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is a
> perfectly
> > reasonable thing to do.
> >
> > >
> > > Ideally with this release we'd do it all properly including maven
> > > artifacts, sources/javadocs, and so on.  The Maven build does already
> > > operate now off a single command at the root to build everything
> > > (build-order is gone) and inherits from the apache parent.
> > >
> > > Will need to incorporate RAT.
> > >
> > > Thanks for all that - definitely gives some stuff to work on and look
> > into.
> > >
> > > Thanks
> > > Joe
> > >
> > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <[hidden email]>
> > wrote:
> > >
> > > > Regardless of what you call it, writing down exactly what was done to
> > > make
> > > > a RC is extremely important early on (I know that I sure can't
> remember
> > > > what I did last week, much less last release). I'm not sure if
> release
> > > > guide is formally defined somewhere, but enough information that any
> > > other
> > > > committer can come by after "you get hit by a train" and make a
> release
> > > is
> > > > extremely important. Using the CMS and making a page on the website
> for
> > > > this is extremely easy to do, IMO.
> > > >
> > > > Another concern for how to actually make a release is the type of
> > > > packaging that is released. I not talking about the source/binary
> > release
> > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a
> > tarball?
> > > > Are there jars in Maven? etc. A tarball is pretty easy to make, and
> > > likely
> > > > the closest to what the build-order.sh script already does (with
> > Maven's
> > > > help). I'm not sure if maven-released artifacts are quite as
> important
> > at
> > > > this point -- if it's not, punting on that will help. If/when you get
> > to
> > > > that point, look into the apache parent pom[1]. It is extremely well
> > > > automated to use the existing infrastructure to do it all for you.
> > > >
> > > > Without looking at official documentation (which is me being lazy,
> > > sorry),
> > > > some other things that will need to be thought about:
> > > >
> > > > Start with the releases page [2]
> > > >
> > > > * You *must* include a source artifact. It cannot have binaries. No
> > Java
> > > > class files, no jars. A user must be able to download the source
> > artifact
> > > > and build it all themselves. Artifacts which include binaries are for
> > > > user-convenience only and can optionally be provided as well.
> > > >
> > > > * LICENSE, NOTICE and README must all be included in the top level of
> > the
> > > > artifact. The NOTICE is the most painful and must include any
> required
> > > > third-party notices. [3]
> > > >
> > > > * You will need to audit your dependencies to make sure that they are
> > > > ASLv2 compatible [4]
> > > >
> > > > * Releases must be cryptographically signed (PGP) [5]. Your public
> key
> > > > should be published to known sites (e.g. pgp.mit.edu) and must be
> > listed
> > > > in NiFi's KEYS file [6] (which does not yet exist, probably needs an
> > > infra
> > > > ticket to create your dist/ directory?).
> > > >
> > > > * Verify that every source file contain the proper ASL header. The
> > > > release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool
> that
> > > can
> > > > (and should, IMO) be integrated into your build process to prevent
> > people
> > > > from accidentally committing new files between releases that do not
> > have
> > > > the correct header.
> > > >
> > > > And suddenly, this is really long :). Short answer: decide what the
> > > > release should look like (just a tarball?), start vetting your source
> > > code
> > > > for license headers, and looking into NiFi's dependencies and their
> > > > licenses.
> > > >
> > > > [1] http://maven.apache.org/pom/asf/
> > > > [2] http://www.apache.org/dev/release.html
> > > > [3] http://www.apache.org/legal/src-headers.html
> > > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > > [5] http://www.apache.org/dev/release-signing.html
> > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > > [7] http://creadur.apache.org/rat/
> > > >
> > > >
> > > > Tony Kurc wrote:
> > > >
> > > >> I read the guide Joe linked and a lot of the sticky parts are marked
> > > >> "TODO"
> > > >> and it looks like a work in progress
> > > >>
> > > >> "Podlings can short circuit this process by starting out with
> written
> > > >> release documentation. It is strongly recommended that Podlings
> invest
> > > >> time
> > > >> looking at the best practices recommended in this document. By
> > selection
> > > >> and modification, sections of this document can be used to quickly
> and
> > > >> easily bootstrap a release guide. "
> > > >>
> > > >> Is step one putting together a release guide?
> > > >>
> > > >>
> > > >>
> > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>
> wrote:
> > > >>
> > > >>  Hello
> > > >>>
> > > >>> Just wanted to stir this one up a bit.  Looks like all tickets
> pegged
> > > as
> > > >>> 0.0.1 are resolved (2 remain as of now but seem likely to be
> resolved
> > > >>> shortly based on their comments).  So working through the release
> > steps
> > > >>> available on apache.org and via the link Brock sent.
> > > >>>
> > > >>> Anyone interested in this part of the process or who has advice to
> > help
> > > >>> us
> > > >>> avoid landmines we're happy to hear it.
> > > >>>
> > > >>> Thanks
> > > >>> Joe
> > > >>>
> > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>
> > wrote:
> > > >>>
> > > >>>  Thanks Brock this is very helpful.
> > > >>>>
> > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<[hidden email]>
> > > >>>>
> > > >>> wrote:
> > > >>>
> > > >>>> Hi,
> > > >>>>>
> > > >>>>> This is a decent guide which can be copied:
> > > >>>>>
> > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > > >>>>>
> > > >>>>> Brock
> > > >>>>>
> > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>
> > > wrote:
> > > >>>>>
> > > >>>>>> Folks,
> > > >>>>>>
> > > >>>>>> Looking at the tickets that remain which are presently tied to
> > 0.0.1
> > > >>>>>>
> > > >>>>> we're
> > > >>>>>
> > > >>>>>> probably 1-2 weeks out from this initial release.  Can you
> provide
> > > >>>>>>
> > > >>>>> some
> > > >>>
> > > >>>> pointers/references or pointers on how to get this ball rolling
> and
> > > >>>>>>
> > > >>>>> any
> > > >>>
> > > >>>> rocks we must move before going down this path?
> > > >>>>>>
> > > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > > >>>>>>
> > > >>>>>> That link seems really helpful but has a lot of TODOs for areas
> > > which
> > > >>>>>>
> > > >>>>> need
> > > >>>>>
> > > >>>>>> more explanation.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Thanks
> > > >>>>>> Joe
> > > >>>>>>
> > > >>>>>>
> > > >>
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Benson Margulies
On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]> wrote:

> Benson,
>
> Yes that would be much appreciated.
>
> Here is a rough gamplan of what I *think* will be needed to do the release:
>
> - Branch from develop against a ticket for the overall release process
> - Update the version of all the artifacts and dependencies to
> 0.0.1-RC1-incubating or something like that
> - Create the tar.gz of the clean source tree.  Sign that and place both the
> signed file and the tar.gz into my people's dir (if i have one of those)
> - Notify folks that this is available so they can pull it and attempt to
> build from that and validate the signature
> - Once folks agree it is good to merge that to master
> - Create a tar.gz of that and a signed file.  Call a vote within the team.
> If that is good call a vote within IPMC.
> - IF NO - fix whatever is wrong.
> - If all good then do a maven deploy of the binary artifacts, source
> bundles, and javadocs
> - Upload the tar.gz of source and the signed file to our dist dir.  And
> upload convenience binary build tar.gz and zip with sigs to our dist
> folder.  Then create links to these from our downloads page.
> - Update JIRA that the release is closed.
> - Wash/Rinse/Repeat
>
> <note this all is with the understanding that we've done all the necessary
> review of dependencies, licenses, properly documented them, have our
> ECCN/encryption stuff properly filed>
>

Sure, except that I think we can make the maven-release-plugin do a lot of
the work.

The idea is that you run the release plugin, and it delivers all these
goodies to repository.apache.org. Then you call the vote. If the vote
passes, you (a) push the promote button to push to Maven Central, and (b)
check the official source release tarball into the svn area for
distribution.

I will take a whack at a PR for point (a).




>
> Might be missing a detail or two but does that sound roughly on the right
> rack?
>
> I had though the maven release plugin would be useful but looks like we'll
> just be able to use the versions plugin to update them and can do the other
> simple steps manually.
>
> Thanks
> Joe
>
> On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <[hidden email]>
> wrote:
>
> > Typically people set the maven assembly plugin for this and include it in
> > the build. Would you like me to pitch this in?
> >
> >
> > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]> wrote:
> >
> > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]> wrote:
> > >
> > > > Josh
> > > >
> > > > Awesome.
> > > >
> > > > Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER,
> > README.
> > > > All dependencies have been reviewed and checked for ASLv2
> > compatibility.
> > > >
> > > > Will submit the infra ticket now for the dist/ path for keys files
> and
> > > > such.
> > > >
> > > > My guess is that the release artifact should be a tarball of all
> > source.
> > > > Could we literally just package up a clean source tree?  Anyone else
> > have
> > > > views on this?
> > > >
> > >
> > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is a
> > perfectly
> > > reasonable thing to do.
> > >
> > > >
> > > > Ideally with this release we'd do it all properly including maven
> > > > artifacts, sources/javadocs, and so on.  The Maven build does already
> > > > operate now off a single command at the root to build everything
> > > > (build-order is gone) and inherits from the apache parent.
> > > >
> > > > Will need to incorporate RAT.
> > > >
> > > > Thanks for all that - definitely gives some stuff to work on and look
> > > into.
> > > >
> > > > Thanks
> > > > Joe
> > > >
> > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <[hidden email]>
> > > wrote:
> > > >
> > > > > Regardless of what you call it, writing down exactly what was done
> to
> > > > make
> > > > > a RC is extremely important early on (I know that I sure can't
> > remember
> > > > > what I did last week, much less last release). I'm not sure if
> > release
> > > > > guide is formally defined somewhere, but enough information that
> any
> > > > other
> > > > > committer can come by after "you get hit by a train" and make a
> > release
> > > > is
> > > > > extremely important. Using the CMS and making a page on the website
> > for
> > > > > this is extremely easy to do, IMO.
> > > > >
> > > > > Another concern for how to actually make a release is the type of
> > > > > packaging that is released. I not talking about the source/binary
> > > release
> > > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a
> > > tarball?
> > > > > Are there jars in Maven? etc. A tarball is pretty easy to make, and
> > > > likely
> > > > > the closest to what the build-order.sh script already does (with
> > > Maven's
> > > > > help). I'm not sure if maven-released artifacts are quite as
> > important
> > > at
> > > > > this point -- if it's not, punting on that will help. If/when you
> get
> > > to
> > > > > that point, look into the apache parent pom[1]. It is extremely
> well
> > > > > automated to use the existing infrastructure to do it all for you.
> > > > >
> > > > > Without looking at official documentation (which is me being lazy,
> > > > sorry),
> > > > > some other things that will need to be thought about:
> > > > >
> > > > > Start with the releases page [2]
> > > > >
> > > > > * You *must* include a source artifact. It cannot have binaries. No
> > > Java
> > > > > class files, no jars. A user must be able to download the source
> > > artifact
> > > > > and build it all themselves. Artifacts which include binaries are
> for
> > > > > user-convenience only and can optionally be provided as well.
> > > > >
> > > > > * LICENSE, NOTICE and README must all be included in the top level
> of
> > > the
> > > > > artifact. The NOTICE is the most painful and must include any
> > required
> > > > > third-party notices. [3]
> > > > >
> > > > > * You will need to audit your dependencies to make sure that they
> are
> > > > > ASLv2 compatible [4]
> > > > >
> > > > > * Releases must be cryptographically signed (PGP) [5]. Your public
> > key
> > > > > should be published to known sites (e.g. pgp.mit.edu) and must be
> > > listed
> > > > > in NiFi's KEYS file [6] (which does not yet exist, probably needs
> an
> > > > infra
> > > > > ticket to create your dist/ directory?).
> > > > >
> > > > > * Verify that every source file contain the proper ASL header. The
> > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful tool
> > that
> > > > can
> > > > > (and should, IMO) be integrated into your build process to prevent
> > > people
> > > > > from accidentally committing new files between releases that do not
> > > have
> > > > > the correct header.
> > > > >
> > > > > And suddenly, this is really long :). Short answer: decide what the
> > > > > release should look like (just a tarball?), start vetting your
> source
> > > > code
> > > > > for license headers, and looking into NiFi's dependencies and their
> > > > > licenses.
> > > > >
> > > > > [1] http://maven.apache.org/pom/asf/
> > > > > [2] http://www.apache.org/dev/release.html
> > > > > [3] http://www.apache.org/legal/src-headers.html
> > > > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > > > [5] http://www.apache.org/dev/release-signing.html
> > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > > > [7] http://creadur.apache.org/rat/
> > > > >
> > > > >
> > > > > Tony Kurc wrote:
> > > > >
> > > > >> I read the guide Joe linked and a lot of the sticky parts are
> marked
> > > > >> "TODO"
> > > > >> and it looks like a work in progress
> > > > >>
> > > > >> "Podlings can short circuit this process by starting out with
> > written
> > > > >> release documentation. It is strongly recommended that Podlings
> > invest
> > > > >> time
> > > > >> looking at the best practices recommended in this document. By
> > > selection
> > > > >> and modification, sections of this document can be used to quickly
> > and
> > > > >> easily bootstrap a release guide. "
> > > > >>
> > > > >> Is step one putting together a release guide?
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>
> > wrote:
> > > > >>
> > > > >>  Hello
> > > > >>>
> > > > >>> Just wanted to stir this one up a bit.  Looks like all tickets
> > pegged
> > > > as
> > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely to be
> > resolved
> > > > >>> shortly based on their comments).  So working through the release
> > > steps
> > > > >>> available on apache.org and via the link Brock sent.
> > > > >>>
> > > > >>> Anyone interested in this part of the process or who has advice
> to
> > > help
> > > > >>> us
> > > > >>> avoid landmines we're happy to hear it.
> > > > >>>
> > > > >>> Thanks
> > > > >>> Joe
> > > > >>>
> > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>
> > > wrote:
> > > > >>>
> > > > >>>  Thanks Brock this is very helpful.
> > > > >>>>
> > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
> [hidden email]>
> > > > >>>>
> > > > >>> wrote:
> > > > >>>
> > > > >>>> Hi,
> > > > >>>>>
> > > > >>>>> This is a decent guide which can be copied:
> > > > >>>>>
> > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > > > >>>>>
> > > > >>>>> Brock
> > > > >>>>>
> > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<[hidden email]>
> > > > wrote:
> > > > >>>>>
> > > > >>>>>> Folks,
> > > > >>>>>>
> > > > >>>>>> Looking at the tickets that remain which are presently tied to
> > > 0.0.1
> > > > >>>>>>
> > > > >>>>> we're
> > > > >>>>>
> > > > >>>>>> probably 1-2 weeks out from this initial release.  Can you
> > provide
> > > > >>>>>>
> > > > >>>>> some
> > > > >>>
> > > > >>>> pointers/references or pointers on how to get this ball rolling
> > and
> > > > >>>>>>
> > > > >>>>> any
> > > > >>>
> > > > >>>> rocks we must move before going down this path?
> > > > >>>>>>
> > > > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > > > >>>>>>
> > > > >>>>>> That link seems really helpful but has a lot of TODOs for
> areas
> > > > which
> > > > >>>>>>
> > > > >>>>> need
> > > > >>>>>
> > > > >>>>>> more explanation.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Thanks
> > > > >>>>>> Joe
> > > > >>>>>>
> > > > >>>>>>
> > > > >>
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joe Witt
Benson,

Well I think we have all the groundwork laid as far as I know for the
release plugin.  To be honest though (speaking for myself at least) I'm not
sure what the steps would be that the release plugin would do.  I'd love it
to be as you describe but am also concerned about how much we'd be bugging
you to figure that out.  I looked at what other projects seem to do and it
seemed shockingly manual.

If you don't mind laying out the steps in more detail when you have time
would love to learn from you on it.  I can also try toying around with it a
bit more off-line.

Thanks
Joe

On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <[hidden email]>
wrote:

> On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]> wrote:
>
> > Benson,
> >
> > Yes that would be much appreciated.
> >
> > Here is a rough gamplan of what I *think* will be needed to do the
> release:
> >
> > - Branch from develop against a ticket for the overall release process
> > - Update the version of all the artifacts and dependencies to
> > 0.0.1-RC1-incubating or something like that
> > - Create the tar.gz of the clean source tree.  Sign that and place both
> the
> > signed file and the tar.gz into my people's dir (if i have one of those)
> > - Notify folks that this is available so they can pull it and attempt to
> > build from that and validate the signature
> > - Once folks agree it is good to merge that to master
> > - Create a tar.gz of that and a signed file.  Call a vote within the
> team.
> > If that is good call a vote within IPMC.
> > - IF NO - fix whatever is wrong.
> > - If all good then do a maven deploy of the binary artifacts, source
> > bundles, and javadocs
> > - Upload the tar.gz of source and the signed file to our dist dir.  And
> > upload convenience binary build tar.gz and zip with sigs to our dist
> > folder.  Then create links to these from our downloads page.
> > - Update JIRA that the release is closed.
> > - Wash/Rinse/Repeat
> >
> > <note this all is with the understanding that we've done all the
> necessary
> > review of dependencies, licenses, properly documented them, have our
> > ECCN/encryption stuff properly filed>
> >
>
> Sure, except that I think we can make the maven-release-plugin do a lot of
> the work.
>
> The idea is that you run the release plugin, and it delivers all these
> goodies to repository.apache.org. Then you call the vote. If the vote
> passes, you (a) push the promote button to push to Maven Central, and (b)
> check the official source release tarball into the svn area for
> distribution.
>
> I will take a whack at a PR for point (a).
>
>
>
>
> >
> > Might be missing a detail or two but does that sound roughly on the right
> > rack?
> >
> > I had though the maven release plugin would be useful but looks like
> we'll
> > just be able to use the versions plugin to update them and can do the
> other
> > simple steps manually.
> >
> > Thanks
> > Joe
> >
> > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <[hidden email]>
> > wrote:
> >
> > > Typically people set the maven assembly plugin for this and include it
> in
> > > the build. Would you like me to pitch this in?
> > >
> > >
> > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]> wrote:
> > >
> > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]> wrote:
> > > >
> > > > > Josh
> > > > >
> > > > > Awesome.
> > > > >
> > > > > Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER,
> > > README.
> > > > > All dependencies have been reviewed and checked for ASLv2
> > > compatibility.
> > > > >
> > > > > Will submit the infra ticket now for the dist/ path for keys files
> > and
> > > > > such.
> > > > >
> > > > > My guess is that the release artifact should be a tarball of all
> > > source.
> > > > > Could we literally just package up a clean source tree?  Anyone
> else
> > > have
> > > > > views on this?
> > > > >
> > > >
> > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is a
> > > perfectly
> > > > reasonable thing to do.
> > > >
> > > > >
> > > > > Ideally with this release we'd do it all properly including maven
> > > > > artifacts, sources/javadocs, and so on.  The Maven build does
> already
> > > > > operate now off a single command at the root to build everything
> > > > > (build-order is gone) and inherits from the apache parent.
> > > > >
> > > > > Will need to incorporate RAT.
> > > > >
> > > > > Thanks for all that - definitely gives some stuff to work on and
> look
> > > > into.
> > > > >
> > > > > Thanks
> > > > > Joe
> > > > >
> > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <[hidden email]>
> > > > wrote:
> > > > >
> > > > > > Regardless of what you call it, writing down exactly what was
> done
> > to
> > > > > make
> > > > > > a RC is extremely important early on (I know that I sure can't
> > > remember
> > > > > > what I did last week, much less last release). I'm not sure if
> > > release
> > > > > > guide is formally defined somewhere, but enough information that
> > any
> > > > > other
> > > > > > committer can come by after "you get hit by a train" and make a
> > > release
> > > > > is
> > > > > > extremely important. Using the CMS and making a page on the
> website
> > > for
> > > > > > this is extremely easy to do, IMO.
> > > > > >
> > > > > > Another concern for how to actually make a release is the type of
> > > > > > packaging that is released. I not talking about the source/binary
> > > > release
> > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a
> > > > tarball?
> > > > > > Are there jars in Maven? etc. A tarball is pretty easy to make,
> and
> > > > > likely
> > > > > > the closest to what the build-order.sh script already does (with
> > > > Maven's
> > > > > > help). I'm not sure if maven-released artifacts are quite as
> > > important
> > > > at
> > > > > > this point -- if it's not, punting on that will help. If/when you
> > get
> > > > to
> > > > > > that point, look into the apache parent pom[1]. It is extremely
> > well
> > > > > > automated to use the existing infrastructure to do it all for
> you.
> > > > > >
> > > > > > Without looking at official documentation (which is me being
> lazy,
> > > > > sorry),
> > > > > > some other things that will need to be thought about:
> > > > > >
> > > > > > Start with the releases page [2]
> > > > > >
> > > > > > * You *must* include a source artifact. It cannot have binaries.
> No
> > > > Java
> > > > > > class files, no jars. A user must be able to download the source
> > > > artifact
> > > > > > and build it all themselves. Artifacts which include binaries are
> > for
> > > > > > user-convenience only and can optionally be provided as well.
> > > > > >
> > > > > > * LICENSE, NOTICE and README must all be included in the top
> level
> > of
> > > > the
> > > > > > artifact. The NOTICE is the most painful and must include any
> > > required
> > > > > > third-party notices. [3]
> > > > > >
> > > > > > * You will need to audit your dependencies to make sure that they
> > are
> > > > > > ASLv2 compatible [4]
> > > > > >
> > > > > > * Releases must be cryptographically signed (PGP) [5]. Your
> public
> > > key
> > > > > > should be published to known sites (e.g. pgp.mit.edu) and must
> be
> > > > listed
> > > > > > in NiFi's KEYS file [6] (which does not yet exist, probably needs
> > an
> > > > > infra
> > > > > > ticket to create your dist/ directory?).
> > > > > >
> > > > > > * Verify that every source file contain the proper ASL header.
> The
> > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful
> tool
> > > that
> > > > > can
> > > > > > (and should, IMO) be integrated into your build process to
> prevent
> > > > people
> > > > > > from accidentally committing new files between releases that do
> not
> > > > have
> > > > > > the correct header.
> > > > > >
> > > > > > And suddenly, this is really long :). Short answer: decide what
> the
> > > > > > release should look like (just a tarball?), start vetting your
> > source
> > > > > code
> > > > > > for license headers, and looking into NiFi's dependencies and
> their
> > > > > > licenses.
> > > > > >
> > > > > > [1] http://maven.apache.org/pom/asf/
> > > > > > [2] http://www.apache.org/dev/release.html
> > > > > > [3] http://www.apache.org/legal/src-headers.html
> > > > > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > > > > [5] http://www.apache.org/dev/release-signing.html
> > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > > > > [7] http://creadur.apache.org/rat/
> > > > > >
> > > > > >
> > > > > > Tony Kurc wrote:
> > > > > >
> > > > > >> I read the guide Joe linked and a lot of the sticky parts are
> > marked
> > > > > >> "TODO"
> > > > > >> and it looks like a work in progress
> > > > > >>
> > > > > >> "Podlings can short circuit this process by starting out with
> > > written
> > > > > >> release documentation. It is strongly recommended that Podlings
> > > invest
> > > > > >> time
> > > > > >> looking at the best practices recommended in this document. By
> > > > selection
> > > > > >> and modification, sections of this document can be used to
> quickly
> > > and
> > > > > >> easily bootstrap a release guide. "
> > > > > >>
> > > > > >> Is step one putting together a release guide?
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>
> > > wrote:
> > > > > >>
> > > > > >>  Hello
> > > > > >>>
> > > > > >>> Just wanted to stir this one up a bit.  Looks like all tickets
> > > pegged
> > > > > as
> > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely to be
> > > resolved
> > > > > >>> shortly based on their comments).  So working through the
> release
> > > > steps
> > > > > >>> available on apache.org and via the link Brock sent.
> > > > > >>>
> > > > > >>> Anyone interested in this part of the process or who has advice
> > to
> > > > help
> > > > > >>> us
> > > > > >>> avoid landmines we're happy to hear it.
> > > > > >>>
> > > > > >>> Thanks
> > > > > >>> Joe
> > > > > >>>
> > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]>
> > > > wrote:
> > > > > >>>
> > > > > >>>  Thanks Brock this is very helpful.
> > > > > >>>>
> > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
> > [hidden email]>
> > > > > >>>>
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>> Hi,
> > > > > >>>>>
> > > > > >>>>> This is a decent guide which can be copied:
> > > > > >>>>>
> > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > > > > >>>>>
> > > > > >>>>> Brock
> > > > > >>>>>
> > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<
> [hidden email]>
> > > > > wrote:
> > > > > >>>>>
> > > > > >>>>>> Folks,
> > > > > >>>>>>
> > > > > >>>>>> Looking at the tickets that remain which are presently tied
> to
> > > > 0.0.1
> > > > > >>>>>>
> > > > > >>>>> we're
> > > > > >>>>>
> > > > > >>>>>> probably 1-2 weeks out from this initial release.  Can you
> > > provide
> > > > > >>>>>>
> > > > > >>>>> some
> > > > > >>>
> > > > > >>>> pointers/references or pointers on how to get this ball
> rolling
> > > and
> > > > > >>>>>>
> > > > > >>>>> any
> > > > > >>>
> > > > > >>>> rocks we must move before going down this path?
> > > > > >>>>>>
> > > > > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > > > > >>>>>>
> > > > > >>>>>> That link seems really helpful but has a lot of TODOs for
> > areas
> > > > > which
> > > > > >>>>>>
> > > > > >>>>> need
> > > > > >>>>>
> > > > > >>>>>> more explanation.
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> Thanks
> > > > > >>>>>> Joe
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Benson Margulies
On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <[hidden email]> wrote:

> Benson,
>
> Well I think we have all the groundwork laid as far as I know for the
> release plugin.  To be honest though (speaking for myself at least) I'm not
> sure what the steps would be that the release plugin would do.  I'd love it
> to be as you describe but am also concerned about how much we'd be bugging
> you to figure that out.  I looked at what other projects seem to do and it
> seemed shockingly manual.
>
> If you don't mind laying out the steps in more detail when you have time
> would love to learn from you on it.  I can also try toying around with it a
> bit more off-line.
>

I don't know what projects you're looking at, but it shouldn't look so
manual for any project that has a Maven top-level build.

Here's the basic workflow of the maven-release-plugin:

release:prepare:

1. edit poms to contain 'release version'.
2. make a commit.
3. make a tag.
4. edit poms to contain 'next snapshot'
5. make a commit.
6. push the whole business (unless you turn that off).

release:perform:

7. check out from tag in target/checkout (using extra git clone)
8. build
9. deploy results to target of deploymentManagement

Now comes the next trick.

Apache runs a copy of Sonatype Nexus, that serves as a 'staging
repository'.  So, when step 9 pushes things to the repository, it's pushing
them to a special 'purgatory' staging area. That area is available for
people to look at for testing, but does not propagate along to Maven
central.

Given all this, then we add one ingredient: an additional maven module that
uses the maven-assembly-plugin to build a tarball that satisfies the
requirements of an Apache source release. This is not the same thing as
what the maven-source-plugin does at all. Since it will have
<attach>true</attach>, the resulting tarball swims upstream to the staging
repo with everything else.

The vote email points people to that location, and supplies a sha1 for the
source ball. Voters download the source release from the staging repo, do
what they need to do, and vote. Three +1's later, and then you get to
publish.

Does this help? The only manual steps are really copying to svn to push to
the Apache dist area. CXF and all the Maven plugins are examples.




>
> Thanks
> Joe
>
> On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <[hidden email]>
> wrote:
>
> > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]> wrote:
> >
> > > Benson,
> > >
> > > Yes that would be much appreciated.
> > >
> > > Here is a rough gamplan of what I *think* will be needed to do the
> > release:
> > >
> > > - Branch from develop against a ticket for the overall release process
> > > - Update the version of all the artifacts and dependencies to
> > > 0.0.1-RC1-incubating or something like that
> > > - Create the tar.gz of the clean source tree.  Sign that and place both
> > the
> > > signed file and the tar.gz into my people's dir (if i have one of
> those)
> > > - Notify folks that this is available so they can pull it and attempt
> to
> > > build from that and validate the signature
> > > - Once folks agree it is good to merge that to master
> > > - Create a tar.gz of that and a signed file.  Call a vote within the
> > team.
> > > If that is good call a vote within IPMC.
> > > - IF NO - fix whatever is wrong.
> > > - If all good then do a maven deploy of the binary artifacts, source
> > > bundles, and javadocs
> > > - Upload the tar.gz of source and the signed file to our dist dir.  And
> > > upload convenience binary build tar.gz and zip with sigs to our dist
> > > folder.  Then create links to these from our downloads page.
> > > - Update JIRA that the release is closed.
> > > - Wash/Rinse/Repeat
> > >
> > > <note this all is with the understanding that we've done all the
> > necessary
> > > review of dependencies, licenses, properly documented them, have our
> > > ECCN/encryption stuff properly filed>
> > >
> >
> > Sure, except that I think we can make the maven-release-plugin do a lot
> of
> > the work.
> >
> > The idea is that you run the release plugin, and it delivers all these
> > goodies to repository.apache.org. Then you call the vote. If the vote
> > passes, you (a) push the promote button to push to Maven Central, and (b)
> > check the official source release tarball into the svn area for
> > distribution.
> >
> > I will take a whack at a PR for point (a).
> >
> >
> >
> >
> > >
> > > Might be missing a detail or two but does that sound roughly on the
> right
> > > rack?
> > >
> > > I had though the maven release plugin would be useful but looks like
> > we'll
> > > just be able to use the versions plugin to update them and can do the
> > other
> > > simple steps manually.
> > >
> > > Thanks
> > > Joe
> > >
> > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <
> [hidden email]>
> > > wrote:
> > >
> > > > Typically people set the maven assembly plugin for this and include
> it
> > in
> > > > the build. Would you like me to pitch this in?
> > > >
> > > >
> > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]>
> wrote:
> > > >
> > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]>
> wrote:
> > > > >
> > > > > > Josh
> > > > > >
> > > > > > Awesome.
> > > > > >
> > > > > > Honestly I believe we're good on the LICENSE, NOTICE, DISCLAIMER,
> > > > README.
> > > > > > All dependencies have been reviewed and checked for ASLv2
> > > > compatibility.
> > > > > >
> > > > > > Will submit the infra ticket now for the dist/ path for keys
> files
> > > and
> > > > > > such.
> > > > > >
> > > > > > My guess is that the release artifact should be a tarball of all
> > > > source.
> > > > > > Could we literally just package up a clean source tree?  Anyone
> > else
> > > > have
> > > > > > views on this?
> > > > > >
> > > > >
> > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is a
> > > > perfectly
> > > > > reasonable thing to do.
> > > > >
> > > > > >
> > > > > > Ideally with this release we'd do it all properly including maven
> > > > > > artifacts, sources/javadocs, and so on.  The Maven build does
> > already
> > > > > > operate now off a single command at the root to build everything
> > > > > > (build-order is gone) and inherits from the apache parent.
> > > > > >
> > > > > > Will need to incorporate RAT.
> > > > > >
> > > > > > Thanks for all that - definitely gives some stuff to work on and
> > look
> > > > > into.
> > > > > >
> > > > > > Thanks
> > > > > > Joe
> > > > > >
> > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <
> [hidden email]>
> > > > > wrote:
> > > > > >
> > > > > > > Regardless of what you call it, writing down exactly what was
> > done
> > > to
> > > > > > make
> > > > > > > a RC is extremely important early on (I know that I sure can't
> > > > remember
> > > > > > > what I did last week, much less last release). I'm not sure if
> > > > release
> > > > > > > guide is formally defined somewhere, but enough information
> that
> > > any
> > > > > > other
> > > > > > > committer can come by after "you get hit by a train" and make a
> > > > release
> > > > > > is
> > > > > > > extremely important. Using the CMS and making a page on the
> > website
> > > > for
> > > > > > > this is extremely easy to do, IMO.
> > > > > > >
> > > > > > > Another concern for how to actually make a release is the type
> of
> > > > > > > packaging that is released. I not talking about the
> source/binary
> > > > > release
> > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is it a
> > > > > tarball?
> > > > > > > Are there jars in Maven? etc. A tarball is pretty easy to make,
> > and
> > > > > > likely
> > > > > > > the closest to what the build-order.sh script already does
> (with
> > > > > Maven's
> > > > > > > help). I'm not sure if maven-released artifacts are quite as
> > > > important
> > > > > at
> > > > > > > this point -- if it's not, punting on that will help. If/when
> you
> > > get
> > > > > to
> > > > > > > that point, look into the apache parent pom[1]. It is extremely
> > > well
> > > > > > > automated to use the existing infrastructure to do it all for
> > you.
> > > > > > >
> > > > > > > Without looking at official documentation (which is me being
> > lazy,
> > > > > > sorry),
> > > > > > > some other things that will need to be thought about:
> > > > > > >
> > > > > > > Start with the releases page [2]
> > > > > > >
> > > > > > > * You *must* include a source artifact. It cannot have
> binaries.
> > No
> > > > > Java
> > > > > > > class files, no jars. A user must be able to download the
> source
> > > > > artifact
> > > > > > > and build it all themselves. Artifacts which include binaries
> are
> > > for
> > > > > > > user-convenience only and can optionally be provided as well.
> > > > > > >
> > > > > > > * LICENSE, NOTICE and README must all be included in the top
> > level
> > > of
> > > > > the
> > > > > > > artifact. The NOTICE is the most painful and must include any
> > > > required
> > > > > > > third-party notices. [3]
> > > > > > >
> > > > > > > * You will need to audit your dependencies to make sure that
> they
> > > are
> > > > > > > ASLv2 compatible [4]
> > > > > > >
> > > > > > > * Releases must be cryptographically signed (PGP) [5]. Your
> > public
> > > > key
> > > > > > > should be published to known sites (e.g. pgp.mit.edu) and must
> > be
> > > > > listed
> > > > > > > in NiFi's KEYS file [6] (which does not yet exist, probably
> needs
> > > an
> > > > > > infra
> > > > > > > ticket to create your dist/ directory?).
> > > > > > >
> > > > > > > * Verify that every source file contain the proper ASL header.
> > The
> > > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful
> > tool
> > > > that
> > > > > > can
> > > > > > > (and should, IMO) be integrated into your build process to
> > prevent
> > > > > people
> > > > > > > from accidentally committing new files between releases that do
> > not
> > > > > have
> > > > > > > the correct header.
> > > > > > >
> > > > > > > And suddenly, this is really long :). Short answer: decide what
> > the
> > > > > > > release should look like (just a tarball?), start vetting your
> > > source
> > > > > > code
> > > > > > > for license headers, and looking into NiFi's dependencies and
> > their
> > > > > > > licenses.
> > > > > > >
> > > > > > > [1] http://maven.apache.org/pom/asf/
> > > > > > > [2] http://www.apache.org/dev/release.html
> > > > > > > [3] http://www.apache.org/legal/src-headers.html
> > > > > > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > > > > > [5] http://www.apache.org/dev/release-signing.html
> > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > > > > > [7] http://creadur.apache.org/rat/
> > > > > > >
> > > > > > >
> > > > > > > Tony Kurc wrote:
> > > > > > >
> > > > > > >> I read the guide Joe linked and a lot of the sticky parts are
> > > marked
> > > > > > >> "TODO"
> > > > > > >> and it looks like a work in progress
> > > > > > >>
> > > > > > >> "Podlings can short circuit this process by starting out with
> > > > written
> > > > > > >> release documentation. It is strongly recommended that
> Podlings
> > > > invest
> > > > > > >> time
> > > > > > >> looking at the best practices recommended in this document. By
> > > > > selection
> > > > > > >> and modification, sections of this document can be used to
> > quickly
> > > > and
> > > > > > >> easily bootstrap a release guide. "
> > > > > > >>
> > > > > > >> Is step one putting together a release guide?
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]>
> > > > wrote:
> > > > > > >>
> > > > > > >>  Hello
> > > > > > >>>
> > > > > > >>> Just wanted to stir this one up a bit.  Looks like all
> tickets
> > > > pegged
> > > > > > as
> > > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely to be
> > > > resolved
> > > > > > >>> shortly based on their comments).  So working through the
> > release
> > > > > steps
> > > > > > >>> available on apache.org and via the link Brock sent.
> > > > > > >>>
> > > > > > >>> Anyone interested in this part of the process or who has
> advice
> > > to
> > > > > help
> > > > > > >>> us
> > > > > > >>> avoid landmines we're happy to hear it.
> > > > > > >>>
> > > > > > >>> Thanks
> > > > > > >>> Joe
> > > > > > >>>
> > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<[hidden email]
> >
> > > > > wrote:
> > > > > > >>>
> > > > > > >>>  Thanks Brock this is very helpful.
> > > > > > >>>>
> > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
> > > [hidden email]>
> > > > > > >>>>
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>>> Hi,
> > > > > > >>>>>
> > > > > > >>>>> This is a decent guide which can be copied:
> > > > > > >>>>>
> > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > > > > > >>>>>
> > > > > > >>>>> Brock
> > > > > > >>>>>
> > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<
> > [hidden email]>
> > > > > > wrote:
> > > > > > >>>>>
> > > > > > >>>>>> Folks,
> > > > > > >>>>>>
> > > > > > >>>>>> Looking at the tickets that remain which are presently
> tied
> > to
> > > > > 0.0.1
> > > > > > >>>>>>
> > > > > > >>>>> we're
> > > > > > >>>>>
> > > > > > >>>>>> probably 1-2 weeks out from this initial release.  Can you
> > > > provide
> > > > > > >>>>>>
> > > > > > >>>>> some
> > > > > > >>>
> > > > > > >>>> pointers/references or pointers on how to get this ball
> > rolling
> > > > and
> > > > > > >>>>>>
> > > > > > >>>>> any
> > > > > > >>>
> > > > > > >>>> rocks we must move before going down this path?
> > > > > > >>>>>>
> > > > > > >>>>>> http://incubator.apache.org/guides/releasemanagement.html
> > > > > > >>>>>>
> > > > > > >>>>>> That link seems really helpful but has a lot of TODOs for
> > > areas
> > > > > > which
> > > > > > >>>>>>
> > > > > > >>>>> need
> > > > > > >>>>>
> > > > > > >>>>>> more explanation.
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> Thanks
> > > > > > >>>>>> Joe
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Adam Taft
Benson,

Quite a bit of NIFI's artifacts are more like application bundles of other
common libraries.  Most of the NIFI nars aren't really useful for
application developers to bind to in their own code.  These nars are more
part of the application and less useful as a shared common library.

One of my concerns was accidentally releasing unneeded nars into maven
central.  It sounds like the "purgatory" area solves this?  Can you speak
to this a little more?

There's been discussion on whether we care that application nars escape
into maven central - I'm not trying to weigh in on that topic (though, I
think you can read my thoughts between the lines).  I'm just asking for
more clarity on the options when using the maven release plugin.  Purgatory
might very well allow us to have our cake and eat it too.

Thanks,

Adam


On Thu, Jan 8, 2015 at 5:07 PM, Benson Margulies <[hidden email]>
wrote:

> On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <[hidden email]> wrote:
>
> > Benson,
> >
> > Well I think we have all the groundwork laid as far as I know for the
> > release plugin.  To be honest though (speaking for myself at least) I'm
> not
> > sure what the steps would be that the release plugin would do.  I'd love
> it
> > to be as you describe but am also concerned about how much we'd be
> bugging
> > you to figure that out.  I looked at what other projects seem to do and
> it
> > seemed shockingly manual.
> >
> > If you don't mind laying out the steps in more detail when you have time
> > would love to learn from you on it.  I can also try toying around with
> it a
> > bit more off-line.
> >
>
> I don't know what projects you're looking at, but it shouldn't look so
> manual for any project that has a Maven top-level build.
>
> Here's the basic workflow of the maven-release-plugin:
>
> release:prepare:
>
> 1. edit poms to contain 'release version'.
> 2. make a commit.
> 3. make a tag.
> 4. edit poms to contain 'next snapshot'
> 5. make a commit.
> 6. push the whole business (unless you turn that off).
>
> release:perform:
>
> 7. check out from tag in target/checkout (using extra git clone)
> 8. build
> 9. deploy results to target of deploymentManagement
>
> Now comes the next trick.
>
> Apache runs a copy of Sonatype Nexus, that serves as a 'staging
> repository'.  So, when step 9 pushes things to the repository, it's pushing
> them to a special 'purgatory' staging area. That area is available for
> people to look at for testing, but does not propagate along to Maven
> central.
>
> Given all this, then we add one ingredient: an additional maven module that
> uses the maven-assembly-plugin to build a tarball that satisfies the
> requirements of an Apache source release. This is not the same thing as
> what the maven-source-plugin does at all. Since it will have
> <attach>true</attach>, the resulting tarball swims upstream to the staging
> repo with everything else.
>
> The vote email points people to that location, and supplies a sha1 for the
> source ball. Voters download the source release from the staging repo, do
> what they need to do, and vote. Three +1's later, and then you get to
> publish.
>
> Does this help? The only manual steps are really copying to svn to push to
> the Apache dist area. CXF and all the Maven plugins are examples.
>
>
>
>
> >
> > Thanks
> > Joe
> >
> > On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <[hidden email]>
> > wrote:
> >
> > > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]> wrote:
> > >
> > > > Benson,
> > > >
> > > > Yes that would be much appreciated.
> > > >
> > > > Here is a rough gamplan of what I *think* will be needed to do the
> > > release:
> > > >
> > > > - Branch from develop against a ticket for the overall release
> process
> > > > - Update the version of all the artifacts and dependencies to
> > > > 0.0.1-RC1-incubating or something like that
> > > > - Create the tar.gz of the clean source tree.  Sign that and place
> both
> > > the
> > > > signed file and the tar.gz into my people's dir (if i have one of
> > those)
> > > > - Notify folks that this is available so they can pull it and attempt
> > to
> > > > build from that and validate the signature
> > > > - Once folks agree it is good to merge that to master
> > > > - Create a tar.gz of that and a signed file.  Call a vote within the
> > > team.
> > > > If that is good call a vote within IPMC.
> > > > - IF NO - fix whatever is wrong.
> > > > - If all good then do a maven deploy of the binary artifacts, source
> > > > bundles, and javadocs
> > > > - Upload the tar.gz of source and the signed file to our dist dir.
> And
> > > > upload convenience binary build tar.gz and zip with sigs to our dist
> > > > folder.  Then create links to these from our downloads page.
> > > > - Update JIRA that the release is closed.
> > > > - Wash/Rinse/Repeat
> > > >
> > > > <note this all is with the understanding that we've done all the
> > > necessary
> > > > review of dependencies, licenses, properly documented them, have our
> > > > ECCN/encryption stuff properly filed>
> > > >
> > >
> > > Sure, except that I think we can make the maven-release-plugin do a lot
> > of
> > > the work.
> > >
> > > The idea is that you run the release plugin, and it delivers all these
> > > goodies to repository.apache.org. Then you call the vote. If the vote
> > > passes, you (a) push the promote button to push to Maven Central, and
> (b)
> > > check the official source release tarball into the svn area for
> > > distribution.
> > >
> > > I will take a whack at a PR for point (a).
> > >
> > >
> > >
> > >
> > > >
> > > > Might be missing a detail or two but does that sound roughly on the
> > right
> > > > rack?
> > > >
> > > > I had though the maven release plugin would be useful but looks like
> > > we'll
> > > > just be able to use the versions plugin to update them and can do the
> > > other
> > > > simple steps manually.
> > > >
> > > > Thanks
> > > > Joe
> > > >
> > > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <
> > [hidden email]>
> > > > wrote:
> > > >
> > > > > Typically people set the maven assembly plugin for this and include
> > it
> > > in
> > > > > the build. Would you like me to pitch this in?
> > > > >
> > > > >
> > > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]>
> > wrote:
> > > > >
> > > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]>
> > wrote:
> > > > > >
> > > > > > > Josh
> > > > > > >
> > > > > > > Awesome.
> > > > > > >
> > > > > > > Honestly I believe we're good on the LICENSE, NOTICE,
> DISCLAIMER,
> > > > > README.
> > > > > > > All dependencies have been reviewed and checked for ASLv2
> > > > > compatibility.
> > > > > > >
> > > > > > > Will submit the infra ticket now for the dist/ path for keys
> > files
> > > > and
> > > > > > > such.
> > > > > > >
> > > > > > > My guess is that the release artifact should be a tarball of
> all
> > > > > source.
> > > > > > > Could we literally just package up a clean source tree?  Anyone
> > > else
> > > > > have
> > > > > > > views on this?
> > > > > > >
> > > > > >
> > > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is a
> > > > > perfectly
> > > > > > reasonable thing to do.
> > > > > >
> > > > > > >
> > > > > > > Ideally with this release we'd do it all properly including
> maven
> > > > > > > artifacts, sources/javadocs, and so on.  The Maven build does
> > > already
> > > > > > > operate now off a single command at the root to build
> everything
> > > > > > > (build-order is gone) and inherits from the apache parent.
> > > > > > >
> > > > > > > Will need to incorporate RAT.
> > > > > > >
> > > > > > > Thanks for all that - definitely gives some stuff to work on
> and
> > > look
> > > > > > into.
> > > > > > >
> > > > > > > Thanks
> > > > > > > Joe
> > > > > > >
> > > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <
> > [hidden email]>
> > > > > > wrote:
> > > > > > >
> > > > > > > > Regardless of what you call it, writing down exactly what was
> > > done
> > > > to
> > > > > > > make
> > > > > > > > a RC is extremely important early on (I know that I sure
> can't
> > > > > remember
> > > > > > > > what I did last week, much less last release). I'm not sure
> if
> > > > > release
> > > > > > > > guide is formally defined somewhere, but enough information
> > that
> > > > any
> > > > > > > other
> > > > > > > > committer can come by after "you get hit by a train" and
> make a
> > > > > release
> > > > > > > is
> > > > > > > > extremely important. Using the CMS and making a page on the
> > > website
> > > > > for
> > > > > > > > this is extremely easy to do, IMO.
> > > > > > > >
> > > > > > > > Another concern for how to actually make a release is the
> type
> > of
> > > > > > > > packaging that is released. I not talking about the
> > source/binary
> > > > > > release
> > > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is
> it a
> > > > > > tarball?
> > > > > > > > Are there jars in Maven? etc. A tarball is pretty easy to
> make,
> > > and
> > > > > > > likely
> > > > > > > > the closest to what the build-order.sh script already does
> > (with
> > > > > > Maven's
> > > > > > > > help). I'm not sure if maven-released artifacts are quite as
> > > > > important
> > > > > > at
> > > > > > > > this point -- if it's not, punting on that will help. If/when
> > you
> > > > get
> > > > > > to
> > > > > > > > that point, look into the apache parent pom[1]. It is
> extremely
> > > > well
> > > > > > > > automated to use the existing infrastructure to do it all for
> > > you.
> > > > > > > >
> > > > > > > > Without looking at official documentation (which is me being
> > > lazy,
> > > > > > > sorry),
> > > > > > > > some other things that will need to be thought about:
> > > > > > > >
> > > > > > > > Start with the releases page [2]
> > > > > > > >
> > > > > > > > * You *must* include a source artifact. It cannot have
> > binaries.
> > > No
> > > > > > Java
> > > > > > > > class files, no jars. A user must be able to download the
> > source
> > > > > > artifact
> > > > > > > > and build it all themselves. Artifacts which include binaries
> > are
> > > > for
> > > > > > > > user-convenience only and can optionally be provided as well.
> > > > > > > >
> > > > > > > > * LICENSE, NOTICE and README must all be included in the top
> > > level
> > > > of
> > > > > > the
> > > > > > > > artifact. The NOTICE is the most painful and must include any
> > > > > required
> > > > > > > > third-party notices. [3]
> > > > > > > >
> > > > > > > > * You will need to audit your dependencies to make sure that
> > they
> > > > are
> > > > > > > > ASLv2 compatible [4]
> > > > > > > >
> > > > > > > > * Releases must be cryptographically signed (PGP) [5]. Your
> > > public
> > > > > key
> > > > > > > > should be published to known sites (e.g. pgp.mit.edu) and
> must
> > > be
> > > > > > listed
> > > > > > > > in NiFi's KEYS file [6] (which does not yet exist, probably
> > needs
> > > > an
> > > > > > > infra
> > > > > > > > ticket to create your dist/ directory?).
> > > > > > > >
> > > > > > > > * Verify that every source file contain the proper ASL
> header.
> > > The
> > > > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a wonderful
> > > tool
> > > > > that
> > > > > > > can
> > > > > > > > (and should, IMO) be integrated into your build process to
> > > prevent
> > > > > > people
> > > > > > > > from accidentally committing new files between releases that
> do
> > > not
> > > > > > have
> > > > > > > > the correct header.
> > > > > > > >
> > > > > > > > And suddenly, this is really long :). Short answer: decide
> what
> > > the
> > > > > > > > release should look like (just a tarball?), start vetting
> your
> > > > source
> > > > > > > code
> > > > > > > > for license headers, and looking into NiFi's dependencies and
> > > their
> > > > > > > > licenses.
> > > > > > > >
> > > > > > > > [1] http://maven.apache.org/pom/asf/
> > > > > > > > [2] http://www.apache.org/dev/release.html
> > > > > > > > [3] http://www.apache.org/legal/src-headers.html
> > > > > > > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > > > > > > [5] http://www.apache.org/dev/release-signing.html
> > > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > > > > > > [7] http://creadur.apache.org/rat/
> > > > > > > >
> > > > > > > >
> > > > > > > > Tony Kurc wrote:
> > > > > > > >
> > > > > > > >> I read the guide Joe linked and a lot of the sticky parts
> are
> > > > marked
> > > > > > > >> "TODO"
> > > > > > > >> and it looks like a work in progress
> > > > > > > >>
> > > > > > > >> "Podlings can short circuit this process by starting out
> with
> > > > > written
> > > > > > > >> release documentation. It is strongly recommended that
> > Podlings
> > > > > invest
> > > > > > > >> time
> > > > > > > >> looking at the best practices recommended in this document.
> By
> > > > > > selection
> > > > > > > >> and modification, sections of this document can be used to
> > > quickly
> > > > > and
> > > > > > > >> easily bootstrap a release guide. "
> > > > > > > >>
> > > > > > > >> Is step one putting together a release guide?
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<[hidden email]
> >
> > > > > wrote:
> > > > > > > >>
> > > > > > > >>  Hello
> > > > > > > >>>
> > > > > > > >>> Just wanted to stir this one up a bit.  Looks like all
> > tickets
> > > > > pegged
> > > > > > > as
> > > > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely to
> be
> > > > > resolved
> > > > > > > >>> shortly based on their comments).  So working through the
> > > release
> > > > > > steps
> > > > > > > >>> available on apache.org and via the link Brock sent.
> > > > > > > >>>
> > > > > > > >>> Anyone interested in this part of the process or who has
> > advice
> > > > to
> > > > > > help
> > > > > > > >>> us
> > > > > > > >>> avoid landmines we're happy to hear it.
> > > > > > > >>>
> > > > > > > >>> Thanks
> > > > > > > >>> Joe
> > > > > > > >>>
> > > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<
> [hidden email]
> > >
> > > > > > wrote:
> > > > > > > >>>
> > > > > > > >>>  Thanks Brock this is very helpful.
> > > > > > > >>>>
> > > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
> > > > [hidden email]>
> > > > > > > >>>>
> > > > > > > >>> wrote:
> > > > > > > >>>
> > > > > > > >>>> Hi,
> > > > > > > >>>>>
> > > > > > > >>>>> This is a decent guide which can be copied:
> > > > > > > >>>>>
> > > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > > > > > > >>>>>
> > > > > > > >>>>> Brock
> > > > > > > >>>>>
> > > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<
> > > [hidden email]>
> > > > > > > wrote:
> > > > > > > >>>>>
> > > > > > > >>>>>> Folks,
> > > > > > > >>>>>>
> > > > > > > >>>>>> Looking at the tickets that remain which are presently
> > tied
> > > to
> > > > > > 0.0.1
> > > > > > > >>>>>>
> > > > > > > >>>>> we're
> > > > > > > >>>>>
> > > > > > > >>>>>> probably 1-2 weeks out from this initial release.  Can
> you
> > > > > provide
> > > > > > > >>>>>>
> > > > > > > >>>>> some
> > > > > > > >>>
> > > > > > > >>>> pointers/references or pointers on how to get this ball
> > > rolling
> > > > > and
> > > > > > > >>>>>>
> > > > > > > >>>>> any
> > > > > > > >>>
> > > > > > > >>>> rocks we must move before going down this path?
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> http://incubator.apache.org/guides/releasemanagement.html
> > > > > > > >>>>>>
> > > > > > > >>>>>> That link seems really helpful but has a lot of TODOs
> for
> > > > areas
> > > > > > > which
> > > > > > > >>>>>>
> > > > > > > >>>>> need
> > > > > > > >>>>>
> > > > > > > >>>>>> more explanation.
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>>>>> Thanks
> > > > > > > >>>>>> Joe
> > > > > > > >>>>>>
> > > > > > > >>>>>>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Benson Margulies
If you don't want to release it you need to not deploy it. That is a pom
change unrelated to the staging repo.
On Jan 8, 2015 5:37 PM, "Adam Taft" <[hidden email]> wrote:

> Benson,
>
> Quite a bit of NIFI's artifacts are more like application bundles of other
> common libraries.  Most of the NIFI nars aren't really useful for
> application developers to bind to in their own code.  These nars are more
> part of the application and less useful as a shared common library.
>
> One of my concerns was accidentally releasing unneeded nars into maven
> central.  It sounds like the "purgatory" area solves this?  Can you speak
> to this a little more?
>
> There's been discussion on whether we care that application nars escape
> into maven central - I'm not trying to weigh in on that topic (though, I
> think you can read my thoughts between the lines).  I'm just asking for
> more clarity on the options when using the maven release plugin.  Purgatory
> might very well allow us to have our cake and eat it too.
>
> Thanks,
>
> Adam
>
>
> On Thu, Jan 8, 2015 at 5:07 PM, Benson Margulies <[hidden email]>
> wrote:
>
> > On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <[hidden email]> wrote:
> >
> > > Benson,
> > >
> > > Well I think we have all the groundwork laid as far as I know for the
> > > release plugin.  To be honest though (speaking for myself at least) I'm
> > not
> > > sure what the steps would be that the release plugin would do.  I'd
> love
> > it
> > > to be as you describe but am also concerned about how much we'd be
> > bugging
> > > you to figure that out.  I looked at what other projects seem to do and
> > it
> > > seemed shockingly manual.
> > >
> > > If you don't mind laying out the steps in more detail when you have
> time
> > > would love to learn from you on it.  I can also try toying around with
> > it a
> > > bit more off-line.
> > >
> >
> > I don't know what projects you're looking at, but it shouldn't look so
> > manual for any project that has a Maven top-level build.
> >
> > Here's the basic workflow of the maven-release-plugin:
> >
> > release:prepare:
> >
> > 1. edit poms to contain 'release version'.
> > 2. make a commit.
> > 3. make a tag.
> > 4. edit poms to contain 'next snapshot'
> > 5. make a commit.
> > 6. push the whole business (unless you turn that off).
> >
> > release:perform:
> >
> > 7. check out from tag in target/checkout (using extra git clone)
> > 8. build
> > 9. deploy results to target of deploymentManagement
> >
> > Now comes the next trick.
> >
> > Apache runs a copy of Sonatype Nexus, that serves as a 'staging
> > repository'.  So, when step 9 pushes things to the repository, it's
> pushing
> > them to a special 'purgatory' staging area. That area is available for
> > people to look at for testing, but does not propagate along to Maven
> > central.
> >
> > Given all this, then we add one ingredient: an additional maven module
> that
> > uses the maven-assembly-plugin to build a tarball that satisfies the
> > requirements of an Apache source release. This is not the same thing as
> > what the maven-source-plugin does at all. Since it will have
> > <attach>true</attach>, the resulting tarball swims upstream to the
> staging
> > repo with everything else.
> >
> > The vote email points people to that location, and supplies a sha1 for
> the
> > source ball. Voters download the source release from the staging repo, do
> > what they need to do, and vote. Three +1's later, and then you get to
> > publish.
> >
> > Does this help? The only manual steps are really copying to svn to push
> to
> > the Apache dist area. CXF and all the Maven plugins are examples.
> >
> >
> >
> >
> > >
> > > Thanks
> > > Joe
> > >
> > > On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <
> [hidden email]>
> > > wrote:
> > >
> > > > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]> wrote:
> > > >
> > > > > Benson,
> > > > >
> > > > > Yes that would be much appreciated.
> > > > >
> > > > > Here is a rough gamplan of what I *think* will be needed to do the
> > > > release:
> > > > >
> > > > > - Branch from develop against a ticket for the overall release
> > process
> > > > > - Update the version of all the artifacts and dependencies to
> > > > > 0.0.1-RC1-incubating or something like that
> > > > > - Create the tar.gz of the clean source tree.  Sign that and place
> > both
> > > > the
> > > > > signed file and the tar.gz into my people's dir (if i have one of
> > > those)
> > > > > - Notify folks that this is available so they can pull it and
> attempt
> > > to
> > > > > build from that and validate the signature
> > > > > - Once folks agree it is good to merge that to master
> > > > > - Create a tar.gz of that and a signed file.  Call a vote within
> the
> > > > team.
> > > > > If that is good call a vote within IPMC.
> > > > > - IF NO - fix whatever is wrong.
> > > > > - If all good then do a maven deploy of the binary artifacts,
> source
> > > > > bundles, and javadocs
> > > > > - Upload the tar.gz of source and the signed file to our dist dir.
> > And
> > > > > upload convenience binary build tar.gz and zip with sigs to our
> dist
> > > > > folder.  Then create links to these from our downloads page.
> > > > > - Update JIRA that the release is closed.
> > > > > - Wash/Rinse/Repeat
> > > > >
> > > > > <note this all is with the understanding that we've done all the
> > > > necessary
> > > > > review of dependencies, licenses, properly documented them, have
> our
> > > > > ECCN/encryption stuff properly filed>
> > > > >
> > > >
> > > > Sure, except that I think we can make the maven-release-plugin do a
> lot
> > > of
> > > > the work.
> > > >
> > > > The idea is that you run the release plugin, and it delivers all
> these
> > > > goodies to repository.apache.org. Then you call the vote. If the
> vote
> > > > passes, you (a) push the promote button to push to Maven Central, and
> > (b)
> > > > check the official source release tarball into the svn area for
> > > > distribution.
> > > >
> > > > I will take a whack at a PR for point (a).
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > Might be missing a detail or two but does that sound roughly on the
> > > right
> > > > > rack?
> > > > >
> > > > > I had though the maven release plugin would be useful but looks
> like
> > > > we'll
> > > > > just be able to use the versions plugin to update them and can do
> the
> > > > other
> > > > > simple steps manually.
> > > > >
> > > > > Thanks
> > > > > Joe
> > > > >
> > > > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <
> > > [hidden email]>
> > > > > wrote:
> > > > >
> > > > > > Typically people set the maven assembly plugin for this and
> include
> > > it
> > > > in
> > > > > > the build. Would you like me to pitch this in?
> > > > > >
> > > > > >
> > > > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]>
> > > wrote:
> > > > > >
> > > > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]>
> > > wrote:
> > > > > > >
> > > > > > > > Josh
> > > > > > > >
> > > > > > > > Awesome.
> > > > > > > >
> > > > > > > > Honestly I believe we're good on the LICENSE, NOTICE,
> > DISCLAIMER,
> > > > > > README.
> > > > > > > > All dependencies have been reviewed and checked for ASLv2
> > > > > > compatibility.
> > > > > > > >
> > > > > > > > Will submit the infra ticket now for the dist/ path for keys
> > > files
> > > > > and
> > > > > > > > such.
> > > > > > > >
> > > > > > > > My guess is that the release artifact should be a tarball of
> > all
> > > > > > source.
> > > > > > > > Could we literally just package up a clean source tree?
> Anyone
> > > > else
> > > > > > have
> > > > > > > > views on this?
> > > > > > > >
> > > > > > >
> > > > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag] is
> a
> > > > > > perfectly
> > > > > > > reasonable thing to do.
> > > > > > >
> > > > > > > >
> > > > > > > > Ideally with this release we'd do it all properly including
> > maven
> > > > > > > > artifacts, sources/javadocs, and so on.  The Maven build does
> > > > already
> > > > > > > > operate now off a single command at the root to build
> > everything
> > > > > > > > (build-order is gone) and inherits from the apache parent.
> > > > > > > >
> > > > > > > > Will need to incorporate RAT.
> > > > > > > >
> > > > > > > > Thanks for all that - definitely gives some stuff to work on
> > and
> > > > look
> > > > > > > into.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Joe
> > > > > > > >
> > > > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <
> > > [hidden email]>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Regardless of what you call it, writing down exactly what
> was
> > > > done
> > > > > to
> > > > > > > > make
> > > > > > > > > a RC is extremely important early on (I know that I sure
> > can't
> > > > > > remember
> > > > > > > > > what I did last week, much less last release). I'm not sure
> > if
> > > > > > release
> > > > > > > > > guide is formally defined somewhere, but enough information
> > > that
> > > > > any
> > > > > > > > other
> > > > > > > > > committer can come by after "you get hit by a train" and
> > make a
> > > > > > release
> > > > > > > > is
> > > > > > > > > extremely important. Using the CMS and making a page on the
> > > > website
> > > > > > for
> > > > > > > > > this is extremely easy to do, IMO.
> > > > > > > > >
> > > > > > > > > Another concern for how to actually make a release is the
> > type
> > > of
> > > > > > > > > packaging that is released. I not talking about the
> > > source/binary
> > > > > > > release
> > > > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is
> > it a
> > > > > > > tarball?
> > > > > > > > > Are there jars in Maven? etc. A tarball is pretty easy to
> > make,
> > > > and
> > > > > > > > likely
> > > > > > > > > the closest to what the build-order.sh script already does
> > > (with
> > > > > > > Maven's
> > > > > > > > > help). I'm not sure if maven-released artifacts are quite
> as
> > > > > > important
> > > > > > > at
> > > > > > > > > this point -- if it's not, punting on that will help.
> If/when
> > > you
> > > > > get
> > > > > > > to
> > > > > > > > > that point, look into the apache parent pom[1]. It is
> > extremely
> > > > > well
> > > > > > > > > automated to use the existing infrastructure to do it all
> for
> > > > you.
> > > > > > > > >
> > > > > > > > > Without looking at official documentation (which is me
> being
> > > > lazy,
> > > > > > > > sorry),
> > > > > > > > > some other things that will need to be thought about:
> > > > > > > > >
> > > > > > > > > Start with the releases page [2]
> > > > > > > > >
> > > > > > > > > * You *must* include a source artifact. It cannot have
> > > binaries.
> > > > No
> > > > > > > Java
> > > > > > > > > class files, no jars. A user must be able to download the
> > > source
> > > > > > > artifact
> > > > > > > > > and build it all themselves. Artifacts which include
> binaries
> > > are
> > > > > for
> > > > > > > > > user-convenience only and can optionally be provided as
> well.
> > > > > > > > >
> > > > > > > > > * LICENSE, NOTICE and README must all be included in the
> top
> > > > level
> > > > > of
> > > > > > > the
> > > > > > > > > artifact. The NOTICE is the most painful and must include
> any
> > > > > > required
> > > > > > > > > third-party notices. [3]
> > > > > > > > >
> > > > > > > > > * You will need to audit your dependencies to make sure
> that
> > > they
> > > > > are
> > > > > > > > > ASLv2 compatible [4]
> > > > > > > > >
> > > > > > > > > * Releases must be cryptographically signed (PGP) [5]. Your
> > > > public
> > > > > > key
> > > > > > > > > should be published to known sites (e.g. pgp.mit.edu) and
> > must
> > > > be
> > > > > > > listed
> > > > > > > > > in NiFi's KEYS file [6] (which does not yet exist, probably
> > > needs
> > > > > an
> > > > > > > > infra
> > > > > > > > > ticket to create your dist/ directory?).
> > > > > > > > >
> > > > > > > > > * Verify that every source file contain the proper ASL
> > header.
> > > > The
> > > > > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a
> wonderful
> > > > tool
> > > > > > that
> > > > > > > > can
> > > > > > > > > (and should, IMO) be integrated into your build process to
> > > > prevent
> > > > > > > people
> > > > > > > > > from accidentally committing new files between releases
> that
> > do
> > > > not
> > > > > > > have
> > > > > > > > > the correct header.
> > > > > > > > >
> > > > > > > > > And suddenly, this is really long :). Short answer: decide
> > what
> > > > the
> > > > > > > > > release should look like (just a tarball?), start vetting
> > your
> > > > > source
> > > > > > > > code
> > > > > > > > > for license headers, and looking into NiFi's dependencies
> and
> > > > their
> > > > > > > > > licenses.
> > > > > > > > >
> > > > > > > > > [1] http://maven.apache.org/pom/asf/
> > > > > > > > > [2] http://www.apache.org/dev/release.html
> > > > > > > > > [3] http://www.apache.org/legal/src-headers.html
> > > > > > > > > [4] http://www.apache.org/legal/resolved.html#category-x
> > > > > > > > > [5] http://www.apache.org/dev/release-signing.html
> > > > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> > > > > > > > > [7] http://creadur.apache.org/rat/
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Tony Kurc wrote:
> > > > > > > > >
> > > > > > > > >> I read the guide Joe linked and a lot of the sticky parts
> > are
> > > > > marked
> > > > > > > > >> "TODO"
> > > > > > > > >> and it looks like a work in progress
> > > > > > > > >>
> > > > > > > > >> "Podlings can short circuit this process by starting out
> > with
> > > > > > written
> > > > > > > > >> release documentation. It is strongly recommended that
> > > Podlings
> > > > > > invest
> > > > > > > > >> time
> > > > > > > > >> looking at the best practices recommended in this
> document.
> > By
> > > > > > > selection
> > > > > > > > >> and modification, sections of this document can be used to
> > > > quickly
> > > > > > and
> > > > > > > > >> easily bootstrap a release guide. "
> > > > > > > > >>
> > > > > > > > >> Is step one putting together a release guide?
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<
> [hidden email]
> > >
> > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >>  Hello
> > > > > > > > >>>
> > > > > > > > >>> Just wanted to stir this one up a bit.  Looks like all
> > > tickets
> > > > > > pegged
> > > > > > > > as
> > > > > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely to
> > be
> > > > > > resolved
> > > > > > > > >>> shortly based on their comments).  So working through the
> > > > release
> > > > > > > steps
> > > > > > > > >>> available on apache.org and via the link Brock sent.
> > > > > > > > >>>
> > > > > > > > >>> Anyone interested in this part of the process or who has
> > > advice
> > > > > to
> > > > > > > help
> > > > > > > > >>> us
> > > > > > > > >>> avoid landmines we're happy to hear it.
> > > > > > > > >>>
> > > > > > > > >>> Thanks
> > > > > > > > >>> Joe
> > > > > > > > >>>
> > > > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<
> > [hidden email]
> > > >
> > > > > > > wrote:
> > > > > > > > >>>
> > > > > > > > >>>  Thanks Brock this is very helpful.
> > > > > > > > >>>>
> > > > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
> > > > > [hidden email]>
> > > > > > > > >>>>
> > > > > > > > >>> wrote:
> > > > > > > > >>>
> > > > > > > > >>>> Hi,
> > > > > > > > >>>>>
> > > > > > > > >>>>> This is a decent guide which can be copied:
> > > > > > > > >>>>>
> > > > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> > > > > > > > >>>>>
> > > > > > > > >>>>> Brock
> > > > > > > > >>>>>
> > > > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<
> > > > [hidden email]>
> > > > > > > > wrote:
> > > > > > > > >>>>>
> > > > > > > > >>>>>> Folks,
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Looking at the tickets that remain which are presently
> > > tied
> > > > to
> > > > > > > 0.0.1
> > > > > > > > >>>>>>
> > > > > > > > >>>>> we're
> > > > > > > > >>>>>
> > > > > > > > >>>>>> probably 1-2 weeks out from this initial release.  Can
> > you
> > > > > > provide
> > > > > > > > >>>>>>
> > > > > > > > >>>>> some
> > > > > > > > >>>
> > > > > > > > >>>> pointers/references or pointers on how to get this ball
> > > > rolling
> > > > > > and
> > > > > > > > >>>>>>
> > > > > > > > >>>>> any
> > > > > > > > >>>
> > > > > > > > >>>> rocks we must move before going down this path?
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > http://incubator.apache.org/guides/releasemanagement.html
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> That link seems really helpful but has a lot of TODOs
> > for
> > > > > areas
> > > > > > > > which
> > > > > > > > >>>>>>
> > > > > > > > >>>>> need
> > > > > > > > >>>>>
> > > > > > > > >>>>>> more explanation.
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>>>>> Thanks
> > > > > > > > >>>>>> Joe
> > > > > > > > >>>>>>
> > > > > > > > >>>>>>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Benson Margulies
To expand,

On the one hand, people dump an amazing variety of crud onto Central.
There's no special reason for NiFi to have more of a conscience than anyone
else.

On the other hand, if you want to use Maven to build things but not have
them push to central, you control:

    the install plugin
    the deploy plugin

and the 'attach' parameters of some other plugins, like the war plugin and
the assembly plugin. Roughly, if you want to avoid treating the primary
artifact of a project as a maven artifact, you have to mess with install
and deploy.

It would be good to sort this out before tackling any other release issues.



On Thu, Jan 8, 2015 at 6:32 PM, Benson Margulies <[hidden email]>
wrote:

> If you don't want to release it you need to not deploy it. That is a pom
> change unrelated to the staging repo.
> On Jan 8, 2015 5:37 PM, "Adam Taft" <[hidden email]> wrote:
>
>> Benson,
>>
>> Quite a bit of NIFI's artifacts are more like application bundles of other
>> common libraries.  Most of the NIFI nars aren't really useful for
>> application developers to bind to in their own code.  These nars are more
>> part of the application and less useful as a shared common library.
>>
>> One of my concerns was accidentally releasing unneeded nars into maven
>> central.  It sounds like the "purgatory" area solves this?  Can you speak
>> to this a little more?
>>
>> There's been discussion on whether we care that application nars escape
>> into maven central - I'm not trying to weigh in on that topic (though, I
>> think you can read my thoughts between the lines).  I'm just asking for
>> more clarity on the options when using the maven release plugin.
>> Purgatory
>> might very well allow us to have our cake and eat it too.
>>
>> Thanks,
>>
>> Adam
>>
>>
>> On Thu, Jan 8, 2015 at 5:07 PM, Benson Margulies <[hidden email]>
>> wrote:
>>
>> > On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <[hidden email]> wrote:
>> >
>> > > Benson,
>> > >
>> > > Well I think we have all the groundwork laid as far as I know for the
>> > > release plugin.  To be honest though (speaking for myself at least)
>> I'm
>> > not
>> > > sure what the steps would be that the release plugin would do.  I'd
>> love
>> > it
>> > > to be as you describe but am also concerned about how much we'd be
>> > bugging
>> > > you to figure that out.  I looked at what other projects seem to do
>> and
>> > it
>> > > seemed shockingly manual.
>> > >
>> > > If you don't mind laying out the steps in more detail when you have
>> time
>> > > would love to learn from you on it.  I can also try toying around with
>> > it a
>> > > bit more off-line.
>> > >
>> >
>> > I don't know what projects you're looking at, but it shouldn't look so
>> > manual for any project that has a Maven top-level build.
>> >
>> > Here's the basic workflow of the maven-release-plugin:
>> >
>> > release:prepare:
>> >
>> > 1. edit poms to contain 'release version'.
>> > 2. make a commit.
>> > 3. make a tag.
>> > 4. edit poms to contain 'next snapshot'
>> > 5. make a commit.
>> > 6. push the whole business (unless you turn that off).
>> >
>> > release:perform:
>> >
>> > 7. check out from tag in target/checkout (using extra git clone)
>> > 8. build
>> > 9. deploy results to target of deploymentManagement
>> >
>> > Now comes the next trick.
>> >
>> > Apache runs a copy of Sonatype Nexus, that serves as a 'staging
>> > repository'.  So, when step 9 pushes things to the repository, it's
>> pushing
>> > them to a special 'purgatory' staging area. That area is available for
>> > people to look at for testing, but does not propagate along to Maven
>> > central.
>> >
>> > Given all this, then we add one ingredient: an additional maven module
>> that
>> > uses the maven-assembly-plugin to build a tarball that satisfies the
>> > requirements of an Apache source release. This is not the same thing as
>> > what the maven-source-plugin does at all. Since it will have
>> > <attach>true</attach>, the resulting tarball swims upstream to the
>> staging
>> > repo with everything else.
>> >
>> > The vote email points people to that location, and supplies a sha1 for
>> the
>> > source ball. Voters download the source release from the staging repo,
>> do
>> > what they need to do, and vote. Three +1's later, and then you get to
>> > publish.
>> >
>> > Does this help? The only manual steps are really copying to svn to push
>> to
>> > the Apache dist area. CXF and all the Maven plugins are examples.
>> >
>> >
>> >
>> >
>> > >
>> > > Thanks
>> > > Joe
>> > >
>> > > On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <
>> [hidden email]>
>> > > wrote:
>> > >
>> > > > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]>
>> wrote:
>> > > >
>> > > > > Benson,
>> > > > >
>> > > > > Yes that would be much appreciated.
>> > > > >
>> > > > > Here is a rough gamplan of what I *think* will be needed to do the
>> > > > release:
>> > > > >
>> > > > > - Branch from develop against a ticket for the overall release
>> > process
>> > > > > - Update the version of all the artifacts and dependencies to
>> > > > > 0.0.1-RC1-incubating or something like that
>> > > > > - Create the tar.gz of the clean source tree.  Sign that and place
>> > both
>> > > > the
>> > > > > signed file and the tar.gz into my people's dir (if i have one of
>> > > those)
>> > > > > - Notify folks that this is available so they can pull it and
>> attempt
>> > > to
>> > > > > build from that and validate the signature
>> > > > > - Once folks agree it is good to merge that to master
>> > > > > - Create a tar.gz of that and a signed file.  Call a vote within
>> the
>> > > > team.
>> > > > > If that is good call a vote within IPMC.
>> > > > > - IF NO - fix whatever is wrong.
>> > > > > - If all good then do a maven deploy of the binary artifacts,
>> source
>> > > > > bundles, and javadocs
>> > > > > - Upload the tar.gz of source and the signed file to our dist dir.
>> > And
>> > > > > upload convenience binary build tar.gz and zip with sigs to our
>> dist
>> > > > > folder.  Then create links to these from our downloads page.
>> > > > > - Update JIRA that the release is closed.
>> > > > > - Wash/Rinse/Repeat
>> > > > >
>> > > > > <note this all is with the understanding that we've done all the
>> > > > necessary
>> > > > > review of dependencies, licenses, properly documented them, have
>> our
>> > > > > ECCN/encryption stuff properly filed>
>> > > > >
>> > > >
>> > > > Sure, except that I think we can make the maven-release-plugin do a
>> lot
>> > > of
>> > > > the work.
>> > > >
>> > > > The idea is that you run the release plugin, and it delivers all
>> these
>> > > > goodies to repository.apache.org. Then you call the vote. If the
>> vote
>> > > > passes, you (a) push the promote button to push to Maven Central,
>> and
>> > (b)
>> > > > check the official source release tarball into the svn area for
>> > > > distribution.
>> > > >
>> > > > I will take a whack at a PR for point (a).
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > >
>> > > > > Might be missing a detail or two but does that sound roughly on
>> the
>> > > right
>> > > > > rack?
>> > > > >
>> > > > > I had though the maven release plugin would be useful but looks
>> like
>> > > > we'll
>> > > > > just be able to use the versions plugin to update them and can do
>> the
>> > > > other
>> > > > > simple steps manually.
>> > > > >
>> > > > > Thanks
>> > > > > Joe
>> > > > >
>> > > > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <
>> > > [hidden email]>
>> > > > > wrote:
>> > > > >
>> > > > > > Typically people set the maven assembly plugin for this and
>> include
>> > > it
>> > > > in
>> > > > > > the build. Would you like me to pitch this in?
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <[hidden email]>
>> > > wrote:
>> > > > > >
>> > > > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <[hidden email]>
>> > > wrote:
>> > > > > > >
>> > > > > > > > Josh
>> > > > > > > >
>> > > > > > > > Awesome.
>> > > > > > > >
>> > > > > > > > Honestly I believe we're good on the LICENSE, NOTICE,
>> > DISCLAIMER,
>> > > > > > README.
>> > > > > > > > All dependencies have been reviewed and checked for ASLv2
>> > > > > > compatibility.
>> > > > > > > >
>> > > > > > > > Will submit the infra ticket now for the dist/ path for keys
>> > > files
>> > > > > and
>> > > > > > > > such.
>> > > > > > > >
>> > > > > > > > My guess is that the release artifact should be a tarball of
>> > all
>> > > > > > source.
>> > > > > > > > Could we literally just package up a clean source tree?
>> Anyone
>> > > > else
>> > > > > > have
>> > > > > > > > views on this?
>> > > > > > > >
>> > > > > > >
>> > > > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag]
>> is a
>> > > > > > perfectly
>> > > > > > > reasonable thing to do.
>> > > > > > >
>> > > > > > > >
>> > > > > > > > Ideally with this release we'd do it all properly including
>> > maven
>> > > > > > > > artifacts, sources/javadocs, and so on.  The Maven build
>> does
>> > > > already
>> > > > > > > > operate now off a single command at the root to build
>> > everything
>> > > > > > > > (build-order is gone) and inherits from the apache parent.
>> > > > > > > >
>> > > > > > > > Will need to incorporate RAT.
>> > > > > > > >
>> > > > > > > > Thanks for all that - definitely gives some stuff to work on
>> > and
>> > > > look
>> > > > > > > into.
>> > > > > > > >
>> > > > > > > > Thanks
>> > > > > > > > Joe
>> > > > > > > >
>> > > > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <
>> > > [hidden email]>
>> > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Regardless of what you call it, writing down exactly what
>> was
>> > > > done
>> > > > > to
>> > > > > > > > make
>> > > > > > > > > a RC is extremely important early on (I know that I sure
>> > can't
>> > > > > > remember
>> > > > > > > > > what I did last week, much less last release). I'm not
>> sure
>> > if
>> > > > > > release
>> > > > > > > > > guide is formally defined somewhere, but enough
>> information
>> > > that
>> > > > > any
>> > > > > > > > other
>> > > > > > > > > committer can come by after "you get hit by a train" and
>> > make a
>> > > > > > release
>> > > > > > > > is
>> > > > > > > > > extremely important. Using the CMS and making a page on
>> the
>> > > > website
>> > > > > > for
>> > > > > > > > > this is extremely easy to do, IMO.
>> > > > > > > > >
>> > > > > > > > > Another concern for how to actually make a release is the
>> > type
>> > > of
>> > > > > > > > > packaging that is released. I not talking about the
>> > > source/binary
>> > > > > > > release
>> > > > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"? Is
>> > it a
>> > > > > > > tarball?
>> > > > > > > > > Are there jars in Maven? etc. A tarball is pretty easy to
>> > make,
>> > > > and
>> > > > > > > > likely
>> > > > > > > > > the closest to what the build-order.sh script already does
>> > > (with
>> > > > > > > Maven's
>> > > > > > > > > help). I'm not sure if maven-released artifacts are quite
>> as
>> > > > > > important
>> > > > > > > at
>> > > > > > > > > this point -- if it's not, punting on that will help.
>> If/when
>> > > you
>> > > > > get
>> > > > > > > to
>> > > > > > > > > that point, look into the apache parent pom[1]. It is
>> > extremely
>> > > > > well
>> > > > > > > > > automated to use the existing infrastructure to do it all
>> for
>> > > > you.
>> > > > > > > > >
>> > > > > > > > > Without looking at official documentation (which is me
>> being
>> > > > lazy,
>> > > > > > > > sorry),
>> > > > > > > > > some other things that will need to be thought about:
>> > > > > > > > >
>> > > > > > > > > Start with the releases page [2]
>> > > > > > > > >
>> > > > > > > > > * You *must* include a source artifact. It cannot have
>> > > binaries.
>> > > > No
>> > > > > > > Java
>> > > > > > > > > class files, no jars. A user must be able to download the
>> > > source
>> > > > > > > artifact
>> > > > > > > > > and build it all themselves. Artifacts which include
>> binaries
>> > > are
>> > > > > for
>> > > > > > > > > user-convenience only and can optionally be provided as
>> well.
>> > > > > > > > >
>> > > > > > > > > * LICENSE, NOTICE and README must all be included in the
>> top
>> > > > level
>> > > > > of
>> > > > > > > the
>> > > > > > > > > artifact. The NOTICE is the most painful and must include
>> any
>> > > > > > required
>> > > > > > > > > third-party notices. [3]
>> > > > > > > > >
>> > > > > > > > > * You will need to audit your dependencies to make sure
>> that
>> > > they
>> > > > > are
>> > > > > > > > > ASLv2 compatible [4]
>> > > > > > > > >
>> > > > > > > > > * Releases must be cryptographically signed (PGP) [5].
>> Your
>> > > > public
>> > > > > > key
>> > > > > > > > > should be published to known sites (e.g. pgp.mit.edu) and
>> > must
>> > > > be
>> > > > > > > listed
>> > > > > > > > > in NiFi's KEYS file [6] (which does not yet exist,
>> probably
>> > > needs
>> > > > > an
>> > > > > > > > infra
>> > > > > > > > > ticket to create your dist/ directory?).
>> > > > > > > > >
>> > > > > > > > > * Verify that every source file contain the proper ASL
>> > header.
>> > > > The
>> > > > > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a
>> wonderful
>> > > > tool
>> > > > > > that
>> > > > > > > > can
>> > > > > > > > > (and should, IMO) be integrated into your build process to
>> > > > prevent
>> > > > > > > people
>> > > > > > > > > from accidentally committing new files between releases
>> that
>> > do
>> > > > not
>> > > > > > > have
>> > > > > > > > > the correct header.
>> > > > > > > > >
>> > > > > > > > > And suddenly, this is really long :). Short answer: decide
>> > what
>> > > > the
>> > > > > > > > > release should look like (just a tarball?), start vetting
>> > your
>> > > > > source
>> > > > > > > > code
>> > > > > > > > > for license headers, and looking into NiFi's dependencies
>> and
>> > > > their
>> > > > > > > > > licenses.
>> > > > > > > > >
>> > > > > > > > > [1] http://maven.apache.org/pom/asf/
>> > > > > > > > > [2] http://www.apache.org/dev/release.html
>> > > > > > > > > [3] http://www.apache.org/legal/src-headers.html
>> > > > > > > > > [4] http://www.apache.org/legal/resolved.html#category-x
>> > > > > > > > > [5] http://www.apache.org/dev/release-signing.html
>> > > > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
>> > > > > > > > > [7] http://creadur.apache.org/rat/
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Tony Kurc wrote:
>> > > > > > > > >
>> > > > > > > > >> I read the guide Joe linked and a lot of the sticky parts
>> > are
>> > > > > marked
>> > > > > > > > >> "TODO"
>> > > > > > > > >> and it looks like a work in progress
>> > > > > > > > >>
>> > > > > > > > >> "Podlings can short circuit this process by starting out
>> > with
>> > > > > > written
>> > > > > > > > >> release documentation. It is strongly recommended that
>> > > Podlings
>> > > > > > invest
>> > > > > > > > >> time
>> > > > > > > > >> looking at the best practices recommended in this
>> document.
>> > By
>> > > > > > > selection
>> > > > > > > > >> and modification, sections of this document can be used
>> to
>> > > > quickly
>> > > > > > and
>> > > > > > > > >> easily bootstrap a release guide. "
>> > > > > > > > >>
>> > > > > > > > >> Is step one putting together a release guide?
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<
>> [hidden email]
>> > >
>> > > > > > wrote:
>> > > > > > > > >>
>> > > > > > > > >>  Hello
>> > > > > > > > >>>
>> > > > > > > > >>> Just wanted to stir this one up a bit.  Looks like all
>> > > tickets
>> > > > > > pegged
>> > > > > > > > as
>> > > > > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely
>> to
>> > be
>> > > > > > resolved
>> > > > > > > > >>> shortly based on their comments).  So working through
>> the
>> > > > release
>> > > > > > > steps
>> > > > > > > > >>> available on apache.org and via the link Brock sent.
>> > > > > > > > >>>
>> > > > > > > > >>> Anyone interested in this part of the process or who has
>> > > advice
>> > > > > to
>> > > > > > > help
>> > > > > > > > >>> us
>> > > > > > > > >>> avoid landmines we're happy to hear it.
>> > > > > > > > >>>
>> > > > > > > > >>> Thanks
>> > > > > > > > >>> Joe
>> > > > > > > > >>>
>> > > > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<
>> > [hidden email]
>> > > >
>> > > > > > > wrote:
>> > > > > > > > >>>
>> > > > > > > > >>>  Thanks Brock this is very helpful.
>> > > > > > > > >>>>
>> > > > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
>> > > > > [hidden email]>
>> > > > > > > > >>>>
>> > > > > > > > >>> wrote:
>> > > > > > > > >>>
>> > > > > > > > >>>> Hi,
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> This is a decent guide which can be copied:
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> Brock
>> > > > > > > > >>>>>
>> > > > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<
>> > > > [hidden email]>
>> > > > > > > > wrote:
>> > > > > > > > >>>>>
>> > > > > > > > >>>>>> Folks,
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>> Looking at the tickets that remain which are
>> presently
>> > > tied
>> > > > to
>> > > > > > > 0.0.1
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>> we're
>> > > > > > > > >>>>>
>> > > > > > > > >>>>>> probably 1-2 weeks out from this initial release.
>> Can
>> > you
>> > > > > > provide
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>> some
>> > > > > > > > >>>
>> > > > > > > > >>>> pointers/references or pointers on how to get this ball
>> > > > rolling
>> > > > > > and
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>> any
>> > > > > > > > >>>
>> > > > > > > > >>>> rocks we must move before going down this path?
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>>
>> > http://incubator.apache.org/guides/releasemanagement.html
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>> That link seems really helpful but has a lot of TODOs
>> > for
>> > > > > areas
>> > > > > > > > which
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>> need
>> > > > > > > > >>>>>
>> > > > > > > > >>>>>> more explanation.
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>> Thanks
>> > > > > > > > >>>>>> Joe
>> > > > > > > > >>>>>>
>> > > > > > > > >>>>>>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joe Witt
I agree we need to be mindful of this.  That said I am not actually sure
there is a problem.

To provide some specific data to consider here are the Nars and their sizes
today (smallest to largest):

29K
./standard-services/standard-services-api-nar/target/standard-services-api-nar-0.0.1-SNAPSHOT.nar
158K
./volatile-provenance-repository-bundle/nar/target/volatile-provenance-repository-nar-0.0.1-SNAPSHOT.nar
537K
./standard-services/ssl-context-bundle/nar/target/ssl-context-service-nar-0.0.1-SNAPSHOT.nar
628K
./standard-services/distributed-cache-services-bundle/distributed-cache-services-nar/target/distributed-cache-services-nar-0.0.1-SNAPSHOT.nar
1.1M ./hadoop-bundle/nar/target/hadoop-nar-0.0.1-SNAPSHOT.nar
4.3M
./monitor-threshold-bundle/nar/target/monitor-threshold-nar-0.0.1-SNAPSHOT.nar
4.4M
./update-attribute-bundle/nar/target/update-attribute-nar-0.0.1-SNAPSHOT.nar
4.5M ./jetty-bundle/target/nifi-jetty-bundle-0.0.1-SNAPSHOT.nar
4.6M
./persistent-provenance-repository-bundle/nar/target/persistent-provenance-repository-nar-0.0.1-SNAPSHOT.nar
12M ./kafka-bundle/kafka-nar/target/kafka-nar-0.0.1-SNAPSHOT.nar
12M ./standard-bundle/nar/target/nifi-standard-nar-0.0.1-SNAPSHOT.nar
26M
./hadoop-libraries-bundle/nar/target/hadoop-libraries-nar-0.0.1-SNAPSHOT.nar
28M ./framework-bundle/nar/target/nifi-framework-nar-0.0.1-SNAPSHOT.nar

Having the nars in a proper central repository would definitely make easier
for people to build their own assemblies of NiFi.  We definitely want all
the Jar artifacts there.

I am not convinced the added complexity (of modifying the build to
distinguish) is necessary.

A quick look through Maven central for things like Wars (which is a very
analogous concept to Nars) suggests what we'd be doing is not at all
uncommon and the sizes are reasonable as well.

I am of the view that we go as plain/keep it simple as we can here and if
it becomes an identifiable problem then we address it.

Thanks
Joe

On Thu, Jan 8, 2015 at 8:19 PM, Benson Margulies <[hidden email]>
wrote:

> To expand,
>
> On the one hand, people dump an amazing variety of crud onto Central.
> There's no special reason for NiFi to have more of a conscience than anyone
> else.
>
> On the other hand, if you want to use Maven to build things but not have
> them push to central, you control:
>
>     the install plugin
>     the deploy plugin
>
> and the 'attach' parameters of some other plugins, like the war plugin and
> the assembly plugin. Roughly, if you want to avoid treating the primary
> artifact of a project as a maven artifact, you have to mess with install
> and deploy.
>
> It would be good to sort this out before tackling any other release issues.
>
>
>
> On Thu, Jan 8, 2015 at 6:32 PM, Benson Margulies <[hidden email]>
> wrote:
>
> > If you don't want to release it you need to not deploy it. That is a pom
> > change unrelated to the staging repo.
> > On Jan 8, 2015 5:37 PM, "Adam Taft" <[hidden email]> wrote:
> >
> >> Benson,
> >>
> >> Quite a bit of NIFI's artifacts are more like application bundles of
> other
> >> common libraries.  Most of the NIFI nars aren't really useful for
> >> application developers to bind to in their own code.  These nars are
> more
> >> part of the application and less useful as a shared common library.
> >>
> >> One of my concerns was accidentally releasing unneeded nars into maven
> >> central.  It sounds like the "purgatory" area solves this?  Can you
> speak
> >> to this a little more?
> >>
> >> There's been discussion on whether we care that application nars escape
> >> into maven central - I'm not trying to weigh in on that topic (though, I
> >> think you can read my thoughts between the lines).  I'm just asking for
> >> more clarity on the options when using the maven release plugin.
> >> Purgatory
> >> might very well allow us to have our cake and eat it too.
> >>
> >> Thanks,
> >>
> >> Adam
> >>
> >>
> >> On Thu, Jan 8, 2015 at 5:07 PM, Benson Margulies <[hidden email]
> >
> >> wrote:
> >>
> >> > On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <[hidden email]> wrote:
> >> >
> >> > > Benson,
> >> > >
> >> > > Well I think we have all the groundwork laid as far as I know for
> the
> >> > > release plugin.  To be honest though (speaking for myself at least)
> >> I'm
> >> > not
> >> > > sure what the steps would be that the release plugin would do.  I'd
> >> love
> >> > it
> >> > > to be as you describe but am also concerned about how much we'd be
> >> > bugging
> >> > > you to figure that out.  I looked at what other projects seem to do
> >> and
> >> > it
> >> > > seemed shockingly manual.
> >> > >
> >> > > If you don't mind laying out the steps in more detail when you have
> >> time
> >> > > would love to learn from you on it.  I can also try toying around
> with
> >> > it a
> >> > > bit more off-line.
> >> > >
> >> >
> >> > I don't know what projects you're looking at, but it shouldn't look so
> >> > manual for any project that has a Maven top-level build.
> >> >
> >> > Here's the basic workflow of the maven-release-plugin:
> >> >
> >> > release:prepare:
> >> >
> >> > 1. edit poms to contain 'release version'.
> >> > 2. make a commit.
> >> > 3. make a tag.
> >> > 4. edit poms to contain 'next snapshot'
> >> > 5. make a commit.
> >> > 6. push the whole business (unless you turn that off).
> >> >
> >> > release:perform:
> >> >
> >> > 7. check out from tag in target/checkout (using extra git clone)
> >> > 8. build
> >> > 9. deploy results to target of deploymentManagement
> >> >
> >> > Now comes the next trick.
> >> >
> >> > Apache runs a copy of Sonatype Nexus, that serves as a 'staging
> >> > repository'.  So, when step 9 pushes things to the repository, it's
> >> pushing
> >> > them to a special 'purgatory' staging area. That area is available for
> >> > people to look at for testing, but does not propagate along to Maven
> >> > central.
> >> >
> >> > Given all this, then we add one ingredient: an additional maven module
> >> that
> >> > uses the maven-assembly-plugin to build a tarball that satisfies the
> >> > requirements of an Apache source release. This is not the same thing
> as
> >> > what the maven-source-plugin does at all. Since it will have
> >> > <attach>true</attach>, the resulting tarball swims upstream to the
> >> staging
> >> > repo with everything else.
> >> >
> >> > The vote email points people to that location, and supplies a sha1 for
> >> the
> >> > source ball. Voters download the source release from the staging repo,
> >> do
> >> > what they need to do, and vote. Three +1's later, and then you get to
> >> > publish.
> >> >
> >> > Does this help? The only manual steps are really copying to svn to
> push
> >> to
> >> > the Apache dist area. CXF and all the Maven plugins are examples.
> >> >
> >> >
> >> >
> >> >
> >> > >
> >> > > Thanks
> >> > > Joe
> >> > >
> >> > > On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <
> >> [hidden email]>
> >> > > wrote:
> >> > >
> >> > > > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]>
> >> wrote:
> >> > > >
> >> > > > > Benson,
> >> > > > >
> >> > > > > Yes that would be much appreciated.
> >> > > > >
> >> > > > > Here is a rough gamplan of what I *think* will be needed to do
> the
> >> > > > release:
> >> > > > >
> >> > > > > - Branch from develop against a ticket for the overall release
> >> > process
> >> > > > > - Update the version of all the artifacts and dependencies to
> >> > > > > 0.0.1-RC1-incubating or something like that
> >> > > > > - Create the tar.gz of the clean source tree.  Sign that and
> place
> >> > both
> >> > > > the
> >> > > > > signed file and the tar.gz into my people's dir (if i have one
> of
> >> > > those)
> >> > > > > - Notify folks that this is available so they can pull it and
> >> attempt
> >> > > to
> >> > > > > build from that and validate the signature
> >> > > > > - Once folks agree it is good to merge that to master
> >> > > > > - Create a tar.gz of that and a signed file.  Call a vote within
> >> the
> >> > > > team.
> >> > > > > If that is good call a vote within IPMC.
> >> > > > > - IF NO - fix whatever is wrong.
> >> > > > > - If all good then do a maven deploy of the binary artifacts,
> >> source
> >> > > > > bundles, and javadocs
> >> > > > > - Upload the tar.gz of source and the signed file to our dist
> dir.
> >> > And
> >> > > > > upload convenience binary build tar.gz and zip with sigs to our
> >> dist
> >> > > > > folder.  Then create links to these from our downloads page.
> >> > > > > - Update JIRA that the release is closed.
> >> > > > > - Wash/Rinse/Repeat
> >> > > > >
> >> > > > > <note this all is with the understanding that we've done all the
> >> > > > necessary
> >> > > > > review of dependencies, licenses, properly documented them, have
> >> our
> >> > > > > ECCN/encryption stuff properly filed>
> >> > > > >
> >> > > >
> >> > > > Sure, except that I think we can make the maven-release-plugin do
> a
> >> lot
> >> > > of
> >> > > > the work.
> >> > > >
> >> > > > The idea is that you run the release plugin, and it delivers all
> >> these
> >> > > > goodies to repository.apache.org. Then you call the vote. If the
> >> vote
> >> > > > passes, you (a) push the promote button to push to Maven Central,
> >> and
> >> > (b)
> >> > > > check the official source release tarball into the svn area for
> >> > > > distribution.
> >> > > >
> >> > > > I will take a whack at a PR for point (a).
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > >
> >> > > > > Might be missing a detail or two but does that sound roughly on
> >> the
> >> > > right
> >> > > > > rack?
> >> > > > >
> >> > > > > I had though the maven release plugin would be useful but looks
> >> like
> >> > > > we'll
> >> > > > > just be able to use the versions plugin to update them and can
> do
> >> the
> >> > > > other
> >> > > > > simple steps manually.
> >> > > > >
> >> > > > > Thanks
> >> > > > > Joe
> >> > > > >
> >> > > > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <
> >> > > [hidden email]>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Typically people set the maven assembly plugin for this and
> >> include
> >> > > it
> >> > > > in
> >> > > > > > the build. Would you like me to pitch this in?
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <
> [hidden email]>
> >> > > wrote:
> >> > > > > >
> >> > > > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <
> [hidden email]>
> >> > > wrote:
> >> > > > > > >
> >> > > > > > > > Josh
> >> > > > > > > >
> >> > > > > > > > Awesome.
> >> > > > > > > >
> >> > > > > > > > Honestly I believe we're good on the LICENSE, NOTICE,
> >> > DISCLAIMER,
> >> > > > > > README.
> >> > > > > > > > All dependencies have been reviewed and checked for ASLv2
> >> > > > > > compatibility.
> >> > > > > > > >
> >> > > > > > > > Will submit the infra ticket now for the dist/ path for
> keys
> >> > > files
> >> > > > > and
> >> > > > > > > > such.
> >> > > > > > > >
> >> > > > > > > > My guess is that the release artifact should be a tarball
> of
> >> > all
> >> > > > > > source.
> >> > > > > > > > Could we literally just package up a clean source tree?
> >> Anyone
> >> > > > else
> >> > > > > > have
> >> > > > > > > > views on this?
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag]
> >> is a
> >> > > > > > perfectly
> >> > > > > > > reasonable thing to do.
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > Ideally with this release we'd do it all properly
> including
> >> > maven
> >> > > > > > > > artifacts, sources/javadocs, and so on.  The Maven build
> >> does
> >> > > > already
> >> > > > > > > > operate now off a single command at the root to build
> >> > everything
> >> > > > > > > > (build-order is gone) and inherits from the apache parent.
> >> > > > > > > >
> >> > > > > > > > Will need to incorporate RAT.
> >> > > > > > > >
> >> > > > > > > > Thanks for all that - definitely gives some stuff to work
> on
> >> > and
> >> > > > look
> >> > > > > > > into.
> >> > > > > > > >
> >> > > > > > > > Thanks
> >> > > > > > > > Joe
> >> > > > > > > >
> >> > > > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <
> >> > > [hidden email]>
> >> > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Regardless of what you call it, writing down exactly
> what
> >> was
> >> > > > done
> >> > > > > to
> >> > > > > > > > make
> >> > > > > > > > > a RC is extremely important early on (I know that I sure
> >> > can't
> >> > > > > > remember
> >> > > > > > > > > what I did last week, much less last release). I'm not
> >> sure
> >> > if
> >> > > > > > release
> >> > > > > > > > > guide is formally defined somewhere, but enough
> >> information
> >> > > that
> >> > > > > any
> >> > > > > > > > other
> >> > > > > > > > > committer can come by after "you get hit by a train" and
> >> > make a
> >> > > > > > release
> >> > > > > > > > is
> >> > > > > > > > > extremely important. Using the CMS and making a page on
> >> the
> >> > > > website
> >> > > > > > for
> >> > > > > > > > > this is extremely easy to do, IMO.
> >> > > > > > > > >
> >> > > > > > > > > Another concern for how to actually make a release is
> the
> >> > type
> >> > > of
> >> > > > > > > > > packaging that is released. I not talking about the
> >> > > source/binary
> >> > > > > > > release
> >> > > > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"?
> Is
> >> > it a
> >> > > > > > > tarball?
> >> > > > > > > > > Are there jars in Maven? etc. A tarball is pretty easy
> to
> >> > make,
> >> > > > and
> >> > > > > > > > likely
> >> > > > > > > > > the closest to what the build-order.sh script already
> does
> >> > > (with
> >> > > > > > > Maven's
> >> > > > > > > > > help). I'm not sure if maven-released artifacts are
> quite
> >> as
> >> > > > > > important
> >> > > > > > > at
> >> > > > > > > > > this point -- if it's not, punting on that will help.
> >> If/when
> >> > > you
> >> > > > > get
> >> > > > > > > to
> >> > > > > > > > > that point, look into the apache parent pom[1]. It is
> >> > extremely
> >> > > > > well
> >> > > > > > > > > automated to use the existing infrastructure to do it
> all
> >> for
> >> > > > you.
> >> > > > > > > > >
> >> > > > > > > > > Without looking at official documentation (which is me
> >> being
> >> > > > lazy,
> >> > > > > > > > sorry),
> >> > > > > > > > > some other things that will need to be thought about:
> >> > > > > > > > >
> >> > > > > > > > > Start with the releases page [2]
> >> > > > > > > > >
> >> > > > > > > > > * You *must* include a source artifact. It cannot have
> >> > > binaries.
> >> > > > No
> >> > > > > > > Java
> >> > > > > > > > > class files, no jars. A user must be able to download
> the
> >> > > source
> >> > > > > > > artifact
> >> > > > > > > > > and build it all themselves. Artifacts which include
> >> binaries
> >> > > are
> >> > > > > for
> >> > > > > > > > > user-convenience only and can optionally be provided as
> >> well.
> >> > > > > > > > >
> >> > > > > > > > > * LICENSE, NOTICE and README must all be included in the
> >> top
> >> > > > level
> >> > > > > of
> >> > > > > > > the
> >> > > > > > > > > artifact. The NOTICE is the most painful and must
> include
> >> any
> >> > > > > > required
> >> > > > > > > > > third-party notices. [3]
> >> > > > > > > > >
> >> > > > > > > > > * You will need to audit your dependencies to make sure
> >> that
> >> > > they
> >> > > > > are
> >> > > > > > > > > ASLv2 compatible [4]
> >> > > > > > > > >
> >> > > > > > > > > * Releases must be cryptographically signed (PGP) [5].
> >> Your
> >> > > > public
> >> > > > > > key
> >> > > > > > > > > should be published to known sites (e.g. pgp.mit.edu)
> and
> >> > must
> >> > > > be
> >> > > > > > > listed
> >> > > > > > > > > in NiFi's KEYS file [6] (which does not yet exist,
> >> probably
> >> > > needs
> >> > > > > an
> >> > > > > > > > infra
> >> > > > > > > > > ticket to create your dist/ directory?).
> >> > > > > > > > >
> >> > > > > > > > > * Verify that every source file contain the proper ASL
> >> > header.
> >> > > > The
> >> > > > > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a
> >> wonderful
> >> > > > tool
> >> > > > > > that
> >> > > > > > > > can
> >> > > > > > > > > (and should, IMO) be integrated into your build process
> to
> >> > > > prevent
> >> > > > > > > people
> >> > > > > > > > > from accidentally committing new files between releases
> >> that
> >> > do
> >> > > > not
> >> > > > > > > have
> >> > > > > > > > > the correct header.
> >> > > > > > > > >
> >> > > > > > > > > And suddenly, this is really long :). Short answer:
> decide
> >> > what
> >> > > > the
> >> > > > > > > > > release should look like (just a tarball?), start
> vetting
> >> > your
> >> > > > > source
> >> > > > > > > > code
> >> > > > > > > > > for license headers, and looking into NiFi's
> dependencies
> >> and
> >> > > > their
> >> > > > > > > > > licenses.
> >> > > > > > > > >
> >> > > > > > > > > [1] http://maven.apache.org/pom/asf/
> >> > > > > > > > > [2] http://www.apache.org/dev/release.html
> >> > > > > > > > > [3] http://www.apache.org/legal/src-headers.html
> >> > > > > > > > > [4]
> http://www.apache.org/legal/resolved.html#category-x
> >> > > > > > > > > [5] http://www.apache.org/dev/release-signing.html
> >> > > > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
> >> > > > > > > > > [7] http://creadur.apache.org/rat/
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Tony Kurc wrote:
> >> > > > > > > > >
> >> > > > > > > > >> I read the guide Joe linked and a lot of the sticky
> parts
> >> > are
> >> > > > > marked
> >> > > > > > > > >> "TODO"
> >> > > > > > > > >> and it looks like a work in progress
> >> > > > > > > > >>
> >> > > > > > > > >> "Podlings can short circuit this process by starting
> out
> >> > with
> >> > > > > > written
> >> > > > > > > > >> release documentation. It is strongly recommended that
> >> > > Podlings
> >> > > > > > invest
> >> > > > > > > > >> time
> >> > > > > > > > >> looking at the best practices recommended in this
> >> document.
> >> > By
> >> > > > > > > selection
> >> > > > > > > > >> and modification, sections of this document can be used
> >> to
> >> > > > quickly
> >> > > > > > and
> >> > > > > > > > >> easily bootstrap a release guide. "
> >> > > > > > > > >>
> >> > > > > > > > >> Is step one putting together a release guide?
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<
> >> [hidden email]
> >> > >
> >> > > > > > wrote:
> >> > > > > > > > >>
> >> > > > > > > > >>  Hello
> >> > > > > > > > >>>
> >> > > > > > > > >>> Just wanted to stir this one up a bit.  Looks like all
> >> > > tickets
> >> > > > > > pegged
> >> > > > > > > > as
> >> > > > > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely
> >> to
> >> > be
> >> > > > > > resolved
> >> > > > > > > > >>> shortly based on their comments).  So working through
> >> the
> >> > > > release
> >> > > > > > > steps
> >> > > > > > > > >>> available on apache.org and via the link Brock sent.
> >> > > > > > > > >>>
> >> > > > > > > > >>> Anyone interested in this part of the process or who
> has
> >> > > advice
> >> > > > > to
> >> > > > > > > help
> >> > > > > > > > >>> us
> >> > > > > > > > >>> avoid landmines we're happy to hear it.
> >> > > > > > > > >>>
> >> > > > > > > > >>> Thanks
> >> > > > > > > > >>> Joe
> >> > > > > > > > >>>
> >> > > > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<
> >> > [hidden email]
> >> > > >
> >> > > > > > > wrote:
> >> > > > > > > > >>>
> >> > > > > > > > >>>  Thanks Brock this is very helpful.
> >> > > > > > > > >>>>
> >> > > > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
> >> > > > > [hidden email]>
> >> > > > > > > > >>>>
> >> > > > > > > > >>> wrote:
> >> > > > > > > > >>>
> >> > > > > > > > >>>> Hi,
> >> > > > > > > > >>>>>
> >> > > > > > > > >>>>> This is a decent guide which can be copied:
> >> > > > > > > > >>>>>
> >> > > > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
> >> > > > > > > > >>>>>
> >> > > > > > > > >>>>> Brock
> >> > > > > > > > >>>>>
> >> > > > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<
> >> > > > [hidden email]>
> >> > > > > > > > wrote:
> >> > > > > > > > >>>>>
> >> > > > > > > > >>>>>> Folks,
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>>> Looking at the tickets that remain which are
> >> presently
> >> > > tied
> >> > > > to
> >> > > > > > > 0.0.1
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>> we're
> >> > > > > > > > >>>>>
> >> > > > > > > > >>>>>> probably 1-2 weeks out from this initial release.
> >> Can
> >> > you
> >> > > > > > provide
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>> some
> >> > > > > > > > >>>
> >> > > > > > > > >>>> pointers/references or pointers on how to get this
> ball
> >> > > > rolling
> >> > > > > > and
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>> any
> >> > > > > > > > >>>
> >> > > > > > > > >>>> rocks we must move before going down this path?
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>>>
> >> > http://incubator.apache.org/guides/releasemanagement.html
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>>> That link seems really helpful but has a lot of
> TODOs
> >> > for
> >> > > > > areas
> >> > > > > > > > which
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>> need
> >> > > > > > > > >>>>>
> >> > > > > > > > >>>>>> more explanation.
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>>> Thanks
> >> > > > > > > > >>>>>> Joe
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>>>>>
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: [Discuss] preparing for release 0.0.1

Joey Echeverria
+1 to deploying NARs to Maven. If it becomes a problem in the future,
we could revisit but keeping the maven configuration as simple as
possible is a huge plus.

-Joey

On Thu, Jan 8, 2015 at 8:07 PM, Joe Witt <[hidden email]> wrote:

> I agree we need to be mindful of this.  That said I am not actually sure
> there is a problem.
>
> To provide some specific data to consider here are the Nars and their sizes
> today (smallest to largest):
>
> 29K
> ./standard-services/standard-services-api-nar/target/standard-services-api-nar-0.0.1-SNAPSHOT.nar
> 158K
> ./volatile-provenance-repository-bundle/nar/target/volatile-provenance-repository-nar-0.0.1-SNAPSHOT.nar
> 537K
> ./standard-services/ssl-context-bundle/nar/target/ssl-context-service-nar-0.0.1-SNAPSHOT.nar
> 628K
> ./standard-services/distributed-cache-services-bundle/distributed-cache-services-nar/target/distributed-cache-services-nar-0.0.1-SNAPSHOT.nar
> 1.1M ./hadoop-bundle/nar/target/hadoop-nar-0.0.1-SNAPSHOT.nar
> 4.3M
> ./monitor-threshold-bundle/nar/target/monitor-threshold-nar-0.0.1-SNAPSHOT.nar
> 4.4M
> ./update-attribute-bundle/nar/target/update-attribute-nar-0.0.1-SNAPSHOT.nar
> 4.5M ./jetty-bundle/target/nifi-jetty-bundle-0.0.1-SNAPSHOT.nar
> 4.6M
> ./persistent-provenance-repository-bundle/nar/target/persistent-provenance-repository-nar-0.0.1-SNAPSHOT.nar
> 12M ./kafka-bundle/kafka-nar/target/kafka-nar-0.0.1-SNAPSHOT.nar
> 12M ./standard-bundle/nar/target/nifi-standard-nar-0.0.1-SNAPSHOT.nar
> 26M
> ./hadoop-libraries-bundle/nar/target/hadoop-libraries-nar-0.0.1-SNAPSHOT.nar
> 28M ./framework-bundle/nar/target/nifi-framework-nar-0.0.1-SNAPSHOT.nar
>
> Having the nars in a proper central repository would definitely make easier
> for people to build their own assemblies of NiFi.  We definitely want all
> the Jar artifacts there.
>
> I am not convinced the added complexity (of modifying the build to
> distinguish) is necessary.
>
> A quick look through Maven central for things like Wars (which is a very
> analogous concept to Nars) suggests what we'd be doing is not at all
> uncommon and the sizes are reasonable as well.
>
> I am of the view that we go as plain/keep it simple as we can here and if
> it becomes an identifiable problem then we address it.
>
> Thanks
> Joe
>
> On Thu, Jan 8, 2015 at 8:19 PM, Benson Margulies <[hidden email]>
> wrote:
>
>> To expand,
>>
>> On the one hand, people dump an amazing variety of crud onto Central.
>> There's no special reason for NiFi to have more of a conscience than anyone
>> else.
>>
>> On the other hand, if you want to use Maven to build things but not have
>> them push to central, you control:
>>
>>     the install plugin
>>     the deploy plugin
>>
>> and the 'attach' parameters of some other plugins, like the war plugin and
>> the assembly plugin. Roughly, if you want to avoid treating the primary
>> artifact of a project as a maven artifact, you have to mess with install
>> and deploy.
>>
>> It would be good to sort this out before tackling any other release issues.
>>
>>
>>
>> On Thu, Jan 8, 2015 at 6:32 PM, Benson Margulies <[hidden email]>
>> wrote:
>>
>> > If you don't want to release it you need to not deploy it. That is a pom
>> > change unrelated to the staging repo.
>> > On Jan 8, 2015 5:37 PM, "Adam Taft" <[hidden email]> wrote:
>> >
>> >> Benson,
>> >>
>> >> Quite a bit of NIFI's artifacts are more like application bundles of
>> other
>> >> common libraries.  Most of the NIFI nars aren't really useful for
>> >> application developers to bind to in their own code.  These nars are
>> more
>> >> part of the application and less useful as a shared common library.
>> >>
>> >> One of my concerns was accidentally releasing unneeded nars into maven
>> >> central.  It sounds like the "purgatory" area solves this?  Can you
>> speak
>> >> to this a little more?
>> >>
>> >> There's been discussion on whether we care that application nars escape
>> >> into maven central - I'm not trying to weigh in on that topic (though, I
>> >> think you can read my thoughts between the lines).  I'm just asking for
>> >> more clarity on the options when using the maven release plugin.
>> >> Purgatory
>> >> might very well allow us to have our cake and eat it too.
>> >>
>> >> Thanks,
>> >>
>> >> Adam
>> >>
>> >>
>> >> On Thu, Jan 8, 2015 at 5:07 PM, Benson Margulies <[hidden email]
>> >
>> >> wrote:
>> >>
>> >> > On Thu, Jan 8, 2015 at 4:53 PM, Joe Witt <[hidden email]> wrote:
>> >> >
>> >> > > Benson,
>> >> > >
>> >> > > Well I think we have all the groundwork laid as far as I know for
>> the
>> >> > > release plugin.  To be honest though (speaking for myself at least)
>> >> I'm
>> >> > not
>> >> > > sure what the steps would be that the release plugin would do.  I'd
>> >> love
>> >> > it
>> >> > > to be as you describe but am also concerned about how much we'd be
>> >> > bugging
>> >> > > you to figure that out.  I looked at what other projects seem to do
>> >> and
>> >> > it
>> >> > > seemed shockingly manual.
>> >> > >
>> >> > > If you don't mind laying out the steps in more detail when you have
>> >> time
>> >> > > would love to learn from you on it.  I can also try toying around
>> with
>> >> > it a
>> >> > > bit more off-line.
>> >> > >
>> >> >
>> >> > I don't know what projects you're looking at, but it shouldn't look so
>> >> > manual for any project that has a Maven top-level build.
>> >> >
>> >> > Here's the basic workflow of the maven-release-plugin:
>> >> >
>> >> > release:prepare:
>> >> >
>> >> > 1. edit poms to contain 'release version'.
>> >> > 2. make a commit.
>> >> > 3. make a tag.
>> >> > 4. edit poms to contain 'next snapshot'
>> >> > 5. make a commit.
>> >> > 6. push the whole business (unless you turn that off).
>> >> >
>> >> > release:perform:
>> >> >
>> >> > 7. check out from tag in target/checkout (using extra git clone)
>> >> > 8. build
>> >> > 9. deploy results to target of deploymentManagement
>> >> >
>> >> > Now comes the next trick.
>> >> >
>> >> > Apache runs a copy of Sonatype Nexus, that serves as a 'staging
>> >> > repository'.  So, when step 9 pushes things to the repository, it's
>> >> pushing
>> >> > them to a special 'purgatory' staging area. That area is available for
>> >> > people to look at for testing, but does not propagate along to Maven
>> >> > central.
>> >> >
>> >> > Given all this, then we add one ingredient: an additional maven module
>> >> that
>> >> > uses the maven-assembly-plugin to build a tarball that satisfies the
>> >> > requirements of an Apache source release. This is not the same thing
>> as
>> >> > what the maven-source-plugin does at all. Since it will have
>> >> > <attach>true</attach>, the resulting tarball swims upstream to the
>> >> staging
>> >> > repo with everything else.
>> >> >
>> >> > The vote email points people to that location, and supplies a sha1 for
>> >> the
>> >> > source ball. Voters download the source release from the staging repo,
>> >> do
>> >> > what they need to do, and vote. Three +1's later, and then you get to
>> >> > publish.
>> >> >
>> >> > Does this help? The only manual steps are really copying to svn to
>> push
>> >> to
>> >> > the Apache dist area. CXF and all the Maven plugins are examples.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > >
>> >> > > Thanks
>> >> > > Joe
>> >> > >
>> >> > > On Thu, Jan 8, 2015 at 4:49 PM, Benson Margulies <
>> >> [hidden email]>
>> >> > > wrote:
>> >> > >
>> >> > > > On Thu, Jan 8, 2015 at 4:41 PM, Joe Witt <[hidden email]>
>> >> wrote:
>> >> > > >
>> >> > > > > Benson,
>> >> > > > >
>> >> > > > > Yes that would be much appreciated.
>> >> > > > >
>> >> > > > > Here is a rough gamplan of what I *think* will be needed to do
>> the
>> >> > > > release:
>> >> > > > >
>> >> > > > > - Branch from develop against a ticket for the overall release
>> >> > process
>> >> > > > > - Update the version of all the artifacts and dependencies to
>> >> > > > > 0.0.1-RC1-incubating or something like that
>> >> > > > > - Create the tar.gz of the clean source tree.  Sign that and
>> place
>> >> > both
>> >> > > > the
>> >> > > > > signed file and the tar.gz into my people's dir (if i have one
>> of
>> >> > > those)
>> >> > > > > - Notify folks that this is available so they can pull it and
>> >> attempt
>> >> > > to
>> >> > > > > build from that and validate the signature
>> >> > > > > - Once folks agree it is good to merge that to master
>> >> > > > > - Create a tar.gz of that and a signed file.  Call a vote within
>> >> the
>> >> > > > team.
>> >> > > > > If that is good call a vote within IPMC.
>> >> > > > > - IF NO - fix whatever is wrong.
>> >> > > > > - If all good then do a maven deploy of the binary artifacts,
>> >> source
>> >> > > > > bundles, and javadocs
>> >> > > > > - Upload the tar.gz of source and the signed file to our dist
>> dir.
>> >> > And
>> >> > > > > upload convenience binary build tar.gz and zip with sigs to our
>> >> dist
>> >> > > > > folder.  Then create links to these from our downloads page.
>> >> > > > > - Update JIRA that the release is closed.
>> >> > > > > - Wash/Rinse/Repeat
>> >> > > > >
>> >> > > > > <note this all is with the understanding that we've done all the
>> >> > > > necessary
>> >> > > > > review of dependencies, licenses, properly documented them, have
>> >> our
>> >> > > > > ECCN/encryption stuff properly filed>
>> >> > > > >
>> >> > > >
>> >> > > > Sure, except that I think we can make the maven-release-plugin do
>> a
>> >> lot
>> >> > > of
>> >> > > > the work.
>> >> > > >
>> >> > > > The idea is that you run the release plugin, and it delivers all
>> >> these
>> >> > > > goodies to repository.apache.org. Then you call the vote. If the
>> >> vote
>> >> > > > passes, you (a) push the promote button to push to Maven Central,
>> >> and
>> >> > (b)
>> >> > > > check the official source release tarball into the svn area for
>> >> > > > distribution.
>> >> > > >
>> >> > > > I will take a whack at a PR for point (a).
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > >
>> >> > > > > Might be missing a detail or two but does that sound roughly on
>> >> the
>> >> > > right
>> >> > > > > rack?
>> >> > > > >
>> >> > > > > I had though the maven release plugin would be useful but looks
>> >> like
>> >> > > > we'll
>> >> > > > > just be able to use the versions plugin to update them and can
>> do
>> >> the
>> >> > > > other
>> >> > > > > simple steps manually.
>> >> > > > >
>> >> > > > > Thanks
>> >> > > > > Joe
>> >> > > > >
>> >> > > > > On Thu, Jan 8, 2015 at 3:46 PM, Benson Margulies <
>> >> > > [hidden email]>
>> >> > > > > wrote:
>> >> > > > >
>> >> > > > > > Typically people set the maven assembly plugin for this and
>> >> include
>> >> > > it
>> >> > > > in
>> >> > > > > > the build. Would you like me to pitch this in?
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > On Thu, Jan 8, 2015 at 2:48 PM, Mike Drob <
>> [hidden email]>
>> >> > > wrote:
>> >> > > > > >
>> >> > > > > > > On Wed, Jan 7, 2015 at 9:22 PM, Joe Witt <
>> [hidden email]>
>> >> > > wrote:
>> >> > > > > > >
>> >> > > > > > > > Josh
>> >> > > > > > > >
>> >> > > > > > > > Awesome.
>> >> > > > > > > >
>> >> > > > > > > > Honestly I believe we're good on the LICENSE, NOTICE,
>> >> > DISCLAIMER,
>> >> > > > > > README.
>> >> > > > > > > > All dependencies have been reviewed and checked for ASLv2
>> >> > > > > > compatibility.
>> >> > > > > > > >
>> >> > > > > > > > Will submit the infra ticket now for the dist/ path for
>> keys
>> >> > > files
>> >> > > > > and
>> >> > > > > > > > such.
>> >> > > > > > > >
>> >> > > > > > > > My guess is that the release artifact should be a tarball
>> of
>> >> > all
>> >> > > > > > source.
>> >> > > > > > > > Could we literally just package up a clean source tree?
>> >> Anyone
>> >> > > > else
>> >> > > > > > have
>> >> > > > > > > > views on this?
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > > > git archive -o [nifi-0.0.1-incubating.tar.gz] [release-tag]
>> >> is a
>> >> > > > > > perfectly
>> >> > > > > > > reasonable thing to do.
>> >> > > > > > >
>> >> > > > > > > >
>> >> > > > > > > > Ideally with this release we'd do it all properly
>> including
>> >> > maven
>> >> > > > > > > > artifacts, sources/javadocs, and so on.  The Maven build
>> >> does
>> >> > > > already
>> >> > > > > > > > operate now off a single command at the root to build
>> >> > everything
>> >> > > > > > > > (build-order is gone) and inherits from the apache parent.
>> >> > > > > > > >
>> >> > > > > > > > Will need to incorporate RAT.
>> >> > > > > > > >
>> >> > > > > > > > Thanks for all that - definitely gives some stuff to work
>> on
>> >> > and
>> >> > > > look
>> >> > > > > > > into.
>> >> > > > > > > >
>> >> > > > > > > > Thanks
>> >> > > > > > > > Joe
>> >> > > > > > > >
>> >> > > > > > > > On Wed, Jan 7, 2015 at 10:10 PM, Josh Elser <
>> >> > > [hidden email]>
>> >> > > > > > > wrote:
>> >> > > > > > > >
>> >> > > > > > > > > Regardless of what you call it, writing down exactly
>> what
>> >> was
>> >> > > > done
>> >> > > > > to
>> >> > > > > > > > make
>> >> > > > > > > > > a RC is extremely important early on (I know that I sure
>> >> > can't
>> >> > > > > > remember
>> >> > > > > > > > > what I did last week, much less last release). I'm not
>> >> sure
>> >> > if
>> >> > > > > > release
>> >> > > > > > > > > guide is formally defined somewhere, but enough
>> >> information
>> >> > > that
>> >> > > > > any
>> >> > > > > > > > other
>> >> > > > > > > > > committer can come by after "you get hit by a train" and
>> >> > make a
>> >> > > > > > release
>> >> > > > > > > > is
>> >> > > > > > > > > extremely important. Using the CMS and making a page on
>> >> the
>> >> > > > website
>> >> > > > > > for
>> >> > > > > > > > > this is extremely easy to do, IMO.
>> >> > > > > > > > >
>> >> > > > > > > > > Another concern for how to actually make a release is
>> the
>> >> > type
>> >> > > of
>> >> > > > > > > > > packaging that is released. I not talking about the
>> >> > > source/binary
>> >> > > > > > > release
>> >> > > > > > > > > here, literally "what is Apache NiFi 0.0.1-incubating"?
>> Is
>> >> > it a
>> >> > > > > > > tarball?
>> >> > > > > > > > > Are there jars in Maven? etc. A tarball is pretty easy
>> to
>> >> > make,
>> >> > > > and
>> >> > > > > > > > likely
>> >> > > > > > > > > the closest to what the build-order.sh script already
>> does
>> >> > > (with
>> >> > > > > > > Maven's
>> >> > > > > > > > > help). I'm not sure if maven-released artifacts are
>> quite
>> >> as
>> >> > > > > > important
>> >> > > > > > > at
>> >> > > > > > > > > this point -- if it's not, punting on that will help.
>> >> If/when
>> >> > > you
>> >> > > > > get
>> >> > > > > > > to
>> >> > > > > > > > > that point, look into the apache parent pom[1]. It is
>> >> > extremely
>> >> > > > > well
>> >> > > > > > > > > automated to use the existing infrastructure to do it
>> all
>> >> for
>> >> > > > you.
>> >> > > > > > > > >
>> >> > > > > > > > > Without looking at official documentation (which is me
>> >> being
>> >> > > > lazy,
>> >> > > > > > > > sorry),
>> >> > > > > > > > > some other things that will need to be thought about:
>> >> > > > > > > > >
>> >> > > > > > > > > Start with the releases page [2]
>> >> > > > > > > > >
>> >> > > > > > > > > * You *must* include a source artifact. It cannot have
>> >> > > binaries.
>> >> > > > No
>> >> > > > > > > Java
>> >> > > > > > > > > class files, no jars. A user must be able to download
>> the
>> >> > > source
>> >> > > > > > > artifact
>> >> > > > > > > > > and build it all themselves. Artifacts which include
>> >> binaries
>> >> > > are
>> >> > > > > for
>> >> > > > > > > > > user-convenience only and can optionally be provided as
>> >> well.
>> >> > > > > > > > >
>> >> > > > > > > > > * LICENSE, NOTICE and README must all be included in the
>> >> top
>> >> > > > level
>> >> > > > > of
>> >> > > > > > > the
>> >> > > > > > > > > artifact. The NOTICE is the most painful and must
>> include
>> >> any
>> >> > > > > > required
>> >> > > > > > > > > third-party notices. [3]
>> >> > > > > > > > >
>> >> > > > > > > > > * You will need to audit your dependencies to make sure
>> >> that
>> >> > > they
>> >> > > > > are
>> >> > > > > > > > > ASLv2 compatible [4]
>> >> > > > > > > > >
>> >> > > > > > > > > * Releases must be cryptographically signed (PGP) [5].
>> >> Your
>> >> > > > public
>> >> > > > > > key
>> >> > > > > > > > > should be published to known sites (e.g. pgp.mit.edu)
>> and
>> >> > must
>> >> > > > be
>> >> > > > > > > listed
>> >> > > > > > > > > in NiFi's KEYS file [6] (which does not yet exist,
>> >> probably
>> >> > > needs
>> >> > > > > an
>> >> > > > > > > > infra
>> >> > > > > > > > > ticket to create your dist/ directory?).
>> >> > > > > > > > >
>> >> > > > > > > > > * Verify that every source file contain the proper ASL
>> >> > header.
>> >> > > > The
>> >> > > > > > > > > release-audit-tool[7] (`mvn apache-rat:check`) is a
>> >> wonderful
>> >> > > > tool
>> >> > > > > > that
>> >> > > > > > > > can
>> >> > > > > > > > > (and should, IMO) be integrated into your build process
>> to
>> >> > > > prevent
>> >> > > > > > > people
>> >> > > > > > > > > from accidentally committing new files between releases
>> >> that
>> >> > do
>> >> > > > not
>> >> > > > > > > have
>> >> > > > > > > > > the correct header.
>> >> > > > > > > > >
>> >> > > > > > > > > And suddenly, this is really long :). Short answer:
>> decide
>> >> > what
>> >> > > > the
>> >> > > > > > > > > release should look like (just a tarball?), start
>> vetting
>> >> > your
>> >> > > > > source
>> >> > > > > > > > code
>> >> > > > > > > > > for license headers, and looking into NiFi's
>> dependencies
>> >> and
>> >> > > > their
>> >> > > > > > > > > licenses.
>> >> > > > > > > > >
>> >> > > > > > > > > [1] http://maven.apache.org/pom/asf/
>> >> > > > > > > > > [2] http://www.apache.org/dev/release.html
>> >> > > > > > > > > [3] http://www.apache.org/legal/src-headers.html
>> >> > > > > > > > > [4]
>> http://www.apache.org/legal/resolved.html#category-x
>> >> > > > > > > > > [5] http://www.apache.org/dev/release-signing.html
>> >> > > > > > > > > [6] http://www.apache.org/dist/incubator/nifi/KEYS
>> >> > > > > > > > > [7] http://creadur.apache.org/rat/
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > > > Tony Kurc wrote:
>> >> > > > > > > > >
>> >> > > > > > > > >> I read the guide Joe linked and a lot of the sticky
>> parts
>> >> > are
>> >> > > > > marked
>> >> > > > > > > > >> "TODO"
>> >> > > > > > > > >> and it looks like a work in progress
>> >> > > > > > > > >>
>> >> > > > > > > > >> "Podlings can short circuit this process by starting
>> out
>> >> > with
>> >> > > > > > written
>> >> > > > > > > > >> release documentation. It is strongly recommended that
>> >> > > Podlings
>> >> > > > > > invest
>> >> > > > > > > > >> time
>> >> > > > > > > > >> looking at the best practices recommended in this
>> >> document.
>> >> > By
>> >> > > > > > > selection
>> >> > > > > > > > >> and modification, sections of this document can be used
>> >> to
>> >> > > > quickly
>> >> > > > > > and
>> >> > > > > > > > >> easily bootstrap a release guide. "
>> >> > > > > > > > >>
>> >> > > > > > > > >> Is step one putting together a release guide?
>> >> > > > > > > > >>
>> >> > > > > > > > >>
>> >> > > > > > > > >>
>> >> > > > > > > > >> On Wed, Jan 7, 2015 at 9:26 PM, Joe Witt<
>> >> [hidden email]
>> >> > >
>> >> > > > > > wrote:
>> >> > > > > > > > >>
>> >> > > > > > > > >>  Hello
>> >> > > > > > > > >>>
>> >> > > > > > > > >>> Just wanted to stir this one up a bit.  Looks like all
>> >> > > tickets
>> >> > > > > > pegged
>> >> > > > > > > > as
>> >> > > > > > > > >>> 0.0.1 are resolved (2 remain as of now but seem likely
>> >> to
>> >> > be
>> >> > > > > > resolved
>> >> > > > > > > > >>> shortly based on their comments).  So working through
>> >> the
>> >> > > > release
>> >> > > > > > > steps
>> >> > > > > > > > >>> available on apache.org and via the link Brock sent.
>> >> > > > > > > > >>>
>> >> > > > > > > > >>> Anyone interested in this part of the process or who
>> has
>> >> > > advice
>> >> > > > > to
>> >> > > > > > > help
>> >> > > > > > > > >>> us
>> >> > > > > > > > >>> avoid landmines we're happy to hear it.
>> >> > > > > > > > >>>
>> >> > > > > > > > >>> Thanks
>> >> > > > > > > > >>> Joe
>> >> > > > > > > > >>>
>> >> > > > > > > > >>> On Thu, Dec 18, 2014 at 1:31 PM, Joe Witt<
>> >> > [hidden email]
>> >> > > >
>> >> > > > > > > wrote:
>> >> > > > > > > > >>>
>> >> > > > > > > > >>>  Thanks Brock this is very helpful.
>> >> > > > > > > > >>>>
>> >> > > > > > > > >>>> On Thu, Dec 18, 2014 at 1:27 PM, Brock Noland<
>> >> > > > > [hidden email]>
>> >> > > > > > > > >>>>
>> >> > > > > > > > >>> wrote:
>> >> > > > > > > > >>>
>> >> > > > > > > > >>>> Hi,
>> >> > > > > > > > >>>>>
>> >> > > > > > > > >>>>> This is a decent guide which can be copied:
>> >> > > > > > > > >>>>>
>> >> > > > > > > > >>>>> http://mrunit.apache.org/pmc/how_to_release.html
>> >> > > > > > > > >>>>>
>> >> > > > > > > > >>>>> Brock
>> >> > > > > > > > >>>>>
>> >> > > > > > > > >>>>> On Thu, Dec 18, 2014 at 10:17 AM, Joe Witt<
>> >> > > > [hidden email]>
>> >> > > > > > > > wrote:
>> >> > > > > > > > >>>>>
>> >> > > > > > > > >>>>>> Folks,
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>>> Looking at the tickets that remain which are
>> >> presently
>> >> > > tied
>> >> > > > to
>> >> > > > > > > 0.0.1
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>> we're
>> >> > > > > > > > >>>>>
>> >> > > > > > > > >>>>>> probably 1-2 weeks out from this initial release.
>> >> Can
>> >> > you
>> >> > > > > > provide
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>> some
>> >> > > > > > > > >>>
>> >> > > > > > > > >>>> pointers/references or pointers on how to get this
>> ball
>> >> > > > rolling
>> >> > > > > > and
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>> any
>> >> > > > > > > > >>>
>> >> > > > > > > > >>>> rocks we must move before going down this path?
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>>>
>> >> > http://incubator.apache.org/guides/releasemanagement.html
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>>> That link seems really helpful but has a lot of
>> TODOs
>> >> > for
>> >> > > > > areas
>> >> > > > > > > > which
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>> need
>> >> > > > > > > > >>>>>
>> >> > > > > > > > >>>>>> more explanation.
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>>> Thanks
>> >> > > > > > > > >>>>>> Joe
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>>>>>
>> >> > > > > > > > >>
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>>



--
Joey Echeverria
12