getting the website build setup

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

getting the website build setup

trkurc
Administrator
My environment:

tkurc@avila:~/development/incubator-nifi/nifi-site$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.1 LTS
Release:        14.04
Codename:       trusty

Commands to get it up:
%> sudo apt-get install npm
%> sudo npm install -g grunt-cli
%> sudo npm install -g bower
%> sudo apt-get install ruby-dev
%> sudo gem install compass
%> sudo apt-get install nodejs-legacy # this got me hung up, grunt refused
to run without the legacy package on ubuntu

%> ruby --version
ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
%>  npm --version
1.3.10
%> gem --version
1.8.23
%>  node --version
v0.10.25

Had to do this for some reason...

%> bower install
%> npm install
%> grunt

That seemed to get me working .. except what was unexpected was the
generation of docs from ../nifi. I sort of expected the web site to be
'versioned' separately from the application (although I admittedly did not
check the mailing list archive as thoroughly as I probably should have),
but it looks like it was traversing up and maven building the docs based on
what I had checked out (I eventually checked out a tag because the build
was failing on develop with some missing SNAPSHOTS from other parts of nifi
I hadn't built). So, when building the website ... it should be somehow
based off the current tagged release of nifi?

I'm happy to work on the README.md, but wanted to make sure I understood
the release 'concept' and process.

Tony
Reply | Threaded
Open this post in threaded view
|

Re: getting the website build setup

Matt Gilman
Tony,

Glad you got up and running! If I'm understanding you correctly...

%> bower install      // downloads front end libraries as configured in
bower.json
%> npm install         // downloads grunt 'plugins' for executing the build
as configured in package.json

You should only need to perform these tasks once. They will download the
artifacts to bower_components and node_modules respectively. Once this is
done they do not need to be executed again unless there is a new version of
a library/plugin. This is basically analogous to maven plugins/dependencies.

%> grunt                  // this will build the website

The process for getting the guides and REST docs that are generated with
the build into our website was discussed but there were no clean obvious
solutions. We could certainly base the build on a specific tag. However,
this would mean that if we wanted to update one of the guides and then
deploy it to the website we would need to either create the tag manually or
wait until we release a new version and use that tag. I'm happy with either
of these options depending on how urgent the update was.

I had originally wanted to keep the website and generated documentation
separated. However, there are some modifications that happen to the guides
and REST docs prior to deployment that should not be included in our normal
Apache release. Look at the grunt configuration if you want to see the
details [1]. The goal was to keep this automated but it assumes that the
nifi source, which is co-located with nifi-site, is in a buildable state.
This assumption seems fine if the intent is to release an updated guide or
REST documentation, however if that is not the case it might be asking too
much especially if it is not clear that's how the build works. I tried to
call that out in the 'Grunt Tasks' section but maybe I could have been more
clear.

Also, we are currently hung up on trying to get the component documentation
into the site. Dan Bress has done a lot of work auto generating
documentation based off our our processor annotations. However at the
moment, this means they aren't available until runtime. There is no clear
path for including them in the site in an automated fashion. Any
thoughts/suggestions on that or the site build process are welcome.

Thanks.

Matt

[1]
https://github.com/apache/incubator-nifi/blob/develop/nifi-site/Gruntfile.js#L298

On Tue, May 19, 2015 at 10:46 PM, Tony Kurc <[hidden email]> wrote:

> My environment:
>
> tkurc@avila:~/development/incubator-nifi/nifi-site$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description:    Ubuntu 14.04.1 LTS
> Release:        14.04
> Codename:       trusty
>
> Commands to get it up:
> %> sudo apt-get install npm
> %> sudo npm install -g grunt-cli
> %> sudo npm install -g bower
> %> sudo apt-get install ruby-dev
> %> sudo gem install compass
> %> sudo apt-get install nodejs-legacy # this got me hung up, grunt refused
> to run without the legacy package on ubuntu
>
> %> ruby --version
> ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
> %>  npm --version
> 1.3.10
> %> gem --version
> 1.8.23
> %>  node --version
> v0.10.25
>
> Had to do this for some reason...
>
> %> bower install
> %> npm install
> %> grunt
>
> That seemed to get me working .. except what was unexpected was the
> generation of docs from ../nifi. I sort of expected the web site to be
> 'versioned' separately from the application (although I admittedly did not
> check the mailing list archive as thoroughly as I probably should have),
> but it looks like it was traversing up and maven building the docs based on
> what I had checked out (I eventually checked out a tag because the build
> was failing on develop with some missing SNAPSHOTS from other parts of nifi
> I hadn't built). So, when building the website ... it should be somehow
> based off the current tagged release of nifi?
>
> I'm happy to work on the README.md, but wanted to make sure I understood
> the release 'concept' and process.
>
> Tony
>
Reply | Threaded
Open this post in threaded view
|

Re: getting the website build setup

Dan Bress
All,
    Like Matt said, if anyone has any ideas how to good generate documentation at build time I'd love to hear them.  What I am proposing I think would work, but is a little bit awkward

   1) Add a target to the nifi startup process, something like ./nifi.sh generate-documentation
   2) This would 'start' nifi enough so that documentation can be generated, then immediately stop
   3) The documentation that is generated would include Processor/ControllerService/Reporting task documentation, Rest API documentation
    3a) This would generate a "listing" page that summarizes all the Processors/ControllerServices/Reporting tasks and links to them
    3b) Perhaps we could take some of the modifications Matt Gilman mentioned are being done in grunt be done in here if that makes sense
   4) Have a new artifact that depends on the assembly, extracts it, runs './nifi.sh generate-documentation' and bundles all the generated docs in in a zip


   I fully concede that it is awkward to tie generating documentation for our website into the application it's self, but I don't see a better way to do it at the moment.  Again if anyone has a better idea, please speak up.

Dan Bress
Software Engineer
ONYX Consulting Services

________________________________________
From: Matt Gilman <[hidden email]>
Sent: Tuesday, May 19, 2015 11:26 PM
To: [hidden email]
Subject: Re: getting the website build setup

Tony,

Glad you got up and running! If I'm understanding you correctly...

%> bower install      // downloads front end libraries as configured in
bower.json
%> npm install         // downloads grunt 'plugins' for executing the build
as configured in package.json

You should only need to perform these tasks once. They will download the
artifacts to bower_components and node_modules respectively. Once this is
done they do not need to be executed again unless there is a new version of
a library/plugin. This is basically analogous to maven plugins/dependencies.

%> grunt                  // this will build the website

The process for getting the guides and REST docs that are generated with
the build into our website was discussed but there were no clean obvious
solutions. We could certainly base the build on a specific tag. However,
this would mean that if we wanted to update one of the guides and then
deploy it to the website we would need to either create the tag manually or
wait until we release a new version and use that tag. I'm happy with either
of these options depending on how urgent the update was.

I had originally wanted to keep the website and generated documentation
separated. However, there are some modifications that happen to the guides
and REST docs prior to deployment that should not be included in our normal
Apache release. Look at the grunt configuration if you want to see the
details [1]. The goal was to keep this automated but it assumes that the
nifi source, which is co-located with nifi-site, is in a buildable state.
This assumption seems fine if the intent is to release an updated guide or
REST documentation, however if that is not the case it might be asking too
much especially if it is not clear that's how the build works. I tried to
call that out in the 'Grunt Tasks' section but maybe I could have been more
clear.

Also, we are currently hung up on trying to get the component documentation
into the site. Dan Bress has done a lot of work auto generating
documentation based off our our processor annotations. However at the
moment, this means they aren't available until runtime. There is no clear
path for including them in the site in an automated fashion. Any
thoughts/suggestions on that or the site build process are welcome.

Thanks.

Matt

[1]
https://github.com/apache/incubator-nifi/blob/develop/nifi-site/Gruntfile.js#L298

On Tue, May 19, 2015 at 10:46 PM, Tony Kurc <[hidden email]> wrote:

> My environment:
>
> tkurc@avila:~/development/incubator-nifi/nifi-site$ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description:    Ubuntu 14.04.1 LTS
> Release:        14.04
> Codename:       trusty
>
> Commands to get it up:
> %> sudo apt-get install npm
> %> sudo npm install -g grunt-cli
> %> sudo npm install -g bower
> %> sudo apt-get install ruby-dev
> %> sudo gem install compass
> %> sudo apt-get install nodejs-legacy # this got me hung up, grunt refused
> to run without the legacy package on ubuntu
>
> %> ruby --version
> ruby 1.9.3p484 (2013-11-22 revision 43786) [x86_64-linux]
> %>  npm --version
> 1.3.10
> %> gem --version
> 1.8.23
> %>  node --version
> v0.10.25
>
> Had to do this for some reason...
>
> %> bower install
> %> npm install
> %> grunt
>
> That seemed to get me working .. except what was unexpected was the
> generation of docs from ../nifi. I sort of expected the web site to be
> 'versioned' separately from the application (although I admittedly did not
> check the mailing list archive as thoroughly as I probably should have),
> but it looks like it was traversing up and maven building the docs based on
> what I had checked out (I eventually checked out a tag because the build
> was failing on develop with some missing SNAPSHOTS from other parts of nifi
> I hadn't built). So, when building the website ... it should be somehow
> based off the current tagged release of nifi?
>
> I'm happy to work on the README.md, but wanted to make sure I understood
> the release 'concept' and process.
>
> Tony
>