NiFi Toolkit CLI Token Creation

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

NiFi Toolkit CLI Token Creation

Shawn Weeks
I work in an environment reluctant to create self signed ssl certificates and I’m looking at the feasibility of having the toolkit cli authenticate via Kerberos. I was expecting it to be as simple as adding another way to get the authentication token but I’m having trouble figuring out exactly when the token is created. I see lots of references to it after it’s been created.

Thanks
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Andy LoPresto-2
Shawn,

I’m not sure I understand your question.

I am in the process of refactoring the TLS Toolkit to integrate with public certificate authorities, so in the near future it will be easier to use certificates signed by external authorities rather than self-signed.

My understanding is that you are talking about the CLI Toolkit rather than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m going to proceed with the understanding that you are referring to the JWT token used to identify an authenticated user when communicating with the NiFi API.

You may want to look at JerseyNiFiClient [1], which has methods for getting various clients given an authentication token.

You can create the token via the POST /access/kerberos API [2].

[1] https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163 <https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163>
[2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>

Andy LoPresto
[hidden email]
[hidden email]
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]> wrote:
>
> I work in an environment reluctant to create self signed ssl certificates and I’m looking at the feasibility of having the toolkit cli authenticate via Kerberos. I was expecting it to be as simple as adding another way to get the authentication token but I’m having trouble figuring out exactly when the token is created. I see lots of references to it after it’s been created.
>
> Thanks
> Shawn

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Bryan Bende
Right now the idea is that whoever is running the CLI would have access to
a NiFi server certificate and then you can proxy any user you want. There
should be examples of this in the readme or toolkit guide.

Supporting Kerberos auth was something I wanted to do, but it’s definitely
not a trivial effort.

On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]> wrote:

> Shawn,
>
> I’m not sure I understand your question.
>
> I am in the process of refactoring the TLS Toolkit to integrate with
> public certificate authorities, so in the near future it will be easier to
> use certificates signed by external authorities rather than self-signed.
>
> My understanding is that you are talking about the CLI Toolkit rather than
> the TLS Toolkit, but your reference to “token” was ambiguous, so I’m going
> to proceed with the understanding that you are referring to the JWT token
> used to identify an authenticated user when communicating with the NiFi
> API.
>
> You may want to look at JerseyNiFiClient [1], which has methods for
> getting various clients given an authentication token.
>
> You can create the token via the POST /access/kerberos API [2].
>
> [1]
> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> <
> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> >
> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>
> Andy LoPresto
> [hidden email]
> [hidden email]
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
> wrote:
> >
> > I work in an environment reluctant to create self signed ssl
> certificates and I’m looking at the feasibility of having the toolkit cli
> authenticate via Kerberos. I was expecting it to be as simple as adding
> another way to get the authentication token but I’m having trouble figuring
> out exactly when the token is created. I see lots of references to it after
> it’s been created.
> >
> > Thanks
> > Shawn
>
> --
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Bryan Bende
I meant to say that you obviously could generate certs for CLI users, but I
was just mentioning an alternative where you can proxy an identity.

Right now the CLI never obtains a token because it is all cert based.

On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:

> Right now the idea is that whoever is running the CLI would have access to
> a NiFi server certificate and then you can proxy any user you want. There
> should be examples of this in the readme or toolkit guide.
>
> Supporting Kerberos auth was something I wanted to do, but it’s definitely
> not a trivial effort.
>
> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
> wrote:
>
>> Shawn,
>>
>> I’m not sure I understand your question.
>>
>> I am in the process of refactoring the TLS Toolkit to integrate with
>> public certificate authorities, so in the near future it will be easier to
>> use certificates signed by external authorities rather than self-signed.
>>
>> My understanding is that you are talking about the CLI Toolkit rather
>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>> going to proceed with the understanding that you are referring to the JWT
>> token used to identify an authenticated user when communicating with the
>> NiFi API.
>>
>> You may want to look at JerseyNiFiClient [1], which has methods for
>> getting various clients given an authentication token.
>>
>> You can create the token via the POST /access/kerberos API [2].
>>
>> [1]
>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>> <
>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>> >
>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>>
>> Andy LoPresto
>> [hidden email]
>> [hidden email]
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>> > On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>> wrote:
>> >
>> > I work in an environment reluctant to create self signed ssl
>> certificates and I’m looking at the feasibility of having the toolkit cli
>> authenticate via Kerberos. I was expecting it to be as simple as adding
>> another way to get the authentication token but I’m having trouble figuring
>> out exactly when the token is created. I see lots of references to it after
>> it’s been created.
>> >
>> > Thanks
>> > Shawn
>>
>> --
> Sent from Gmail Mobile
>
--
Sent from Gmail Mobile
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Shawn Weeks
For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.

As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.

Thanks
Shawn

On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:

    I meant to say that you obviously could generate certs for CLI users, but I
    was just mentioning an alternative where you can proxy an identity.
   
    Right now the CLI never obtains a token because it is all cert based.
   
    On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
   
    > Right now the idea is that whoever is running the CLI would have access to
    > a NiFi server certificate and then you can proxy any user you want. There
    > should be examples of this in the readme or toolkit guide.
    >
    > Supporting Kerberos auth was something I wanted to do, but it’s definitely
    > not a trivial effort.
    >
    > On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
    > wrote:
    >
    >> Shawn,
    >>
    >> I’m not sure I understand your question.
    >>
    >> I am in the process of refactoring the TLS Toolkit to integrate with
    >> public certificate authorities, so in the near future it will be easier to
    >> use certificates signed by external authorities rather than self-signed.
    >>
    >> My understanding is that you are talking about the CLI Toolkit rather
    >> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
    >> going to proceed with the understanding that you are referring to the JWT
    >> token used to identify an authenticated user when communicating with the
    >> NiFi API.
    >>
    >> You may want to look at JerseyNiFiClient [1], which has methods for
    >> getting various clients given an authentication token.
    >>
    >> You can create the token via the POST /access/kerberos API [2].
    >>
    >> [1]
    >> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >> <
    >> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >> >
    >> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
    >> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
    >>
    >> Andy LoPresto
    >> [hidden email]
    >> [hidden email]
    >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >>
    >> > On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
    >> wrote:
    >> >
    >> > I work in an environment reluctant to create self signed ssl
    >> certificates and I’m looking at the feasibility of having the toolkit cli
    >> authenticate via Kerberos. I was expecting it to be as simple as adding
    >> another way to get the authentication token but I’m having trouble figuring
    >> out exactly when the token is created. I see lots of references to it after
    >> it’s been created.
    >> >
    >> > Thanks
    >> > Shawn
    >>
    >> --
    > Sent from Gmail Mobile
    >
    --
    Sent from Gmail Mobile
   

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Andy LoPresto-2
I see a couple choices here:

1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.

Andy LoPresto
[hidden email]
[hidden email]
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>
> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>
> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>
> Thanks
> Shawn
>
> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>
>    I meant to say that you obviously could generate certs for CLI users, but I
>    was just mentioning an alternative where you can proxy an identity.
>
>    Right now the CLI never obtains a token because it is all cert based.
>
>    On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>
>> Right now the idea is that whoever is running the CLI would have access to
>> a NiFi server certificate and then you can proxy any user you want. There
>> should be examples of this in the readme or toolkit guide.
>>
>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>> not a trivial effort.
>>
>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>> wrote:
>>
>>> Shawn,
>>>
>>> I’m not sure I understand your question.
>>>
>>> I am in the process of refactoring the TLS Toolkit to integrate with
>>> public certificate authorities, so in the near future it will be easier to
>>> use certificates signed by external authorities rather than self-signed.
>>>
>>> My understanding is that you are talking about the CLI Toolkit rather
>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>>> going to proceed with the understanding that you are referring to the JWT
>>> token used to identify an authenticated user when communicating with the
>>> NiFi API.
>>>
>>> You may want to look at JerseyNiFiClient [1], which has methods for
>>> getting various clients given an authentication token.
>>>
>>> You can create the token via the POST /access/kerberos API [2].
>>>
>>> [1]
>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>> <
>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>
>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>>>
>>> Andy LoPresto
>>> [hidden email]
>>> [hidden email]
>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>
>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>>> wrote:
>>>>
>>>> I work in an environment reluctant to create self signed ssl
>>> certificates and I’m looking at the feasibility of having the toolkit cli
>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>>> another way to get the authentication token but I’m having trouble figuring
>>> out exactly when the token is created. I see lots of references to it after
>>> it’s been created.
>>>>
>>>> Thanks
>>>> Shawn
>>>
>>> --
>> Sent from Gmail Mobile
>>
>    --
>    Sent from Gmail Mobile
>
>

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Shawn Weeks
Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?

Thanks
Shawn

Sent from my iPhone

> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
>
> I see a couple choices here:
>
> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
>
> Andy LoPresto
> [hidden email]
> [hidden email]
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>>
>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>>
>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>>
>> Thanks
>> Shawn
>>
>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>>
>>   I meant to say that you obviously could generate certs for CLI users, but I
>>   was just mentioning an alternative where you can proxy an identity.
>>
>>   Right now the CLI never obtains a token because it is all cert based.
>>
>>>   On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>>>
>>> Right now the idea is that whoever is running the CLI would have access to
>>> a NiFi server certificate and then you can proxy any user you want. There
>>> should be examples of this in the readme or toolkit guide.
>>>
>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>>> not a trivial effort.
>>>
>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>>> wrote:
>>>
>>>> Shawn,
>>>>
>>>> I’m not sure I understand your question.
>>>>
>>>> I am in the process of refactoring the TLS Toolkit to integrate with
>>>> public certificate authorities, so in the near future it will be easier to
>>>> use certificates signed by external authorities rather than self-signed.
>>>>
>>>> My understanding is that you are talking about the CLI Toolkit rather
>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>>>> going to proceed with the understanding that you are referring to the JWT
>>>> token used to identify an authenticated user when communicating with the
>>>> NiFi API.
>>>>
>>>> You may want to look at JerseyNiFiClient [1], which has methods for
>>>> getting various clients given an authentication token.
>>>>
>>>> You can create the token via the POST /access/kerberos API [2].
>>>>
>>>> [1]
>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>> <
>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>>
>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>>>>
>>>> Andy LoPresto
>>>> [hidden email]
>>>> [hidden email]
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>
>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>>>> wrote:
>>>>>
>>>>> I work in an environment reluctant to create self signed ssl
>>>> certificates and I’m looking at the feasibility of having the toolkit cli
>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>>>> another way to get the authentication token but I’m having trouble figuring
>>>> out exactly when the token is created. I see lots of references to it after
>>>> it’s been created.
>>>>>
>>>>> Thanks
>>>>> Shawn
>>>>
>>>> --
>>> Sent from Gmail Mobile
>>>
>>   --
>>   Sent from Gmail Mobile
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Andy LoPresto-2
You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.

Andy LoPresto
[hidden email]
[hidden email]
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
>
> Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
>
> Thanks
> Shawn
>
> Sent from my iPhone
>
>> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
>>
>> I see a couple choices here:
>>
>> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
>> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
>> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
>>
>> Andy LoPresto
>> [hidden email]
>> [hidden email]
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>>>
>>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>>>
>>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>>>
>>> Thanks
>>> Shawn
>>>
>>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>>>
>>>  I meant to say that you obviously could generate certs for CLI users, but I
>>>  was just mentioning an alternative where you can proxy an identity.
>>>
>>>  Right now the CLI never obtains a token because it is all cert based.
>>>
>>>>  On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>>>>
>>>> Right now the idea is that whoever is running the CLI would have access to
>>>> a NiFi server certificate and then you can proxy any user you want. There
>>>> should be examples of this in the readme or toolkit guide.
>>>>
>>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>>>> not a trivial effort.
>>>>
>>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>>>> wrote:
>>>>
>>>>> Shawn,
>>>>>
>>>>> I’m not sure I understand your question.
>>>>>
>>>>> I am in the process of refactoring the TLS Toolkit to integrate with
>>>>> public certificate authorities, so in the near future it will be easier to
>>>>> use certificates signed by external authorities rather than self-signed.
>>>>>
>>>>> My understanding is that you are talking about the CLI Toolkit rather
>>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>>>>> going to proceed with the understanding that you are referring to the JWT
>>>>> token used to identify an authenticated user when communicating with the
>>>>> NiFi API.
>>>>>
>>>>> You may want to look at JerseyNiFiClient [1], which has methods for
>>>>> getting various clients given an authentication token.
>>>>>
>>>>> You can create the token via the POST /access/kerberos API [2].
>>>>>
>>>>> [1]
>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>> <
>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>>>
>>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>>>>>
>>>>> Andy LoPresto
>>>>> [hidden email]
>>>>> [hidden email]
>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>>
>>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> I work in an environment reluctant to create self signed ssl
>>>>> certificates and I’m looking at the feasibility of having the toolkit cli
>>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>>>>> another way to get the authentication token but I’m having trouble figuring
>>>>> out exactly when the token is created. I see lots of references to it after
>>>>> it’s been created.
>>>>>>
>>>>>> Thanks
>>>>>> Shawn
>>>>>
>>>>> --
>>>> Sent from Gmail Mobile
>>>>
>>>  --
>>>  Sent from Gmail Mobile
>>>
>>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Bryan Bende
Just to further elaborate, within the CLI there are commands that work
against registry and commands that work against NiFi. For registry
commands, they use the Java client that is provided by registry [1].
For NiFi commands, there is mini client developed as need with in the
CLI [2].

None of these client calls currently have any concept of a JWT/token.

In order to do the kerberos auth correctly across both systems, I
think both of these clients would need to be updated to support a
method that called the /access/kerberos end point to obtain a token,
and then also provide a way to pass back that token on future
requests. It would likely be the CLI's job to store that token
somewhere (in memory for interactive shell, or on filesystem for
individual executions) and pass it back on each request. In order to
call the /access/kerberos end-point there also needs to be code in the
client that handles the negotiation to provide the kerberos
credentials that are present from having done a kinit.

Long story short, Andy's first suggest would be a much easier option
with no code changes.

[1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
[2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client

On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:

>
> You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
>
> Andy LoPresto
> [hidden email]
> [hidden email]
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
> >
> > Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
> >
> > Thanks
> > Shawn
> >
> > Sent from my iPhone
> >
> >> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
> >>
> >> I see a couple choices here:
> >>
> >> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
> >> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
> >> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
> >>
> >> Andy LoPresto
> >> [hidden email]
> >> [hidden email]
> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>
> >>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
> >>>
> >>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
> >>>
> >>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
> >>>
> >>> Thanks
> >>> Shawn
> >>>
> >>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
> >>>
> >>>  I meant to say that you obviously could generate certs for CLI users, but I
> >>>  was just mentioning an alternative where you can proxy an identity.
> >>>
> >>>  Right now the CLI never obtains a token because it is all cert based.
> >>>
> >>>>  On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
> >>>>
> >>>> Right now the idea is that whoever is running the CLI would have access to
> >>>> a NiFi server certificate and then you can proxy any user you want. There
> >>>> should be examples of this in the readme or toolkit guide.
> >>>>
> >>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
> >>>> not a trivial effort.
> >>>>
> >>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
> >>>> wrote:
> >>>>
> >>>>> Shawn,
> >>>>>
> >>>>> I’m not sure I understand your question.
> >>>>>
> >>>>> I am in the process of refactoring the TLS Toolkit to integrate with
> >>>>> public certificate authorities, so in the near future it will be easier to
> >>>>> use certificates signed by external authorities rather than self-signed.
> >>>>>
> >>>>> My understanding is that you are talking about the CLI Toolkit rather
> >>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
> >>>>> going to proceed with the understanding that you are referring to the JWT
> >>>>> token used to identify an authenticated user when communicating with the
> >>>>> NiFi API.
> >>>>>
> >>>>> You may want to look at JerseyNiFiClient [1], which has methods for
> >>>>> getting various clients given an authentication token.
> >>>>>
> >>>>> You can create the token via the POST /access/kerberos API [2].
> >>>>>
> >>>>> [1]
> >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> >>>>> <
> >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> >>>>>>
> >>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
> >>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
> >>>>>
> >>>>> Andy LoPresto
> >>>>> [hidden email]
> >>>>> [hidden email]
> >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>>
> >>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
> >>>>> wrote:
> >>>>>>
> >>>>>> I work in an environment reluctant to create self signed ssl
> >>>>> certificates and I’m looking at the feasibility of having the toolkit cli
> >>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
> >>>>> another way to get the authentication token but I’m having trouble figuring
> >>>>> out exactly when the token is created. I see lots of references to it after
> >>>>> it’s been created.
> >>>>>>
> >>>>>> Thanks
> >>>>>> Shawn
> >>>>>
> >>>>> --
> >>>> Sent from Gmail Mobile
> >>>>
> >>>  --
> >>>  Sent from Gmail Mobile
> >>>
> >>>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Shawn Weeks
Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.

Thanks
Shawn

On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:

    Just to further elaborate, within the CLI there are commands that work
    against registry and commands that work against NiFi. For registry
    commands, they use the Java client that is provided by registry [1].
    For NiFi commands, there is mini client developed as need with in the
    CLI [2].
   
    None of these client calls currently have any concept of a JWT/token.
   
    In order to do the kerberos auth correctly across both systems, I
    think both of these clients would need to be updated to support a
    method that called the /access/kerberos end point to obtain a token,
    and then also provide a way to pass back that token on future
    requests. It would likely be the CLI's job to store that token
    somewhere (in memory for interactive shell, or on filesystem for
    individual executions) and pass it back on each request. In order to
    call the /access/kerberos end-point there also needs to be code in the
    client that handles the negotiation to provide the kerberos
    credentials that are present from having done a kinit.
   
    Long story short, Andy's first suggest would be a much easier option
    with no code changes.
   
    [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
    [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
   
    On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
    >
    > You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
    >
    > Andy LoPresto
    > [hidden email]
    > [hidden email]
    > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >
    > > On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
    > >
    > > Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
    > >
    > > Thanks
    > > Shawn
    > >
    > > Sent from my iPhone
    > >
    > >> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
    > >>
    > >> I see a couple choices here:
    > >>
    > >> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
    > >> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
    > >> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
    > >>
    > >> Andy LoPresto
    > >> [hidden email]
    > >> [hidden email]
    > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    > >>
    > >>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
    > >>>
    > >>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
    > >>>
    > >>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
    > >>>
    > >>> Thanks
    > >>> Shawn
    > >>>
    > >>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
    > >>>
    > >>>  I meant to say that you obviously could generate certs for CLI users, but I
    > >>>  was just mentioning an alternative where you can proxy an identity.
    > >>>
    > >>>  Right now the CLI never obtains a token because it is all cert based.
    > >>>
    > >>>>  On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
    > >>>>
    > >>>> Right now the idea is that whoever is running the CLI would have access to
    > >>>> a NiFi server certificate and then you can proxy any user you want. There
    > >>>> should be examples of this in the readme or toolkit guide.
    > >>>>
    > >>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
    > >>>> not a trivial effort.
    > >>>>
    > >>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
    > >>>> wrote:
    > >>>>
    > >>>>> Shawn,
    > >>>>>
    > >>>>> I’m not sure I understand your question.
    > >>>>>
    > >>>>> I am in the process of refactoring the TLS Toolkit to integrate with
    > >>>>> public certificate authorities, so in the near future it will be easier to
    > >>>>> use certificates signed by external authorities rather than self-signed.
    > >>>>>
    > >>>>> My understanding is that you are talking about the CLI Toolkit rather
    > >>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
    > >>>>> going to proceed with the understanding that you are referring to the JWT
    > >>>>> token used to identify an authenticated user when communicating with the
    > >>>>> NiFi API.
    > >>>>>
    > >>>>> You may want to look at JerseyNiFiClient [1], which has methods for
    > >>>>> getting various clients given an authentication token.
    > >>>>>
    > >>>>> You can create the token via the POST /access/kerberos API [2].
    > >>>>>
    > >>>>> [1]
    > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    > >>>>> <
    > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    > >>>>>>
    > >>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
    > >>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
    > >>>>>
    > >>>>> Andy LoPresto
    > >>>>> [hidden email]
    > >>>>> [hidden email]
    > >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    > >>>>>
    > >>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
    > >>>>> wrote:
    > >>>>>>
    > >>>>>> I work in an environment reluctant to create self signed ssl
    > >>>>> certificates and I’m looking at the feasibility of having the toolkit cli
    > >>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
    > >>>>> another way to get the authentication token but I’m having trouble figuring
    > >>>>> out exactly when the token is created. I see lots of references to it after
    > >>>>> it’s been created.
    > >>>>>>
    > >>>>>> Thanks
    > >>>>>> Shawn
    > >>>>>
    > >>>>> --
    > >>>> Sent from Gmail Mobile
    > >>>>
    > >>>  --
    > >>>  Sent from Gmail Mobile
    > >>>
    > >>>
    > >>
    >
   

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Bryan Bende
I'm not sure if I confused things... the clients that I mentioned are
wrappers for the REST API implemented with Jersey client, so the CLI
does exclusively use the REST API.

I was just drawing attention to the clients to say that part of the
work is outside of the CLI in nifi-registry-client to allow it to
support kerberos auth.

On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:

>
> Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
>
> Thanks
> Shawn
>
> On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
>
>     Just to further elaborate, within the CLI there are commands that work
>     against registry and commands that work against NiFi. For registry
>     commands, they use the Java client that is provided by registry [1].
>     For NiFi commands, there is mini client developed as need with in the
>     CLI [2].
>
>     None of these client calls currently have any concept of a JWT/token.
>
>     In order to do the kerberos auth correctly across both systems, I
>     think both of these clients would need to be updated to support a
>     method that called the /access/kerberos end point to obtain a token,
>     and then also provide a way to pass back that token on future
>     requests. It would likely be the CLI's job to store that token
>     somewhere (in memory for interactive shell, or on filesystem for
>     individual executions) and pass it back on each request. In order to
>     call the /access/kerberos end-point there also needs to be code in the
>     client that handles the negotiation to provide the kerberos
>     credentials that are present from having done a kinit.
>
>     Long story short, Andy's first suggest would be a much easier option
>     with no code changes.
>
>     [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
>     [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
>
>     On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
>     >
>     > You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
>     >
>     > Andy LoPresto
>     > [hidden email]
>     > [hidden email]
>     > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >
>     > > On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
>     > >
>     > > Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
>     > >
>     > > Thanks
>     > > Shawn
>     > >
>     > > Sent from my iPhone
>     > >
>     > >> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
>     > >>
>     > >> I see a couple choices here:
>     > >>
>     > >> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
>     > >> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
>     > >> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
>     > >>
>     > >> Andy LoPresto
>     > >> [hidden email]
>     > >> [hidden email]
>     > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     > >>
>     > >>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>     > >>>
>     > >>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>     > >>>
>     > >>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>     > >>>
>     > >>> Thanks
>     > >>> Shawn
>     > >>>
>     > >>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>     > >>>
>     > >>>  I meant to say that you obviously could generate certs for CLI users, but I
>     > >>>  was just mentioning an alternative where you can proxy an identity.
>     > >>>
>     > >>>  Right now the CLI never obtains a token because it is all cert based.
>     > >>>
>     > >>>>  On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>     > >>>>
>     > >>>> Right now the idea is that whoever is running the CLI would have access to
>     > >>>> a NiFi server certificate and then you can proxy any user you want. There
>     > >>>> should be examples of this in the readme or toolkit guide.
>     > >>>>
>     > >>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>     > >>>> not a trivial effort.
>     > >>>>
>     > >>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>     > >>>> wrote:
>     > >>>>
>     > >>>>> Shawn,
>     > >>>>>
>     > >>>>> I’m not sure I understand your question.
>     > >>>>>
>     > >>>>> I am in the process of refactoring the TLS Toolkit to integrate with
>     > >>>>> public certificate authorities, so in the near future it will be easier to
>     > >>>>> use certificates signed by external authorities rather than self-signed.
>     > >>>>>
>     > >>>>> My understanding is that you are talking about the CLI Toolkit rather
>     > >>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>     > >>>>> going to proceed with the understanding that you are referring to the JWT
>     > >>>>> token used to identify an authenticated user when communicating with the
>     > >>>>> NiFi API.
>     > >>>>>
>     > >>>>> You may want to look at JerseyNiFiClient [1], which has methods for
>     > >>>>> getting various clients given an authentication token.
>     > >>>>>
>     > >>>>> You can create the token via the POST /access/kerberos API [2].
>     > >>>>>
>     > >>>>> [1]
>     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>     > >>>>> <
>     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>     > >>>>>>
>     > >>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>     > >>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>     > >>>>>
>     > >>>>> Andy LoPresto
>     > >>>>> [hidden email]
>     > >>>>> [hidden email]
>     > >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     > >>>>>
>     > >>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>     > >>>>> wrote:
>     > >>>>>>
>     > >>>>>> I work in an environment reluctant to create self signed ssl
>     > >>>>> certificates and I’m looking at the feasibility of having the toolkit cli
>     > >>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>     > >>>>> another way to get the authentication token but I’m having trouble figuring
>     > >>>>> out exactly when the token is created. I see lots of references to it after
>     > >>>>> it’s been created.
>     > >>>>>>
>     > >>>>>> Thanks
>     > >>>>>> Shawn
>     > >>>>>
>     > >>>>> --
>     > >>>> Sent from Gmail Mobile
>     > >>>>
>     > >>>  --
>     > >>>  Sent from Gmail Mobile
>     > >>>
>     > >>>
>     > >>
>     >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Shawn Weeks
Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.

It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.

https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request

Thanks
Shawn

On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:

    I'm not sure if I confused things... the clients that I mentioned are
    wrappers for the REST API implemented with Jersey client, so the CLI
    does exclusively use the REST API.
   
    I was just drawing attention to the clients to say that part of the
    work is outside of the CLI in nifi-registry-client to allow it to
    support kerberos auth.
   
    On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
    >
    > Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
    >
    > Thanks
    > Shawn
    >
    > On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
    >
    >     Just to further elaborate, within the CLI there are commands that work
    >     against registry and commands that work against NiFi. For registry
    >     commands, they use the Java client that is provided by registry [1].
    >     For NiFi commands, there is mini client developed as need with in the
    >     CLI [2].
    >
    >     None of these client calls currently have any concept of a JWT/token.
    >
    >     In order to do the kerberos auth correctly across both systems, I
    >     think both of these clients would need to be updated to support a
    >     method that called the /access/kerberos end point to obtain a token,
    >     and then also provide a way to pass back that token on future
    >     requests. It would likely be the CLI's job to store that token
    >     somewhere (in memory for interactive shell, or on filesystem for
    >     individual executions) and pass it back on each request. In order to
    >     call the /access/kerberos end-point there also needs to be code in the
    >     client that handles the negotiation to provide the kerberos
    >     credentials that are present from having done a kinit.
    >
    >     Long story short, Andy's first suggest would be a much easier option
    >     with no code changes.
    >
    >     [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
    >     [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
    >
    >     On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
    >     >
    >     > You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
    >     >
    >     > Andy LoPresto
    >     > [hidden email]
    >     > [hidden email]
    >     > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >     >
    >     > > On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
    >     > >
    >     > > Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
    >     > >
    >     > > Thanks
    >     > > Shawn
    >     > >
    >     > > Sent from my iPhone
    >     > >
    >     > >> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
    >     > >>
    >     > >> I see a couple choices here:
    >     > >>
    >     > >> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
    >     > >> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
    >     > >> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
    >     > >>
    >     > >> Andy LoPresto
    >     > >> [hidden email]
    >     > >> [hidden email]
    >     > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >     > >>
    >     > >>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
    >     > >>>
    >     > >>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
    >     > >>>
    >     > >>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
    >     > >>>
    >     > >>> Thanks
    >     > >>> Shawn
    >     > >>>
    >     > >>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
    >     > >>>
    >     > >>>  I meant to say that you obviously could generate certs for CLI users, but I
    >     > >>>  was just mentioning an alternative where you can proxy an identity.
    >     > >>>
    >     > >>>  Right now the CLI never obtains a token because it is all cert based.
    >     > >>>
    >     > >>>>  On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
    >     > >>>>
    >     > >>>> Right now the idea is that whoever is running the CLI would have access to
    >     > >>>> a NiFi server certificate and then you can proxy any user you want. There
    >     > >>>> should be examples of this in the readme or toolkit guide.
    >     > >>>>
    >     > >>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
    >     > >>>> not a trivial effort.
    >     > >>>>
    >     > >>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
    >     > >>>> wrote:
    >     > >>>>
    >     > >>>>> Shawn,
    >     > >>>>>
    >     > >>>>> I’m not sure I understand your question.
    >     > >>>>>
    >     > >>>>> I am in the process of refactoring the TLS Toolkit to integrate with
    >     > >>>>> public certificate authorities, so in the near future it will be easier to
    >     > >>>>> use certificates signed by external authorities rather than self-signed.
    >     > >>>>>
    >     > >>>>> My understanding is that you are talking about the CLI Toolkit rather
    >     > >>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
    >     > >>>>> going to proceed with the understanding that you are referring to the JWT
    >     > >>>>> token used to identify an authenticated user when communicating with the
    >     > >>>>> NiFi API.
    >     > >>>>>
    >     > >>>>> You may want to look at JerseyNiFiClient [1], which has methods for
    >     > >>>>> getting various clients given an authentication token.
    >     > >>>>>
    >     > >>>>> You can create the token via the POST /access/kerberos API [2].
    >     > >>>>>
    >     > >>>>> [1]
    >     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >     > >>>>> <
    >     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >     > >>>>>>
    >     > >>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
    >     > >>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
    >     > >>>>>
    >     > >>>>> Andy LoPresto
    >     > >>>>> [hidden email]
    >     > >>>>> [hidden email]
    >     > >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >     > >>>>>
    >     > >>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
    >     > >>>>> wrote:
    >     > >>>>>>
    >     > >>>>>> I work in an environment reluctant to create self signed ssl
    >     > >>>>> certificates and I’m looking at the feasibility of having the toolkit cli
    >     > >>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
    >     > >>>>> another way to get the authentication token but I’m having trouble figuring
    >     > >>>>> out exactly when the token is created. I see lots of references to it after
    >     > >>>>> it’s been created.
    >     > >>>>>>
    >     > >>>>>> Thanks
    >     > >>>>>> Shawn
    >     > >>>>>
    >     > >>>>> --
    >     > >>>> Sent from Gmail Mobile
    >     > >>>>
    >     > >>>  --
    >     > >>>  Sent from Gmail Mobile
    >     > >>>
    >     > >>>
    >     > >>
    >     >
    >
    >
   

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Bryan Bende
Ah thanks for pointing that out, I completely forgot that apparently I
was thinking ahead in the JerseyNiFiClient of how we could support
tokens :)

You would need to make the same changes in the
JerseyNiFiRegistryClient, and then build a new toolkit based on a new
version of nifi-registry-client.

Also, you are correct that we could support username/password, but I
think Kerberos is much better from a security perspective since you
don't really need to give your credentials to the CLI. With
username/password, you would either need to add those properties to
the .props files for the CLI, which then gets into encrypting the
password, or you need to provide them on a command as arguments which
again gets into whether the password is in plain text or not.

On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:

>
> Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
>
> It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
>
> https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
>
> Thanks
> Shawn
>
> On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
>
>     I'm not sure if I confused things... the clients that I mentioned are
>     wrappers for the REST API implemented with Jersey client, so the CLI
>     does exclusively use the REST API.
>
>     I was just drawing attention to the clients to say that part of the
>     work is outside of the CLI in nifi-registry-client to allow it to
>     support kerberos auth.
>
>     On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
>     >
>     > Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
>     >
>     > Thanks
>     > Shawn
>     >
>     > On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
>     >
>     >     Just to further elaborate, within the CLI there are commands that work
>     >     against registry and commands that work against NiFi. For registry
>     >     commands, they use the Java client that is provided by registry [1].
>     >     For NiFi commands, there is mini client developed as need with in the
>     >     CLI [2].
>     >
>     >     None of these client calls currently have any concept of a JWT/token.
>     >
>     >     In order to do the kerberos auth correctly across both systems, I
>     >     think both of these clients would need to be updated to support a
>     >     method that called the /access/kerberos end point to obtain a token,
>     >     and then also provide a way to pass back that token on future
>     >     requests. It would likely be the CLI's job to store that token
>     >     somewhere (in memory for interactive shell, or on filesystem for
>     >     individual executions) and pass it back on each request. In order to
>     >     call the /access/kerberos end-point there also needs to be code in the
>     >     client that handles the negotiation to provide the kerberos
>     >     credentials that are present from having done a kinit.
>     >
>     >     Long story short, Andy's first suggest would be a much easier option
>     >     with no code changes.
>     >
>     >     [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
>     >     [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
>     >
>     >     On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
>     >     >
>     >     > You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
>     >     >
>     >     > Andy LoPresto
>     >     > [hidden email]
>     >     > [hidden email]
>     >     > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >     >
>     >     > > On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
>     >     > >
>     >     > > Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
>     >     > >
>     >     > > Thanks
>     >     > > Shawn
>     >     > >
>     >     > > Sent from my iPhone
>     >     > >
>     >     > >> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
>     >     > >>
>     >     > >> I see a couple choices here:
>     >     > >>
>     >     > >> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
>     >     > >> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
>     >     > >> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
>     >     > >>
>     >     > >> Andy LoPresto
>     >     > >> [hidden email]
>     >     > >> [hidden email]
>     >     > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >     > >>
>     >     > >>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>     >     > >>>
>     >     > >>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>     >     > >>>
>     >     > >>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>     >     > >>>
>     >     > >>> Thanks
>     >     > >>> Shawn
>     >     > >>>
>     >     > >>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>     >     > >>>
>     >     > >>>  I meant to say that you obviously could generate certs for CLI users, but I
>     >     > >>>  was just mentioning an alternative where you can proxy an identity.
>     >     > >>>
>     >     > >>>  Right now the CLI never obtains a token because it is all cert based.
>     >     > >>>
>     >     > >>>>  On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>     >     > >>>>
>     >     > >>>> Right now the idea is that whoever is running the CLI would have access to
>     >     > >>>> a NiFi server certificate and then you can proxy any user you want. There
>     >     > >>>> should be examples of this in the readme or toolkit guide.
>     >     > >>>>
>     >     > >>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>     >     > >>>> not a trivial effort.
>     >     > >>>>
>     >     > >>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>     >     > >>>> wrote:
>     >     > >>>>
>     >     > >>>>> Shawn,
>     >     > >>>>>
>     >     > >>>>> I’m not sure I understand your question.
>     >     > >>>>>
>     >     > >>>>> I am in the process of refactoring the TLS Toolkit to integrate with
>     >     > >>>>> public certificate authorities, so in the near future it will be easier to
>     >     > >>>>> use certificates signed by external authorities rather than self-signed.
>     >     > >>>>>
>     >     > >>>>> My understanding is that you are talking about the CLI Toolkit rather
>     >     > >>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>     >     > >>>>> going to proceed with the understanding that you are referring to the JWT
>     >     > >>>>> token used to identify an authenticated user when communicating with the
>     >     > >>>>> NiFi API.
>     >     > >>>>>
>     >     > >>>>> You may want to look at JerseyNiFiClient [1], which has methods for
>     >     > >>>>> getting various clients given an authentication token.
>     >     > >>>>>
>     >     > >>>>> You can create the token via the POST /access/kerberos API [2].
>     >     > >>>>>
>     >     > >>>>> [1]
>     >     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>     >     > >>>>> <
>     >     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>     >     > >>>>>>
>     >     > >>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>     >     > >>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>     >     > >>>>>
>     >     > >>>>> Andy LoPresto
>     >     > >>>>> [hidden email]
>     >     > >>>>> [hidden email]
>     >     > >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >     > >>>>>
>     >     > >>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>     >     > >>>>> wrote:
>     >     > >>>>>>
>     >     > >>>>>> I work in an environment reluctant to create self signed ssl
>     >     > >>>>> certificates and I’m looking at the feasibility of having the toolkit cli
>     >     > >>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>     >     > >>>>> another way to get the authentication token but I’m having trouble figuring
>     >     > >>>>> out exactly when the token is created. I see lots of references to it after
>     >     > >>>>> it’s been created.
>     >     > >>>>>>
>     >     > >>>>>> Thanks
>     >     > >>>>>> Shawn
>     >     > >>>>>
>     >     > >>>>> --
>     >     > >>>> Sent from Gmail Mobile
>     >     > >>>>
>     >     > >>>  --
>     >     > >>>  Sent from Gmail Mobile
>     >     > >>>
>     >     > >>>
>     >     > >>
>     >     >
>     >
>     >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Shawn Weeks
Completely agree on username and password but I'll probably still do something somewhat generic around access tokens vs 2 way ssl as in the future there might be something else. On a related note is it possible to get a JWT with 2 way ssl? If so we could use the same auth method for everything.

Thanks
Shawn

On 6/13/19, 1:36 PM, "Bryan Bende" <[hidden email]> wrote:

    Ah thanks for pointing that out, I completely forgot that apparently I
    was thinking ahead in the JerseyNiFiClient of how we could support
    tokens :)
   
    You would need to make the same changes in the
    JerseyNiFiRegistryClient, and then build a new toolkit based on a new
    version of nifi-registry-client.
   
    Also, you are correct that we could support username/password, but I
    think Kerberos is much better from a security perspective since you
    don't really need to give your credentials to the CLI. With
    username/password, you would either need to add those properties to
    the .props files for the CLI, which then gets into encrypting the
    password, or you need to provide them on a command as arguments which
    again gets into whether the password is in plain text or not.
   
    On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:
    >
    > Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
    >
    > It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
    >
    > https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
    >
    > Thanks
    > Shawn
    >
    > On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
    >
    >     I'm not sure if I confused things... the clients that I mentioned are
    >     wrappers for the REST API implemented with Jersey client, so the CLI
    >     does exclusively use the REST API.
    >
    >     I was just drawing attention to the clients to say that part of the
    >     work is outside of the CLI in nifi-registry-client to allow it to
    >     support kerberos auth.
    >
    >     On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
    >     >
    >     > Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
    >     >
    >     > Thanks
    >     > Shawn
    >     >
    >     > On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
    >     >
    >     >     Just to further elaborate, within the CLI there are commands that work
    >     >     against registry and commands that work against NiFi. For registry
    >     >     commands, they use the Java client that is provided by registry [1].
    >     >     For NiFi commands, there is mini client developed as need with in the
    >     >     CLI [2].
    >     >
    >     >     None of these client calls currently have any concept of a JWT/token.
    >     >
    >     >     In order to do the kerberos auth correctly across both systems, I
    >     >     think both of these clients would need to be updated to support a
    >     >     method that called the /access/kerberos end point to obtain a token,
    >     >     and then also provide a way to pass back that token on future
    >     >     requests. It would likely be the CLI's job to store that token
    >     >     somewhere (in memory for interactive shell, or on filesystem for
    >     >     individual executions) and pass it back on each request. In order to
    >     >     call the /access/kerberos end-point there also needs to be code in the
    >     >     client that handles the negotiation to provide the kerberos
    >     >     credentials that are present from having done a kinit.
    >     >
    >     >     Long story short, Andy's first suggest would be a much easier option
    >     >     with no code changes.
    >     >
    >     >     [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
    >     >     [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
    >     >
    >     >     On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
    >     >     >
    >     >     > You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
    >     >     >
    >     >     > Andy LoPresto
    >     >     > [hidden email]
    >     >     > [hidden email]
    >     >     > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >     >     >
    >     >     > > On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
    >     >     > >
    >     >     > > Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
    >     >     > >
    >     >     > > Thanks
    >     >     > > Shawn
    >     >     > >
    >     >     > > Sent from my iPhone
    >     >     > >
    >     >     > >> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
    >     >     > >>
    >     >     > >> I see a couple choices here:
    >     >     > >>
    >     >     > >> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
    >     >     > >> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
    >     >     > >> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
    >     >     > >>
    >     >     > >> Andy LoPresto
    >     >     > >> [hidden email]
    >     >     > >> [hidden email]
    >     >     > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >     >     > >>
    >     >     > >>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
    >     >     > >>>
    >     >     > >>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
    >     >     > >>>
    >     >     > >>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
    >     >     > >>>
    >     >     > >>> Thanks
    >     >     > >>> Shawn
    >     >     > >>>
    >     >     > >>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
    >     >     > >>>
    >     >     > >>>  I meant to say that you obviously could generate certs for CLI users, but I
    >     >     > >>>  was just mentioning an alternative where you can proxy an identity.
    >     >     > >>>
    >     >     > >>>  Right now the CLI never obtains a token because it is all cert based.
    >     >     > >>>
    >     >     > >>>>  On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
    >     >     > >>>>
    >     >     > >>>> Right now the idea is that whoever is running the CLI would have access to
    >     >     > >>>> a NiFi server certificate and then you can proxy any user you want. There
    >     >     > >>>> should be examples of this in the readme or toolkit guide.
    >     >     > >>>>
    >     >     > >>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
    >     >     > >>>> not a trivial effort.
    >     >     > >>>>
    >     >     > >>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
    >     >     > >>>> wrote:
    >     >     > >>>>
    >     >     > >>>>> Shawn,
    >     >     > >>>>>
    >     >     > >>>>> I’m not sure I understand your question.
    >     >     > >>>>>
    >     >     > >>>>> I am in the process of refactoring the TLS Toolkit to integrate with
    >     >     > >>>>> public certificate authorities, so in the near future it will be easier to
    >     >     > >>>>> use certificates signed by external authorities rather than self-signed.
    >     >     > >>>>>
    >     >     > >>>>> My understanding is that you are talking about the CLI Toolkit rather
    >     >     > >>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
    >     >     > >>>>> going to proceed with the understanding that you are referring to the JWT
    >     >     > >>>>> token used to identify an authenticated user when communicating with the
    >     >     > >>>>> NiFi API.
    >     >     > >>>>>
    >     >     > >>>>> You may want to look at JerseyNiFiClient [1], which has methods for
    >     >     > >>>>> getting various clients given an authentication token.
    >     >     > >>>>>
    >     >     > >>>>> You can create the token via the POST /access/kerberos API [2].
    >     >     > >>>>>
    >     >     > >>>>> [1]
    >     >     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >     >     > >>>>> <
    >     >     > >>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >     >     > >>>>>>
    >     >     > >>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
    >     >     > >>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
    >     >     > >>>>>
    >     >     > >>>>> Andy LoPresto
    >     >     > >>>>> [hidden email]
    >     >     > >>>>> [hidden email]
    >     >     > >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >     >     > >>>>>
    >     >     > >>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
    >     >     > >>>>> wrote:
    >     >     > >>>>>>
    >     >     > >>>>>> I work in an environment reluctant to create self signed ssl
    >     >     > >>>>> certificates and I’m looking at the feasibility of having the toolkit cli
    >     >     > >>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
    >     >     > >>>>> another way to get the authentication token but I’m having trouble figuring
    >     >     > >>>>> out exactly when the token is created. I see lots of references to it after
    >     >     > >>>>> it’s been created.
    >     >     > >>>>>>
    >     >     > >>>>>> Thanks
    >     >     > >>>>>> Shawn
    >     >     > >>>>>
    >     >     > >>>>> --
    >     >     > >>>> Sent from Gmail Mobile
    >     >     > >>>>
    >     >     > >>>  --
    >     >     > >>>  Sent from Gmail Mobile
    >     >     > >>>
    >     >     > >>>
    >     >     > >>
    >     >     >
    >     >
    >     >
    >
    >
   

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Andy LoPresto-2
No, client cert authentication bypasses the JWT behavior completely. Because a client cert is automatically sent on every request, it makes no sense to delegate the credential to a token in that case.

Andy LoPresto
[hidden email]
[hidden email]
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 13, 2019, at 11:52 AM, Shawn Weeks <[hidden email]> wrote:
>
> Completely agree on username and password but I'll probably still do something somewhat generic around access tokens vs 2 way ssl as in the future there might be something else. On a related note is it possible to get a JWT with 2 way ssl? If so we could use the same auth method for everything.
>
> Thanks
> Shawn
>
> On 6/13/19, 1:36 PM, "Bryan Bende" <[hidden email]> wrote:
>
>    Ah thanks for pointing that out, I completely forgot that apparently I
>    was thinking ahead in the JerseyNiFiClient of how we could support
>    tokens :)
>
>    You would need to make the same changes in the
>    JerseyNiFiRegistryClient, and then build a new toolkit based on a new
>    version of nifi-registry-client.
>
>    Also, you are correct that we could support username/password, but I
>    think Kerberos is much better from a security perspective since you
>    don't really need to give your credentials to the CLI. With
>    username/password, you would either need to add those properties to
>    the .props files for the CLI, which then gets into encrypting the
>    password, or you need to provide them on a command as arguments which
>    again gets into whether the password is in plain text or not.
>
>    On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:
>>
>> Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
>>
>> It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
>>
>> https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
>>
>> Thanks
>> Shawn
>>
>> On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
>>
>>    I'm not sure if I confused things... the clients that I mentioned are
>>    wrappers for the REST API implemented with Jersey client, so the CLI
>>    does exclusively use the REST API.
>>
>>    I was just drawing attention to the clients to say that part of the
>>    work is outside of the CLI in nifi-registry-client to allow it to
>>    support kerberos auth.
>>
>>    On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
>>>
>>> Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
>>>
>>> Thanks
>>> Shawn
>>>
>>> On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
>>>
>>>    Just to further elaborate, within the CLI there are commands that work
>>>    against registry and commands that work against NiFi. For registry
>>>    commands, they use the Java client that is provided by registry [1].
>>>    For NiFi commands, there is mini client developed as need with in the
>>>    CLI [2].
>>>
>>>    None of these client calls currently have any concept of a JWT/token.
>>>
>>>    In order to do the kerberos auth correctly across both systems, I
>>>    think both of these clients would need to be updated to support a
>>>    method that called the /access/kerberos end point to obtain a token,
>>>    and then also provide a way to pass back that token on future
>>>    requests. It would likely be the CLI's job to store that token
>>>    somewhere (in memory for interactive shell, or on filesystem for
>>>    individual executions) and pass it back on each request. In order to
>>>    call the /access/kerberos end-point there also needs to be code in the
>>>    client that handles the negotiation to provide the kerberos
>>>    credentials that are present from having done a kinit.
>>>
>>>    Long story short, Andy's first suggest would be a much easier option
>>>    with no code changes.
>>>
>>>    [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
>>>    [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
>>>
>>>    On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
>>>>
>>>> You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
>>>>
>>>> Andy LoPresto
>>>> [hidden email]
>>>> [hidden email]
>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>
>>>>> On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
>>>>>
>>>>> Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
>>>>>
>>>>> Thanks
>>>>> Shawn
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>>> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
>>>>>>
>>>>>> I see a couple choices here:
>>>>>>
>>>>>> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
>>>>>> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
>>>>>> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
>>>>>>
>>>>>> Andy LoPresto
>>>>>> [hidden email]
>>>>>> [hidden email]
>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>>>
>>>>>>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>>>>>>>
>>>>>>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>>>>>>>
>>>>>>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Shawn
>>>>>>>
>>>>>>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>>>>>>>
>>>>>>> I meant to say that you obviously could generate certs for CLI users, but I
>>>>>>> was just mentioning an alternative where you can proxy an identity.
>>>>>>>
>>>>>>> Right now the CLI never obtains a token because it is all cert based.
>>>>>>>
>>>>>>>> On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> Right now the idea is that whoever is running the CLI would have access to
>>>>>>>> a NiFi server certificate and then you can proxy any user you want. There
>>>>>>>> should be examples of this in the readme or toolkit guide.
>>>>>>>>
>>>>>>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>>>>>>>> not a trivial effort.
>>>>>>>>
>>>>>>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Shawn,
>>>>>>>>>
>>>>>>>>> I’m not sure I understand your question.
>>>>>>>>>
>>>>>>>>> I am in the process of refactoring the TLS Toolkit to integrate with
>>>>>>>>> public certificate authorities, so in the near future it will be easier to
>>>>>>>>> use certificates signed by external authorities rather than self-signed.
>>>>>>>>>
>>>>>>>>> My understanding is that you are talking about the CLI Toolkit rather
>>>>>>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>>>>>>>>> going to proceed with the understanding that you are referring to the JWT
>>>>>>>>> token used to identify an authenticated user when communicating with the
>>>>>>>>> NiFi API.
>>>>>>>>>
>>>>>>>>> You may want to look at JerseyNiFiClient [1], which has methods for
>>>>>>>>> getting various clients given an authentication token.
>>>>>>>>>
>>>>>>>>> You can create the token via the POST /access/kerberos API [2].
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>>>>>> <
>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>>>>>>>
>>>>>>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>>>>>>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>>>>>>>>>
>>>>>>>>> Andy LoPresto
>>>>>>>>> [hidden email]
>>>>>>>>> [hidden email]
>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>>>>>>
>>>>>>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I work in an environment reluctant to create self signed ssl
>>>>>>>>> certificates and I’m looking at the feasibility of having the toolkit cli
>>>>>>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>>>>>>>>> another way to get the authentication token but I’m having trouble figuring
>>>>>>>>> out exactly when the token is created. I see lots of references to it after
>>>>>>>>> it’s been created.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> Shawn
>>>>>>>>>
>>>>>>>>> --
>>>>>>>> Sent from Gmail Mobile
>>>>>>>>
>>>>>>> --
>>>>>>> Sent from Gmail Mobile
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>>
>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Bryan Bende
It would be a little bit weird because you'd still need the client
cert for the initial request to get the JWT, so then in that case why
not just keep using the client cert.

Registry does things a little bit different than NiFi and has a few
variations of the token end-point:

/access/token/login (looks for credentials using basic auth)
/access/token/kerberos (same as NiFi)
/access/token/identity-provider (passes request to the configured
identity provider)
/access/token (tries all identity providers in order, the first of
which is X509 identity provider)

So if you sent a client cert to the last one, it would do what you are
suggesting.

On Thu, Jun 13, 2019 at 2:55 PM Andy LoPresto <[hidden email]> wrote:

>
> No, client cert authentication bypasses the JWT behavior completely. Because a client cert is automatically sent on every request, it makes no sense to delegate the credential to a token in that case.
>
> Andy LoPresto
> [hidden email]
> [hidden email]
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> > On Jun 13, 2019, at 11:52 AM, Shawn Weeks <[hidden email]> wrote:
> >
> > Completely agree on username and password but I'll probably still do something somewhat generic around access tokens vs 2 way ssl as in the future there might be something else. On a related note is it possible to get a JWT with 2 way ssl? If so we could use the same auth method for everything.
> >
> > Thanks
> > Shawn
> >
> > On 6/13/19, 1:36 PM, "Bryan Bende" <[hidden email]> wrote:
> >
> >    Ah thanks for pointing that out, I completely forgot that apparently I
> >    was thinking ahead in the JerseyNiFiClient of how we could support
> >    tokens :)
> >
> >    You would need to make the same changes in the
> >    JerseyNiFiRegistryClient, and then build a new toolkit based on a new
> >    version of nifi-registry-client.
> >
> >    Also, you are correct that we could support username/password, but I
> >    think Kerberos is much better from a security perspective since you
> >    don't really need to give your credentials to the CLI. With
> >    username/password, you would either need to add those properties to
> >    the .props files for the CLI, which then gets into encrypting the
> >    password, or you need to provide them on a command as arguments which
> >    again gets into whether the password is in plain text or not.
> >
> >    On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:
> >>
> >> Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
> >>
> >> It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
> >>
> >> https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
> >>
> >> Thanks
> >> Shawn
> >>
> >> On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
> >>
> >>    I'm not sure if I confused things... the clients that I mentioned are
> >>    wrappers for the REST API implemented with Jersey client, so the CLI
> >>    does exclusively use the REST API.
> >>
> >>    I was just drawing attention to the clients to say that part of the
> >>    work is outside of the CLI in nifi-registry-client to allow it to
> >>    support kerberos auth.
> >>
> >>    On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
> >>>
> >>> Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
> >>>
> >>> Thanks
> >>> Shawn
> >>>
> >>> On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
> >>>
> >>>    Just to further elaborate, within the CLI there are commands that work
> >>>    against registry and commands that work against NiFi. For registry
> >>>    commands, they use the Java client that is provided by registry [1].
> >>>    For NiFi commands, there is mini client developed as need with in the
> >>>    CLI [2].
> >>>
> >>>    None of these client calls currently have any concept of a JWT/token.
> >>>
> >>>    In order to do the kerberos auth correctly across both systems, I
> >>>    think both of these clients would need to be updated to support a
> >>>    method that called the /access/kerberos end point to obtain a token,
> >>>    and then also provide a way to pass back that token on future
> >>>    requests. It would likely be the CLI's job to store that token
> >>>    somewhere (in memory for interactive shell, or on filesystem for
> >>>    individual executions) and pass it back on each request. In order to
> >>>    call the /access/kerberos end-point there also needs to be code in the
> >>>    client that handles the negotiation to provide the kerberos
> >>>    credentials that are present from having done a kinit.
> >>>
> >>>    Long story short, Andy's first suggest would be a much easier option
> >>>    with no code changes.
> >>>
> >>>    [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
> >>>    [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
> >>>
> >>>    On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
> >>>>
> >>>> You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
> >>>>
> >>>> Andy LoPresto
> >>>> [hidden email]
> >>>> [hidden email]
> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>
> >>>>> On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
> >>>>>
> >>>>> Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
> >>>>>
> >>>>> Thanks
> >>>>> Shawn
> >>>>>
> >>>>> Sent from my iPhone
> >>>>>
> >>>>>> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
> >>>>>>
> >>>>>> I see a couple choices here:
> >>>>>>
> >>>>>> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
> >>>>>> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
> >>>>>> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
> >>>>>>
> >>>>>> Andy LoPresto
> >>>>>> [hidden email]
> >>>>>> [hidden email]
> >>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>>>
> >>>>>>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
> >>>>>>>
> >>>>>>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
> >>>>>>>
> >>>>>>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
> >>>>>>>
> >>>>>>> Thanks
> >>>>>>> Shawn
> >>>>>>>
> >>>>>>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
> >>>>>>>
> >>>>>>> I meant to say that you obviously could generate certs for CLI users, but I
> >>>>>>> was just mentioning an alternative where you can proxy an identity.
> >>>>>>>
> >>>>>>> Right now the CLI never obtains a token because it is all cert based.
> >>>>>>>
> >>>>>>>> On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
> >>>>>>>>
> >>>>>>>> Right now the idea is that whoever is running the CLI would have access to
> >>>>>>>> a NiFi server certificate and then you can proxy any user you want. There
> >>>>>>>> should be examples of this in the readme or toolkit guide.
> >>>>>>>>
> >>>>>>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
> >>>>>>>> not a trivial effort.
> >>>>>>>>
> >>>>>>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Shawn,
> >>>>>>>>>
> >>>>>>>>> I’m not sure I understand your question.
> >>>>>>>>>
> >>>>>>>>> I am in the process of refactoring the TLS Toolkit to integrate with
> >>>>>>>>> public certificate authorities, so in the near future it will be easier to
> >>>>>>>>> use certificates signed by external authorities rather than self-signed.
> >>>>>>>>>
> >>>>>>>>> My understanding is that you are talking about the CLI Toolkit rather
> >>>>>>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
> >>>>>>>>> going to proceed with the understanding that you are referring to the JWT
> >>>>>>>>> token used to identify an authenticated user when communicating with the
> >>>>>>>>> NiFi API.
> >>>>>>>>>
> >>>>>>>>> You may want to look at JerseyNiFiClient [1], which has methods for
> >>>>>>>>> getting various clients given an authentication token.
> >>>>>>>>>
> >>>>>>>>> You can create the token via the POST /access/kerberos API [2].
> >>>>>>>>>
> >>>>>>>>> [1]
> >>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> >>>>>>>>> <
> >>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
> >>>>>>>>>>
> >>>>>>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
> >>>>>>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
> >>>>>>>>>
> >>>>>>>>> Andy LoPresto
> >>>>>>>>> [hidden email]
> >>>>>>>>> [hidden email]
> >>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
> >>>>>>>>>
> >>>>>>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I work in an environment reluctant to create self signed ssl
> >>>>>>>>> certificates and I’m looking at the feasibility of having the toolkit cli
> >>>>>>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
> >>>>>>>>> another way to get the authentication token but I’m having trouble figuring
> >>>>>>>>> out exactly when the token is created. I see lots of references to it after
> >>>>>>>>> it’s been created.
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>> Shawn
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>> Sent from Gmail Mobile
> >>>>>>>>
> >>>>>>> --
> >>>>>>> Sent from Gmail Mobile
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>>
> >>
> >>
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Shawn Weeks
Got it, for now I'm just going to work on implementing a Kerberos solution that either allows you to configure a keytab and principal or pulls from the current subject if your already logged in.

I created NIFI-6378 and NIFIREG-281. Can one of you assign the registry one to me as I'm not a contributor there yet.

Thanks
Shawn

On 6/13/19, 2:03 PM, "Bryan Bende" <[hidden email]> wrote:

    It would be a little bit weird because you'd still need the client
    cert for the initial request to get the JWT, so then in that case why
    not just keep using the client cert.
   
    Registry does things a little bit different than NiFi and has a few
    variations of the token end-point:
   
    /access/token/login (looks for credentials using basic auth)
    /access/token/kerberos (same as NiFi)
    /access/token/identity-provider (passes request to the configured
    identity provider)
    /access/token (tries all identity providers in order, the first of
    which is X509 identity provider)
   
    So if you sent a client cert to the last one, it would do what you are
    suggesting.
   
    On Thu, Jun 13, 2019 at 2:55 PM Andy LoPresto <[hidden email]> wrote:
    >
    > No, client cert authentication bypasses the JWT behavior completely. Because a client cert is automatically sent on every request, it makes no sense to delegate the credential to a token in that case.
    >
    > Andy LoPresto
    > [hidden email]
    > [hidden email]
    > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >
    > > On Jun 13, 2019, at 11:52 AM, Shawn Weeks <[hidden email]> wrote:
    > >
    > > Completely agree on username and password but I'll probably still do something somewhat generic around access tokens vs 2 way ssl as in the future there might be something else. On a related note is it possible to get a JWT with 2 way ssl? If so we could use the same auth method for everything.
    > >
    > > Thanks
    > > Shawn
    > >
    > > On 6/13/19, 1:36 PM, "Bryan Bende" <[hidden email]> wrote:
    > >
    > >    Ah thanks for pointing that out, I completely forgot that apparently I
    > >    was thinking ahead in the JerseyNiFiClient of how we could support
    > >    tokens :)
    > >
    > >    You would need to make the same changes in the
    > >    JerseyNiFiRegistryClient, and then build a new toolkit based on a new
    > >    version of nifi-registry-client.
    > >
    > >    Also, you are correct that we could support username/password, but I
    > >    think Kerberos is much better from a security perspective since you
    > >    don't really need to give your credentials to the CLI. With
    > >    username/password, you would either need to add those properties to
    > >    the .props files for the CLI, which then gets into encrypting the
    > >    password, or you need to provide them on a command as arguments which
    > >    again gets into whether the password is in plain text or not.
    > >
    > >    On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:
    > >>
    > >> Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
    > >>
    > >> It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
    > >>
    > >> https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
    > >>
    > >> Thanks
    > >> Shawn
    > >>
    > >> On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
    > >>
    > >>    I'm not sure if I confused things... the clients that I mentioned are
    > >>    wrappers for the REST API implemented with Jersey client, so the CLI
    > >>    does exclusively use the REST API.
    > >>
    > >>    I was just drawing attention to the clients to say that part of the
    > >>    work is outside of the CLI in nifi-registry-client to allow it to
    > >>    support kerberos auth.
    > >>
    > >>    On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
    > >>>
    > >>> Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
    > >>>
    > >>> Thanks
    > >>> Shawn
    > >>>
    > >>> On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
    > >>>
    > >>>    Just to further elaborate, within the CLI there are commands that work
    > >>>    against registry and commands that work against NiFi. For registry
    > >>>    commands, they use the Java client that is provided by registry [1].
    > >>>    For NiFi commands, there is mini client developed as need with in the
    > >>>    CLI [2].
    > >>>
    > >>>    None of these client calls currently have any concept of a JWT/token.
    > >>>
    > >>>    In order to do the kerberos auth correctly across both systems, I
    > >>>    think both of these clients would need to be updated to support a
    > >>>    method that called the /access/kerberos end point to obtain a token,
    > >>>    and then also provide a way to pass back that token on future
    > >>>    requests. It would likely be the CLI's job to store that token
    > >>>    somewhere (in memory for interactive shell, or on filesystem for
    > >>>    individual executions) and pass it back on each request. In order to
    > >>>    call the /access/kerberos end-point there also needs to be code in the
    > >>>    client that handles the negotiation to provide the kerberos
    > >>>    credentials that are present from having done a kinit.
    > >>>
    > >>>    Long story short, Andy's first suggest would be a much easier option
    > >>>    with no code changes.
    > >>>
    > >>>    [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
    > >>>    [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
    > >>>
    > >>>    On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
    > >>>>
    > >>>> You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
    > >>>>
    > >>>> Andy LoPresto
    > >>>> [hidden email]
    > >>>> [hidden email]
    > >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    > >>>>
    > >>>>> On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
    > >>>>>
    > >>>>> Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
    > >>>>>
    > >>>>> Thanks
    > >>>>> Shawn
    > >>>>>
    > >>>>> Sent from my iPhone
    > >>>>>
    > >>>>>> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
    > >>>>>>
    > >>>>>> I see a couple choices here:
    > >>>>>>
    > >>>>>> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
    > >>>>>> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
    > >>>>>> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
    > >>>>>>
    > >>>>>> Andy LoPresto
    > >>>>>> [hidden email]
    > >>>>>> [hidden email]
    > >>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    > >>>>>>
    > >>>>>>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
    > >>>>>>>
    > >>>>>>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
    > >>>>>>>
    > >>>>>>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
    > >>>>>>>
    > >>>>>>> Thanks
    > >>>>>>> Shawn
    > >>>>>>>
    > >>>>>>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
    > >>>>>>>
    > >>>>>>> I meant to say that you obviously could generate certs for CLI users, but I
    > >>>>>>> was just mentioning an alternative where you can proxy an identity.
    > >>>>>>>
    > >>>>>>> Right now the CLI never obtains a token because it is all cert based.
    > >>>>>>>
    > >>>>>>>> On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
    > >>>>>>>>
    > >>>>>>>> Right now the idea is that whoever is running the CLI would have access to
    > >>>>>>>> a NiFi server certificate and then you can proxy any user you want. There
    > >>>>>>>> should be examples of this in the readme or toolkit guide.
    > >>>>>>>>
    > >>>>>>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
    > >>>>>>>> not a trivial effort.
    > >>>>>>>>
    > >>>>>>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
    > >>>>>>>> wrote:
    > >>>>>>>>
    > >>>>>>>>> Shawn,
    > >>>>>>>>>
    > >>>>>>>>> I’m not sure I understand your question.
    > >>>>>>>>>
    > >>>>>>>>> I am in the process of refactoring the TLS Toolkit to integrate with
    > >>>>>>>>> public certificate authorities, so in the near future it will be easier to
    > >>>>>>>>> use certificates signed by external authorities rather than self-signed.
    > >>>>>>>>>
    > >>>>>>>>> My understanding is that you are talking about the CLI Toolkit rather
    > >>>>>>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
    > >>>>>>>>> going to proceed with the understanding that you are referring to the JWT
    > >>>>>>>>> token used to identify an authenticated user when communicating with the
    > >>>>>>>>> NiFi API.
    > >>>>>>>>>
    > >>>>>>>>> You may want to look at JerseyNiFiClient [1], which has methods for
    > >>>>>>>>> getting various clients given an authentication token.
    > >>>>>>>>>
    > >>>>>>>>> You can create the token via the POST /access/kerberos API [2].
    > >>>>>>>>>
    > >>>>>>>>> [1]
    > >>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    > >>>>>>>>> <
    > >>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    > >>>>>>>>>>
    > >>>>>>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
    > >>>>>>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
    > >>>>>>>>>
    > >>>>>>>>> Andy LoPresto
    > >>>>>>>>> [hidden email]
    > >>>>>>>>> [hidden email]
    > >>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    > >>>>>>>>>
    > >>>>>>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
    > >>>>>>>>> wrote:
    > >>>>>>>>>>
    > >>>>>>>>>> I work in an environment reluctant to create self signed ssl
    > >>>>>>>>> certificates and I’m looking at the feasibility of having the toolkit cli
    > >>>>>>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
    > >>>>>>>>> another way to get the authentication token but I’m having trouble figuring
    > >>>>>>>>> out exactly when the token is created. I see lots of references to it after
    > >>>>>>>>> it’s been created.
    > >>>>>>>>>>
    > >>>>>>>>>> Thanks
    > >>>>>>>>>> Shawn
    > >>>>>>>>>
    > >>>>>>>>> --
    > >>>>>>>> Sent from Gmail Mobile
    > >>>>>>>>
    > >>>>>>> --
    > >>>>>>> Sent from Gmail Mobile
    > >>>>>>>
    > >>>>>>>
    > >>>>>>
    > >>>>
    > >>>
    > >>>
    > >>
    > >>
    > >
    > >
    >
   

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Andy LoPresto-2
Assigned you that Jira and added you to the contributors role so you can do the same in the future. Thanks.

Andy LoPresto
[hidden email]
[hidden email]
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Jun 13, 2019, at 12:11 PM, Shawn Weeks <[hidden email]> wrote:
>
> Got it, for now I'm just going to work on implementing a Kerberos solution that either allows you to configure a keytab and principal or pulls from the current subject if your already logged in.
>
> I created NIFI-6378 and NIFIREG-281. Can one of you assign the registry one to me as I'm not a contributor there yet.
>
> Thanks
> Shawn
>
> On 6/13/19, 2:03 PM, "Bryan Bende" <[hidden email]> wrote:
>
>    It would be a little bit weird because you'd still need the client
>    cert for the initial request to get the JWT, so then in that case why
>    not just keep using the client cert.
>
>    Registry does things a little bit different than NiFi and has a few
>    variations of the token end-point:
>
>    /access/token/login (looks for credentials using basic auth)
>    /access/token/kerberos (same as NiFi)
>    /access/token/identity-provider (passes request to the configured
>    identity provider)
>    /access/token (tries all identity providers in order, the first of
>    which is X509 identity provider)
>
>    So if you sent a client cert to the last one, it would do what you are
>    suggesting.
>
>    On Thu, Jun 13, 2019 at 2:55 PM Andy LoPresto <[hidden email]> wrote:
>>
>> No, client cert authentication bypasses the JWT behavior completely. Because a client cert is automatically sent on every request, it makes no sense to delegate the credential to a token in that case.
>>
>> Andy LoPresto
>> [hidden email]
>> [hidden email]
>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>
>>> On Jun 13, 2019, at 11:52 AM, Shawn Weeks <[hidden email]> wrote:
>>>
>>> Completely agree on username and password but I'll probably still do something somewhat generic around access tokens vs 2 way ssl as in the future there might be something else. On a related note is it possible to get a JWT with 2 way ssl? If so we could use the same auth method for everything.
>>>
>>> Thanks
>>> Shawn
>>>
>>> On 6/13/19, 1:36 PM, "Bryan Bende" <[hidden email]> wrote:
>>>
>>>   Ah thanks for pointing that out, I completely forgot that apparently I
>>>   was thinking ahead in the JerseyNiFiClient of how we could support
>>>   tokens :)
>>>
>>>   You would need to make the same changes in the
>>>   JerseyNiFiRegistryClient, and then build a new toolkit based on a new
>>>   version of nifi-registry-client.
>>>
>>>   Also, you are correct that we could support username/password, but I
>>>   think Kerberos is much better from a security perspective since you
>>>   don't really need to give your credentials to the CLI. With
>>>   username/password, you would either need to add those properties to
>>>   the .props files for the CLI, which then gets into encrypting the
>>>   password, or you need to provide them on a command as arguments which
>>>   again gets into whether the password is in plain text or not.
>>>
>>>   On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:
>>>>
>>>> Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
>>>>
>>>> It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
>>>>
>>>> https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
>>>>
>>>> Thanks
>>>> Shawn
>>>>
>>>> On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
>>>>
>>>>   I'm not sure if I confused things... the clients that I mentioned are
>>>>   wrappers for the REST API implemented with Jersey client, so the CLI
>>>>   does exclusively use the REST API.
>>>>
>>>>   I was just drawing attention to the clients to say that part of the
>>>>   work is outside of the CLI in nifi-registry-client to allow it to
>>>>   support kerberos auth.
>>>>
>>>>   On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
>>>>>
>>>>> Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
>>>>>
>>>>> Thanks
>>>>> Shawn
>>>>>
>>>>> On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
>>>>>
>>>>>   Just to further elaborate, within the CLI there are commands that work
>>>>>   against registry and commands that work against NiFi. For registry
>>>>>   commands, they use the Java client that is provided by registry [1].
>>>>>   For NiFi commands, there is mini client developed as need with in the
>>>>>   CLI [2].
>>>>>
>>>>>   None of these client calls currently have any concept of a JWT/token.
>>>>>
>>>>>   In order to do the kerberos auth correctly across both systems, I
>>>>>   think both of these clients would need to be updated to support a
>>>>>   method that called the /access/kerberos end point to obtain a token,
>>>>>   and then also provide a way to pass back that token on future
>>>>>   requests. It would likely be the CLI's job to store that token
>>>>>   somewhere (in memory for interactive shell, or on filesystem for
>>>>>   individual executions) and pass it back on each request. In order to
>>>>>   call the /access/kerberos end-point there also needs to be code in the
>>>>>   client that handles the negotiation to provide the kerberos
>>>>>   credentials that are present from having done a kinit.
>>>>>
>>>>>   Long story short, Andy's first suggest would be a much easier option
>>>>>   with no code changes.
>>>>>
>>>>>   [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
>>>>>   [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
>>>>>
>>>>>   On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
>>>>>>
>>>>>> You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
>>>>>>
>>>>>> Andy LoPresto
>>>>>> [hidden email]
>>>>>> [hidden email]
>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>>>
>>>>>>> On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
>>>>>>>
>>>>>>> Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
>>>>>>>
>>>>>>> Thanks
>>>>>>> Shawn
>>>>>>>
>>>>>>> Sent from my iPhone
>>>>>>>
>>>>>>>> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
>>>>>>>>
>>>>>>>> I see a couple choices here:
>>>>>>>>
>>>>>>>> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
>>>>>>>> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
>>>>>>>> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
>>>>>>>>
>>>>>>>> Andy LoPresto
>>>>>>>> [hidden email]
>>>>>>>> [hidden email]
>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>>>>>
>>>>>>>>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>>>>>>>>>
>>>>>>>>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Shawn
>>>>>>>>>
>>>>>>>>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>>>>>>>>>
>>>>>>>>> I meant to say that you obviously could generate certs for CLI users, but I
>>>>>>>>> was just mentioning an alternative where you can proxy an identity.
>>>>>>>>>
>>>>>>>>> Right now the CLI never obtains a token because it is all cert based.
>>>>>>>>>
>>>>>>>>>> On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>>>>>>>>>>
>>>>>>>>>> Right now the idea is that whoever is running the CLI would have access to
>>>>>>>>>> a NiFi server certificate and then you can proxy any user you want. There
>>>>>>>>>> should be examples of this in the readme or toolkit guide.
>>>>>>>>>>
>>>>>>>>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>>>>>>>>>> not a trivial effort.
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Shawn,
>>>>>>>>>>>
>>>>>>>>>>> I’m not sure I understand your question.
>>>>>>>>>>>
>>>>>>>>>>> I am in the process of refactoring the TLS Toolkit to integrate with
>>>>>>>>>>> public certificate authorities, so in the near future it will be easier to
>>>>>>>>>>> use certificates signed by external authorities rather than self-signed.
>>>>>>>>>>>
>>>>>>>>>>> My understanding is that you are talking about the CLI Toolkit rather
>>>>>>>>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>>>>>>>>>>> going to proceed with the understanding that you are referring to the JWT
>>>>>>>>>>> token used to identify an authenticated user when communicating with the
>>>>>>>>>>> NiFi API.
>>>>>>>>>>>
>>>>>>>>>>> You may want to look at JerseyNiFiClient [1], which has methods for
>>>>>>>>>>> getting various clients given an authentication token.
>>>>>>>>>>>
>>>>>>>>>>> You can create the token via the POST /access/kerberos API [2].
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>>>>>>>> <
>>>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>>>>>>>>>>>>
>>>>>>>>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>>>>>>>>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>>>>>>>>>>>
>>>>>>>>>>> Andy LoPresto
>>>>>>>>>>> [hidden email]
>>>>>>>>>>> [hidden email]
>>>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>>>>>>>>>>>
>>>>>>>>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I work in an environment reluctant to create self signed ssl
>>>>>>>>>>> certificates and I’m looking at the feasibility of having the toolkit cli
>>>>>>>>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>>>>>>>>>>> another way to get the authentication token but I’m having trouble figuring
>>>>>>>>>>> out exactly when the token is created. I see lots of references to it after
>>>>>>>>>>> it’s been created.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>> Shawn
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>> Sent from Gmail Mobile
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sent from Gmail Mobile
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Shawn Weeks
Turns out there isn't going to be an easy way like I thought. Looking at everything it seems to me that you can generalize all the different jersey clients as with headers or without headers. Currently they all support this as a way to implement proxies that the Client Factory wraps things at creation time. Obviously I don't want to break a number of folks work and I was wondering if it's likely that external projects are relying on the NiFiRegistryClient.java and NiFiClient.java interfaces.

I was thinking about adding with header methods to them and with that the proxied entity versions could eventually be swapped over since proxy is just another header. Thoughts? Horrible Idea?

Thanks
Shawn

On 6/13/19, 2:15 PM, "Andy LoPresto" <[hidden email]> wrote:

    Assigned you that Jira and added you to the contributors role so you can do the same in the future. Thanks.
   
    Andy LoPresto
    [hidden email]
    [hidden email]
    PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
   
    > On Jun 13, 2019, at 12:11 PM, Shawn Weeks <[hidden email]> wrote:
    >
    > Got it, for now I'm just going to work on implementing a Kerberos solution that either allows you to configure a keytab and principal or pulls from the current subject if your already logged in.
    >
    > I created NIFI-6378 and NIFIREG-281. Can one of you assign the registry one to me as I'm not a contributor there yet.
    >
    > Thanks
    > Shawn
    >
    > On 6/13/19, 2:03 PM, "Bryan Bende" <[hidden email]> wrote:
    >
    >    It would be a little bit weird because you'd still need the client
    >    cert for the initial request to get the JWT, so then in that case why
    >    not just keep using the client cert.
    >
    >    Registry does things a little bit different than NiFi and has a few
    >    variations of the token end-point:
    >
    >    /access/token/login (looks for credentials using basic auth)
    >    /access/token/kerberos (same as NiFi)
    >    /access/token/identity-provider (passes request to the configured
    >    identity provider)
    >    /access/token (tries all identity providers in order, the first of
    >    which is X509 identity provider)
    >
    >    So if you sent a client cert to the last one, it would do what you are
    >    suggesting.
    >
    >    On Thu, Jun 13, 2019 at 2:55 PM Andy LoPresto <[hidden email]> wrote:
    >>
    >> No, client cert authentication bypasses the JWT behavior completely. Because a client cert is automatically sent on every request, it makes no sense to delegate the credential to a token in that case.
    >>
    >> Andy LoPresto
    >> [hidden email]
    >> [hidden email]
    >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >>
    >>> On Jun 13, 2019, at 11:52 AM, Shawn Weeks <[hidden email]> wrote:
    >>>
    >>> Completely agree on username and password but I'll probably still do something somewhat generic around access tokens vs 2 way ssl as in the future there might be something else. On a related note is it possible to get a JWT with 2 way ssl? If so we could use the same auth method for everything.
    >>>
    >>> Thanks
    >>> Shawn
    >>>
    >>> On 6/13/19, 1:36 PM, "Bryan Bende" <[hidden email]> wrote:
    >>>
    >>>   Ah thanks for pointing that out, I completely forgot that apparently I
    >>>   was thinking ahead in the JerseyNiFiClient of how we could support
    >>>   tokens :)
    >>>
    >>>   You would need to make the same changes in the
    >>>   JerseyNiFiRegistryClient, and then build a new toolkit based on a new
    >>>   version of nifi-registry-client.
    >>>
    >>>   Also, you are correct that we could support username/password, but I
    >>>   think Kerberos is much better from a security perspective since you
    >>>   don't really need to give your credentials to the CLI. With
    >>>   username/password, you would either need to add those properties to
    >>>   the .props files for the CLI, which then gets into encrypting the
    >>>   password, or you need to provide them on a command as arguments which
    >>>   again gets into whether the password is in plain text or not.
    >>>
    >>>   On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:
    >>>>
    >>>> Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
    >>>>
    >>>> It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
    >>>>
    >>>> https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
    >>>>
    >>>> Thanks
    >>>> Shawn
    >>>>
    >>>> On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
    >>>>
    >>>>   I'm not sure if I confused things... the clients that I mentioned are
    >>>>   wrappers for the REST API implemented with Jersey client, so the CLI
    >>>>   does exclusively use the REST API.
    >>>>
    >>>>   I was just drawing attention to the clients to say that part of the
    >>>>   work is outside of the CLI in nifi-registry-client to allow it to
    >>>>   support kerberos auth.
    >>>>
    >>>>   On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
    >>>>>
    >>>>> Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
    >>>>>
    >>>>> Thanks
    >>>>> Shawn
    >>>>>
    >>>>> On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
    >>>>>
    >>>>>   Just to further elaborate, within the CLI there are commands that work
    >>>>>   against registry and commands that work against NiFi. For registry
    >>>>>   commands, they use the Java client that is provided by registry [1].
    >>>>>   For NiFi commands, there is mini client developed as need with in the
    >>>>>   CLI [2].
    >>>>>
    >>>>>   None of these client calls currently have any concept of a JWT/token.
    >>>>>
    >>>>>   In order to do the kerberos auth correctly across both systems, I
    >>>>>   think both of these clients would need to be updated to support a
    >>>>>   method that called the /access/kerberos end point to obtain a token,
    >>>>>   and then also provide a way to pass back that token on future
    >>>>>   requests. It would likely be the CLI's job to store that token
    >>>>>   somewhere (in memory for interactive shell, or on filesystem for
    >>>>>   individual executions) and pass it back on each request. In order to
    >>>>>   call the /access/kerberos end-point there also needs to be code in the
    >>>>>   client that handles the negotiation to provide the kerberos
    >>>>>   credentials that are present from having done a kinit.
    >>>>>
    >>>>>   Long story short, Andy's first suggest would be a much easier option
    >>>>>   with no code changes.
    >>>>>
    >>>>>   [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
    >>>>>   [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
    >>>>>
    >>>>>   On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
    >>>>>>
    >>>>>> You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
    >>>>>>
    >>>>>> Andy LoPresto
    >>>>>> [hidden email]
    >>>>>> [hidden email]
    >>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >>>>>>
    >>>>>>> On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
    >>>>>>>
    >>>>>>> Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
    >>>>>>>
    >>>>>>> Thanks
    >>>>>>> Shawn
    >>>>>>>
    >>>>>>> Sent from my iPhone
    >>>>>>>
    >>>>>>>> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
    >>>>>>>>
    >>>>>>>> I see a couple choices here:
    >>>>>>>>
    >>>>>>>> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
    >>>>>>>> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
    >>>>>>>> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
    >>>>>>>>
    >>>>>>>> Andy LoPresto
    >>>>>>>> [hidden email]
    >>>>>>>> [hidden email]
    >>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >>>>>>>>
    >>>>>>>>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
    >>>>>>>>>
    >>>>>>>>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
    >>>>>>>>>
    >>>>>>>>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
    >>>>>>>>>
    >>>>>>>>> Thanks
    >>>>>>>>> Shawn
    >>>>>>>>>
    >>>>>>>>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
    >>>>>>>>>
    >>>>>>>>> I meant to say that you obviously could generate certs for CLI users, but I
    >>>>>>>>> was just mentioning an alternative where you can proxy an identity.
    >>>>>>>>>
    >>>>>>>>> Right now the CLI never obtains a token because it is all cert based.
    >>>>>>>>>
    >>>>>>>>>> On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
    >>>>>>>>>>
    >>>>>>>>>> Right now the idea is that whoever is running the CLI would have access to
    >>>>>>>>>> a NiFi server certificate and then you can proxy any user you want. There
    >>>>>>>>>> should be examples of this in the readme or toolkit guide.
    >>>>>>>>>>
    >>>>>>>>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
    >>>>>>>>>> not a trivial effort.
    >>>>>>>>>>
    >>>>>>>>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
    >>>>>>>>>> wrote:
    >>>>>>>>>>
    >>>>>>>>>>> Shawn,
    >>>>>>>>>>>
    >>>>>>>>>>> I’m not sure I understand your question.
    >>>>>>>>>>>
    >>>>>>>>>>> I am in the process of refactoring the TLS Toolkit to integrate with
    >>>>>>>>>>> public certificate authorities, so in the near future it will be easier to
    >>>>>>>>>>> use certificates signed by external authorities rather than self-signed.
    >>>>>>>>>>>
    >>>>>>>>>>> My understanding is that you are talking about the CLI Toolkit rather
    >>>>>>>>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
    >>>>>>>>>>> going to proceed with the understanding that you are referring to the JWT
    >>>>>>>>>>> token used to identify an authenticated user when communicating with the
    >>>>>>>>>>> NiFi API.
    >>>>>>>>>>>
    >>>>>>>>>>> You may want to look at JerseyNiFiClient [1], which has methods for
    >>>>>>>>>>> getting various clients given an authentication token.
    >>>>>>>>>>>
    >>>>>>>>>>> You can create the token via the POST /access/kerberos API [2].
    >>>>>>>>>>>
    >>>>>>>>>>> [1]
    >>>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >>>>>>>>>>> <
    >>>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
    >>>>>>>>>>>>
    >>>>>>>>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
    >>>>>>>>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
    >>>>>>>>>>>
    >>>>>>>>>>> Andy LoPresto
    >>>>>>>>>>> [hidden email]
    >>>>>>>>>>> [hidden email]
    >>>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
    >>>>>>>>>>>
    >>>>>>>>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
    >>>>>>>>>>> wrote:
    >>>>>>>>>>>>
    >>>>>>>>>>>> I work in an environment reluctant to create self signed ssl
    >>>>>>>>>>> certificates and I’m looking at the feasibility of having the toolkit cli
    >>>>>>>>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
    >>>>>>>>>>> another way to get the authentication token but I’m having trouble figuring
    >>>>>>>>>>> out exactly when the token is created. I see lots of references to it after
    >>>>>>>>>>> it’s been created.
    >>>>>>>>>>>>
    >>>>>>>>>>>> Thanks
    >>>>>>>>>>>> Shawn
    >>>>>>>>>>>
    >>>>>>>>>>> --
    >>>>>>>>>> Sent from Gmail Mobile
    >>>>>>>>>>
    >>>>>>>>> --
    >>>>>>>>> Sent from Gmail Mobile
    >>>>>>>>>
    >>>>>>>>>
    >>>>>>>>
    >>>>>>
    >>>>>
    >>>>>
    >>>>
    >>>>
    >>>
    >>>
    >>
    >
    >
   
   

Reply | Threaded
Open this post in threaded view
|

Re: NiFi Toolkit CLI Token Creation

Bryan Bende
I'm not sure I totally follow... is there a reason that the token
methods in NiFIClient can't be used?

I would expect that it is the CLI's decision whether to call
getXYZForProxiedEntities or getXYZForToken.

The client factories in CLI currently wrap the client and force the
use of proxied entities because that is all that was supported, so
that would have to be changed to do something different when the CLI
knows it using a token.

On Fri, Jun 14, 2019 at 11:26 AM Shawn Weeks <[hidden email]> wrote:

>
> Turns out there isn't going to be an easy way like I thought. Looking at everything it seems to me that you can generalize all the different jersey clients as with headers or without headers. Currently they all support this as a way to implement proxies that the Client Factory wraps things at creation time. Obviously I don't want to break a number of folks work and I was wondering if it's likely that external projects are relying on the NiFiRegistryClient.java and NiFiClient.java interfaces.
>
> I was thinking about adding with header methods to them and with that the proxied entity versions could eventually be swapped over since proxy is just another header. Thoughts? Horrible Idea?
>
> Thanks
> Shawn
>
> On 6/13/19, 2:15 PM, "Andy LoPresto" <[hidden email]> wrote:
>
>     Assigned you that Jira and added you to the contributors role so you can do the same in the future. Thanks.
>
>     Andy LoPresto
>     [hidden email]
>     [hidden email]
>     PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
>     > On Jun 13, 2019, at 12:11 PM, Shawn Weeks <[hidden email]> wrote:
>     >
>     > Got it, for now I'm just going to work on implementing a Kerberos solution that either allows you to configure a keytab and principal or pulls from the current subject if your already logged in.
>     >
>     > I created NIFI-6378 and NIFIREG-281. Can one of you assign the registry one to me as I'm not a contributor there yet.
>     >
>     > Thanks
>     > Shawn
>     >
>     > On 6/13/19, 2:03 PM, "Bryan Bende" <[hidden email]> wrote:
>     >
>     >    It would be a little bit weird because you'd still need the client
>     >    cert for the initial request to get the JWT, so then in that case why
>     >    not just keep using the client cert.
>     >
>     >    Registry does things a little bit different than NiFi and has a few
>     >    variations of the token end-point:
>     >
>     >    /access/token/login (looks for credentials using basic auth)
>     >    /access/token/kerberos (same as NiFi)
>     >    /access/token/identity-provider (passes request to the configured
>     >    identity provider)
>     >    /access/token (tries all identity providers in order, the first of
>     >    which is X509 identity provider)
>     >
>     >    So if you sent a client cert to the last one, it would do what you are
>     >    suggesting.
>     >
>     >    On Thu, Jun 13, 2019 at 2:55 PM Andy LoPresto <[hidden email]> wrote:
>     >>
>     >> No, client cert authentication bypasses the JWT behavior completely. Because a client cert is automatically sent on every request, it makes no sense to delegate the credential to a token in that case.
>     >>
>     >> Andy LoPresto
>     >> [hidden email]
>     >> [hidden email]
>     >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >>
>     >>> On Jun 13, 2019, at 11:52 AM, Shawn Weeks <[hidden email]> wrote:
>     >>>
>     >>> Completely agree on username and password but I'll probably still do something somewhat generic around access tokens vs 2 way ssl as in the future there might be something else. On a related note is it possible to get a JWT with 2 way ssl? If so we could use the same auth method for everything.
>     >>>
>     >>> Thanks
>     >>> Shawn
>     >>>
>     >>> On 6/13/19, 1:36 PM, "Bryan Bende" <[hidden email]> wrote:
>     >>>
>     >>>   Ah thanks for pointing that out, I completely forgot that apparently I
>     >>>   was thinking ahead in the JerseyNiFiClient of how we could support
>     >>>   tokens :)
>     >>>
>     >>>   You would need to make the same changes in the
>     >>>   JerseyNiFiRegistryClient, and then build a new toolkit based on a new
>     >>>   version of nifi-registry-client.
>     >>>
>     >>>   Also, you are correct that we could support username/password, but I
>     >>>   think Kerberos is much better from a security perspective since you
>     >>>   don't really need to give your credentials to the CLI. With
>     >>>   username/password, you would either need to add those properties to
>     >>>   the .props files for the CLI, which then gets into encrypting the
>     >>>   password, or you need to provide them on a command as arguments which
>     >>>   again gets into whether the password is in plain text or not.
>     >>>
>     >>>   On Thu, Jun 13, 2019 at 2:28 PM Shawn Weeks <[hidden email]> wrote:
>     >>>>
>     >>>> Got it, I've been trying to read some of this on my phone and missed something. Currently it looks like the NiFi Client JerseyNiFiClient.java was setup to support token(JWT) based requests but from what I can tell those methods are never called anywhere. NiFi Registry Client only implemented the implicit security and proxied entity methods.
>     >>>>
>     >>>> It looks like I should be able to lookup the auth token and add it to the Jersey WebTarget Headers in the two clients so it would be there on every request. I'll have to do some testing but that might not require too many changes. In theory it could also support username/password auth as well doing it the same way.
>     >>>>
>     >>>> https://stackoverflow.com/questions/29056051/adding-authorization-header-to-jersey-sse-client-request
>     >>>>
>     >>>> Thanks
>     >>>> Shawn
>     >>>>
>     >>>> On 6/13/19, 1:04 PM, "Bryan Bende" <[hidden email]> wrote:
>     >>>>
>     >>>>   I'm not sure if I confused things... the clients that I mentioned are
>     >>>>   wrappers for the REST API implemented with Jersey client, so the CLI
>     >>>>   does exclusively use the REST API.
>     >>>>
>     >>>>   I was just drawing attention to the clients to say that part of the
>     >>>>   work is outside of the CLI in nifi-registry-client to allow it to
>     >>>>   support kerberos auth.
>     >>>>
>     >>>>   On Thu, Jun 13, 2019 at 1:54 PM Shawn Weeks <[hidden email]> wrote:
>     >>>>>
>     >>>>> Ok, I was thinking the CLI used the Rest API exclusively and that's what I was missing. Unfortunately I don't have the option to use self-signed certificate due to organizational security policies and we don't have a way to get SSL Certificates issued to individuals only servers.
>     >>>>>
>     >>>>> Thanks
>     >>>>> Shawn
>     >>>>>
>     >>>>> On 6/13/19, 12:30 PM, "Bryan Bende" <[hidden email]> wrote:
>     >>>>>
>     >>>>>   Just to further elaborate, within the CLI there are commands that work
>     >>>>>   against registry and commands that work against NiFi. For registry
>     >>>>>   commands, they use the Java client that is provided by registry [1].
>     >>>>>   For NiFi commands, there is mini client developed as need with in the
>     >>>>>   CLI [2].
>     >>>>>
>     >>>>>   None of these client calls currently have any concept of a JWT/token.
>     >>>>>
>     >>>>>   In order to do the kerberos auth correctly across both systems, I
>     >>>>>   think both of these clients would need to be updated to support a
>     >>>>>   method that called the /access/kerberos end point to obtain a token,
>     >>>>>   and then also provide a way to pass back that token on future
>     >>>>>   requests. It would likely be the CLI's job to store that token
>     >>>>>   somewhere (in memory for interactive shell, or on filesystem for
>     >>>>>   individual executions) and pass it back on each request. In order to
>     >>>>>   call the /access/kerberos end-point there also needs to be code in the
>     >>>>>   client that handles the negotiation to provide the kerberos
>     >>>>>   credentials that are present from having done a kinit.
>     >>>>>
>     >>>>>   Long story short, Andy's first suggest would be a much easier option
>     >>>>>   with no code changes.
>     >>>>>
>     >>>>>   [1] https://github.com/apache/nifi-registry/tree/master/nifi-registry-core/nifi-registry-client
>     >>>>>   [2] https://github.com/apache/nifi/tree/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client
>     >>>>>
>     >>>>>   On Thu, Jun 13, 2019 at 1:28 PM Andy LoPresto <[hidden email]> wrote:
>     >>>>>>
>     >>>>>> You’ll probably have to write (minimal) code to expose the ClientBuilder constructor/factory methods to the part that parses command-line arguments.
>     >>>>>>
>     >>>>>> Andy LoPresto
>     >>>>>> [hidden email]
>     >>>>>> [hidden email]
>     >>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >>>>>>
>     >>>>>>> On Jun 13, 2019, at 10:27 AM, Shawn Weeks <[hidden email]> wrote:
>     >>>>>>>
>     >>>>>>> Is there a way to pass 2 currently? Because you can get the token via curl like I’m currently doing?
>     >>>>>>>
>     >>>>>>> Thanks
>     >>>>>>> Shawn
>     >>>>>>>
>     >>>>>>> Sent from my iPhone
>     >>>>>>>
>     >>>>>>>> On Jun 13, 2019, at 12:21 PM, Andy LoPresto <[hidden email]> wrote:
>     >>>>>>>>
>     >>>>>>>> I see a couple choices here:
>     >>>>>>>>
>     >>>>>>>> 1. Use the CA to generate and sign a new certificate for deployments. This certificate would not be as sensitive as the server certificate, as you can put stricter permissions on that identity within the NiFi access controls, and the cert would be issued for a DN that cannot be used to impersonate the server itself. Use this certificate to authenticate for deployment activities.
>     >>>>>>>> 2. Manually extract the user’s JWT from the Developer Tools in your browser and pass that into the CLI. This token expires regularly, so you will need to continually update it.
>     >>>>>>>> 3. Build the Kerberos implementation of the authentication aspects of the CLI toolkit.
>     >>>>>>>>
>     >>>>>>>> Andy LoPresto
>     >>>>>>>> [hidden email]
>     >>>>>>>> [hidden email]
>     >>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >>>>>>>>
>     >>>>>>>>> On Jun 13, 2019, at 10:00 AM, Shawn Weeks <[hidden email]> wrote:
>     >>>>>>>>>
>     >>>>>>>>> For our organization the server certificate is considered sensitive and not available to the users who need to deploy to NiFi. Actual authentication to NiFi is handled through Knox and our SSO Service so the end user never deals with SSL or has access to a certificate. Originally I started down the path of writing a bunch of tools based on NiPyAPI to handle deployments but since the CLI already does that I was hoping to save some work. Currently we do several other things via rest using the Kerberos Token.
>     >>>>>>>>>
>     >>>>>>>>> As I looked through the tool kit CLI I was seeing that auth token being passed into all the rest calls so I was hoping I could hijack wherever that was being generated via 2way ssl and add an option to call Kerberos instead to get the token. When I say token I mean the auth bearer token that you can get from a post request to /access/kerberos in NiFi and /access/token/Kerberos in NiFi registry.
>     >>>>>>>>>
>     >>>>>>>>> Thanks
>     >>>>>>>>> Shawn
>     >>>>>>>>>
>     >>>>>>>>> On 6/12/19, 12:06 PM, "Bryan Bende" <[hidden email]> wrote:
>     >>>>>>>>>
>     >>>>>>>>> I meant to say that you obviously could generate certs for CLI users, but I
>     >>>>>>>>> was just mentioning an alternative where you can proxy an identity.
>     >>>>>>>>>
>     >>>>>>>>> Right now the CLI never obtains a token because it is all cert based.
>     >>>>>>>>>
>     >>>>>>>>>> On Wed, Jun 12, 2019 at 1:03 PM Bryan Bende <[hidden email]> wrote:
>     >>>>>>>>>>
>     >>>>>>>>>> Right now the idea is that whoever is running the CLI would have access to
>     >>>>>>>>>> a NiFi server certificate and then you can proxy any user you want. There
>     >>>>>>>>>> should be examples of this in the readme or toolkit guide.
>     >>>>>>>>>>
>     >>>>>>>>>> Supporting Kerberos auth was something I wanted to do, but it’s definitely
>     >>>>>>>>>> not a trivial effort.
>     >>>>>>>>>>
>     >>>>>>>>>> On Wed, Jun 12, 2019 at 12:57 PM Andy LoPresto <[hidden email]>
>     >>>>>>>>>> wrote:
>     >>>>>>>>>>
>     >>>>>>>>>>> Shawn,
>     >>>>>>>>>>>
>     >>>>>>>>>>> I’m not sure I understand your question.
>     >>>>>>>>>>>
>     >>>>>>>>>>> I am in the process of refactoring the TLS Toolkit to integrate with
>     >>>>>>>>>>> public certificate authorities, so in the near future it will be easier to
>     >>>>>>>>>>> use certificates signed by external authorities rather than self-signed.
>     >>>>>>>>>>>
>     >>>>>>>>>>> My understanding is that you are talking about the CLI Toolkit rather
>     >>>>>>>>>>> than the TLS Toolkit, but your reference to “token” was ambiguous, so I’m
>     >>>>>>>>>>> going to proceed with the understanding that you are referring to the JWT
>     >>>>>>>>>>> token used to identify an authenticated user when communicating with the
>     >>>>>>>>>>> NiFi API.
>     >>>>>>>>>>>
>     >>>>>>>>>>> You may want to look at JerseyNiFiClient [1], which has methods for
>     >>>>>>>>>>> getting various clients given an authentication token.
>     >>>>>>>>>>>
>     >>>>>>>>>>> You can create the token via the POST /access/kerberos API [2].
>     >>>>>>>>>>>
>     >>>>>>>>>>> [1]
>     >>>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>     >>>>>>>>>>> <
>     >>>>>>>>>>> https://github.com/apache/nifi/blob/master/nifi-toolkit/nifi-toolkit-cli/src/main/java/org/apache/nifi/toolkit/cli/impl/client/nifi/impl/JerseyNiFiClient.java#L163
>     >>>>>>>>>>>>
>     >>>>>>>>>>> [2] https://nifi.apache.org/docs/nifi-docs/rest-api/index.html <
>     >>>>>>>>>>> https://nifi.apache.org/docs/nifi-docs/rest-api/index.html>
>     >>>>>>>>>>>
>     >>>>>>>>>>> Andy LoPresto
>     >>>>>>>>>>> [hidden email]
>     >>>>>>>>>>> [hidden email]
>     >>>>>>>>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>     >>>>>>>>>>>
>     >>>>>>>>>>>> On Jun 12, 2019, at 9:39 AM, Shawn Weeks <[hidden email]>
>     >>>>>>>>>>> wrote:
>     >>>>>>>>>>>>
>     >>>>>>>>>>>> I work in an environment reluctant to create self signed ssl
>     >>>>>>>>>>> certificates and I’m looking at the feasibility of having the toolkit cli
>     >>>>>>>>>>> authenticate via Kerberos. I was expecting it to be as simple as adding
>     >>>>>>>>>>> another way to get the authentication token but I’m having trouble figuring
>     >>>>>>>>>>> out exactly when the token is created. I see lots of references to it after
>     >>>>>>>>>>> it’s been created.
>     >>>>>>>>>>>>
>     >>>>>>>>>>>> Thanks
>     >>>>>>>>>>>> Shawn
>     >>>>>>>>>>>
>     >>>>>>>>>>> --
>     >>>>>>>>>> Sent from Gmail Mobile
>     >>>>>>>>>>
>     >>>>>>>>> --
>     >>>>>>>>> Sent from Gmail Mobile
>     >>>>>>>>>
>     >>>>>>>>>
>     >>>>>>>>
>     >>>>>>
>     >>>>>
>     >>>>>
>     >>>>
>     >>>>
>     >>>
>     >>>
>     >>
>     >
>     >
>
>
>