Local development and testing w/ kerberos

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Local development and testing w/ kerberos

Mike Thomsen
Looking for suggestions on local development and testing with kerberos. We
have a kerberized cluster set up in an AWS instance, but it's more for UAT
than development. Anyone have any suggestions/experience, say, setting up a
Mac or Linux box for developing and testing like this?

Thanks,

Mike
Reply | Threaded
Open this post in threaded view
|

Re: Local development and testing w/ kerberos

Bryan Bende
There is a docker-kdc project that is easy to use:

https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication

It was made before docker for mac was good/popular and it previously
relied on boot2docker, but I made the following modification to not
use boot2docker....

docker-kdc$ git diff
diff --git a/kdc b/kdc
index 9410fc5..0a887e1 100755
--- a/kdc
+++ b/kdc
@@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm'
 GET_KDC_HOST="echo $KDC_NATHOST"

 # Adjust container in case of OSX.
-if [[ $OSTYPE =~ darwin.+ ]]; then
-       CONTAINER='boot2docker'
-       GET_KDC_HOST='boot2docker ip'
-fi
+#if [[ $OSTYPE =~ darwin.+ ]]; then
+#      CONTAINER='boot2docker'
+#      GET_KDC_HOST='boot2docker ip'
+#fi

On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen <[hidden email]> wrote:
>
> Looking for suggestions on local development and testing with kerberos. We
> have a kerberized cluster set up in an AWS instance, but it's more for UAT
> than development. Anyone have any suggestions/experience, say, setting up a
> Mac or Linux box for developing and testing like this?
>
> Thanks,
>
> Mike
Reply | Threaded
Open this post in threaded view
|

Re: Local development and testing w/ kerberos

Mike Thomsen
Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's
working great so far.

On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende <[hidden email]> wrote:

> There is a docker-kdc project that is easy to use:
>
>
> https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication
>
> It was made before docker for mac was good/popular and it previously
> relied on boot2docker, but I made the following modification to not
> use boot2docker....
>
> docker-kdc$ git diff
> diff --git a/kdc b/kdc
> index 9410fc5..0a887e1 100755
> --- a/kdc
> +++ b/kdc
> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm'
>  GET_KDC_HOST="echo $KDC_NATHOST"
>
>  # Adjust container in case of OSX.
> -if [[ $OSTYPE =~ darwin.+ ]]; then
> -       CONTAINER='boot2docker'
> -       GET_KDC_HOST='boot2docker ip'
> -fi
> +#if [[ $OSTYPE =~ darwin.+ ]]; then
> +#      CONTAINER='boot2docker'
> +#      GET_KDC_HOST='boot2docker ip'
> +#fi
>
> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen <[hidden email]>
> wrote:
> >
> > Looking for suggestions on local development and testing with kerberos.
> We
> > have a kerberized cluster set up in an AWS instance, but it's more for
> UAT
> > than development. Anyone have any suggestions/experience, say, setting
> up a
> > Mac or Linux box for developing and testing like this?
> >
> > Thanks,
> >
> > Mike
>
Reply | Threaded
Open this post in threaded view
|

Re: Local development and testing w/ kerberos

Mike Thomsen
Alright, I think I'm pretty close here. I followed all of those steps,
except I changed bbende to mthomsen.

* I can run kinit [hidden email] and it works.
* I can run klist and see the expected output.

When I bring up NiFi, I get the following (trimmed for brevity):

Caused by:
org.apache.nifi.authorization.exception.AuthorizerCreationException:
org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable
to locate initial admin [hidden email] to seed policies
    at
org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at
org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54)
    at com.sun.proxy.$Proxy76.onConfigured(Unknown Source)
    at
org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152)
    at
org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
    ... 96 common frames omitted
Caused by:
org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable
to locate initial admin [hidden email] to seed policies
    at
org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598)
    at
org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541)
    at
org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254)
    ... 104 common frames omitted

I double-checked the paths to krb5.conf and the keytab and they're both
pointing to /tmp/docker-kdc

Any ideas?

Thanks,

Mike


On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen <[hidden email]>
wrote:

> Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's
> working great so far.
>
> On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende <[hidden email]> wrote:
>
>> There is a docker-kdc project that is easy to use:
>>
>>
>> https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication
>>
>> It was made before docker for mac was good/popular and it previously
>> relied on boot2docker, but I made the following modification to not
>> use boot2docker....
>>
>> docker-kdc$ git diff
>> diff --git a/kdc b/kdc
>> index 9410fc5..0a887e1 100755
>> --- a/kdc
>> +++ b/kdc
>> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm'
>>  GET_KDC_HOST="echo $KDC_NATHOST"
>>
>>  # Adjust container in case of OSX.
>> -if [[ $OSTYPE =~ darwin.+ ]]; then
>> -       CONTAINER='boot2docker'
>> -       GET_KDC_HOST='boot2docker ip'
>> -fi
>> +#if [[ $OSTYPE =~ darwin.+ ]]; then
>> +#      CONTAINER='boot2docker'
>> +#      GET_KDC_HOST='boot2docker ip'
>> +#fi
>>
>> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen <[hidden email]>
>> wrote:
>> >
>> > Looking for suggestions on local development and testing with kerberos.
>> We
>> > have a kerberized cluster set up in an AWS instance, but it's more for
>> UAT
>> > than development. Anyone have any suggestions/experience, say, setting
>> up a
>> > Mac or Linux box for developing and testing like this?
>> >
>> > Thanks,
>> >
>> > Mike
>>
>
Reply | Threaded
Open this post in threaded view
|

Re: Local development and testing w/ kerberos

Sivaprasanna Sethuraman
Can you share the authorizers.xml? I guess something wrong with the CN
that’s mentioned there.

-
Sivaprasanna

On Wed, 24 Oct 2018 at 8:48 PM, Mike Thomsen <[hidden email]> wrote:

> Alright, I think I'm pretty close here. I followed all of those steps,
> except I changed bbende to mthomsen.
>
> * I can run kinit [hidden email] and it works.
> * I can run klist and see the expected output.
>
> When I bring up NiFi, I get the following (trimmed for brevity):
>
> Caused by:
> org.apache.nifi.authorization.exception.AuthorizerCreationException:
> org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable
> to locate initial admin [hidden email] to seed policies
>     at
>
> org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at
>
> org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54)
>     at com.sun.proxy.$Proxy76.onConfigured(Unknown Source)
>     at
>
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152)
>     at
>
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
>     ... 96 common frames omitted
> Caused by:
> org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable
> to locate initial admin [hidden email] to seed policies
>     at
>
> org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598)
>     at
>
> org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541)
>     at
>
> org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254)
>     ... 104 common frames omitted
>
> I double-checked the paths to krb5.conf and the keytab and they're both
> pointing to /tmp/docker-kdc
>
> Any ideas?
>
> Thanks,
>
> Mike
>
>
> On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen <[hidden email]>
> wrote:
>
> > Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's
> > working great so far.
> >
> > On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende <[hidden email]> wrote:
> >
> >> There is a docker-kdc project that is easy to use:
> >>
> >>
> >>
> https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication
> >>
> >> It was made before docker for mac was good/popular and it previously
> >> relied on boot2docker, but I made the following modification to not
> >> use boot2docker....
> >>
> >> docker-kdc$ git diff
> >> diff --git a/kdc b/kdc
> >> index 9410fc5..0a887e1 100755
> >> --- a/kdc
> >> +++ b/kdc
> >> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm'
> >>  GET_KDC_HOST="echo $KDC_NATHOST"
> >>
> >>  # Adjust container in case of OSX.
> >> -if [[ $OSTYPE =~ darwin.+ ]]; then
> >> -       CONTAINER='boot2docker'
> >> -       GET_KDC_HOST='boot2docker ip'
> >> -fi
> >> +#if [[ $OSTYPE =~ darwin.+ ]]; then
> >> +#      CONTAINER='boot2docker'
> >> +#      GET_KDC_HOST='boot2docker ip'
> >> +#fi
> >>
> >> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen <[hidden email]>
> >> wrote:
> >> >
> >> > Looking for suggestions on local development and testing with
> kerberos.
> >> We
> >> > have a kerberized cluster set up in an AWS instance, but it's more for
> >> UAT
> >> > than development. Anyone have any suggestions/experience, say, setting
> >> up a
> >> > Mac or Linux box for developing and testing like this?
> >> >
> >> > Thanks,
> >> >
> >> > Mike
> >>
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Local development and testing w/ kerberos

Bryan Bende
In reply to this post by Mike Thomsen
I think all your kerberos/KDC stuff is fine, you just need to add
[hidden email] to the user-group-provider.

My post was old before we had separated authorizer into
user-group-provider and access-policy-provider.
On Wed, Oct 24, 2018 at 11:18 AM Mike Thomsen <[hidden email]> wrote:

>
> Alright, I think I'm pretty close here. I followed all of those steps,
> except I changed bbende to mthomsen.
>
> * I can run kinit [hidden email] and it works.
> * I can run klist and see the expected output.
>
> When I bring up NiFi, I get the following (trimmed for brevity):
>
> Caused by:
> org.apache.nifi.authorization.exception.AuthorizerCreationException:
> org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable
> to locate initial admin [hidden email] to seed policies
>     at
> org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at
> org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54)
>     at com.sun.proxy.$Proxy76.onConfigured(Unknown Source)
>     at
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152)
>     at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
>     ... 96 common frames omitted
> Caused by:
> org.apache.nifi.authorization.exception.AuthorizerCreationException: Unable
> to locate initial admin [hidden email] to seed policies
>     at
> org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598)
>     at
> org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541)
>     at
> org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254)
>     ... 104 common frames omitted
>
> I double-checked the paths to krb5.conf and the keytab and they're both
> pointing to /tmp/docker-kdc
>
> Any ideas?
>
> Thanks,
>
> Mike
>
>
> On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen <[hidden email]>
> wrote:
>
> > Awesome, thanks Bryan! I'm halfway through that (got klist view) and it's
> > working great so far.
> >
> > On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende <[hidden email]> wrote:
> >
> >> There is a docker-kdc project that is easy to use:
> >>
> >>
> >> https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication
> >>
> >> It was made before docker for mac was good/popular and it previously
> >> relied on boot2docker, but I made the following modification to not
> >> use boot2docker....
> >>
> >> docker-kdc$ git diff
> >> diff --git a/kdc b/kdc
> >> index 9410fc5..0a887e1 100755
> >> --- a/kdc
> >> +++ b/kdc
> >> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm'
> >>  GET_KDC_HOST="echo $KDC_NATHOST"
> >>
> >>  # Adjust container in case of OSX.
> >> -if [[ $OSTYPE =~ darwin.+ ]]; then
> >> -       CONTAINER='boot2docker'
> >> -       GET_KDC_HOST='boot2docker ip'
> >> -fi
> >> +#if [[ $OSTYPE =~ darwin.+ ]]; then
> >> +#      CONTAINER='boot2docker'
> >> +#      GET_KDC_HOST='boot2docker ip'
> >> +#fi
> >>
> >> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen <[hidden email]>
> >> wrote:
> >> >
> >> > Looking for suggestions on local development and testing with kerberos.
> >> We
> >> > have a kerberized cluster set up in an AWS instance, but it's more for
> >> UAT
> >> > than development. Anyone have any suggestions/experience, say, setting
> >> up a
> >> > Mac or Linux box for developing and testing like this?
> >> >
> >> > Thanks,
> >> >
> >> > Mike
> >>
> >
Reply | Threaded
Open this post in threaded view
|

Re: Local development and testing w/ kerberos

Mike Thomsen
Thanks. Got that working and was able to login as [hidden email].

On Wed, Oct 24, 2018 at 11:24 AM Bryan Bende <[hidden email]> wrote:

> I think all your kerberos/KDC stuff is fine, you just need to add
> [hidden email] to the user-group-provider.
>
> My post was old before we had separated authorizer into
> user-group-provider and access-policy-provider.
> On Wed, Oct 24, 2018 at 11:18 AM Mike Thomsen <[hidden email]>
> wrote:
> >
> > Alright, I think I'm pretty close here. I followed all of those steps,
> > except I changed bbende to mthomsen.
> >
> > * I can run kinit [hidden email] and it works.
> > * I can run klist and see the expected output.
> >
> > When I bring up NiFi, I get the following (trimmed for brevity):
> >
> > Caused by:
> > org.apache.nifi.authorization.exception.AuthorizerCreationException:
> > org.apache.nifi.authorization.exception.AuthorizerCreationException:
> Unable
> > to locate initial admin [hidden email] to seed policies
> >     at
> >
> org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:263)
> >     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >     at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> >     at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >     at java.lang.reflect.Method.invoke(Method.java:498)
> >     at
> >
> org.apache.nifi.authorization.AccessPolicyProviderInvocationHandler.invoke(AccessPolicyProviderInvocationHandler.java:54)
> >     at com.sun.proxy.$Proxy76.onConfigured(Unknown Source)
> >     at
> >
> org.apache.nifi.authorization.AuthorizerFactoryBean.getObject(AuthorizerFactoryBean.java:152)
> >     at
> >
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:178)
> >     ... 96 common frames omitted
> > Caused by:
> > org.apache.nifi.authorization.exception.AuthorizerCreationException:
> Unable
> > to locate initial admin [hidden email] to seed policies
> >     at
> >
> org.apache.nifi.authorization.FileAccessPolicyProvider.populateInitialAdmin(FileAccessPolicyProvider.java:598)
> >     at
> >
> org.apache.nifi.authorization.FileAccessPolicyProvider.load(FileAccessPolicyProvider.java:541)
> >     at
> >
> org.apache.nifi.authorization.FileAccessPolicyProvider.onConfigured(FileAccessPolicyProvider.java:254)
> >     ... 104 common frames omitted
> >
> > I double-checked the paths to krb5.conf and the keytab and they're both
> > pointing to /tmp/docker-kdc
> >
> > Any ideas?
> >
> > Thanks,
> >
> > Mike
> >
> >
> > On Wed, Oct 24, 2018 at 10:28 AM Mike Thomsen <[hidden email]>
> > wrote:
> >
> > > Awesome, thanks Bryan! I'm halfway through that (got klist view) and
> it's
> > > working great so far.
> > >
> > > On Wed, Oct 24, 2018 at 9:36 AM Bryan Bende <[hidden email]> wrote:
> > >
> > >> There is a docker-kdc project that is easy to use:
> > >>
> > >>
> > >>
> https://bryanbende.com/development/2016/08/31/apache-nifi-1.0.0-kerberos-authentication
> > >>
> > >> It was made before docker for mac was good/popular and it previously
> > >> relied on boot2docker, but I made the following modification to not
> > >> use boot2docker....
> > >>
> > >> docker-kdc$ git diff
> > >> diff --git a/kdc b/kdc
> > >> index 9410fc5..0a887e1 100755
> > >> --- a/kdc
> > >> +++ b/kdc
> > >> @@ -90,10 +90,10 @@ CONTROL_VM='VBoxManage controlvm boot2docker-vm'
> > >>  GET_KDC_HOST="echo $KDC_NATHOST"
> > >>
> > >>  # Adjust container in case of OSX.
> > >> -if [[ $OSTYPE =~ darwin.+ ]]; then
> > >> -       CONTAINER='boot2docker'
> > >> -       GET_KDC_HOST='boot2docker ip'
> > >> -fi
> > >> +#if [[ $OSTYPE =~ darwin.+ ]]; then
> > >> +#      CONTAINER='boot2docker'
> > >> +#      GET_KDC_HOST='boot2docker ip'
> > >> +#fi
> > >>
> > >> On Wed, Oct 24, 2018 at 7:35 AM Mike Thomsen <[hidden email]>
> > >> wrote:
> > >> >
> > >> > Looking for suggestions on local development and testing with
> kerberos.
> > >> We
> > >> > have a kerberized cluster set up in an AWS instance, but it's more
> for
> > >> UAT
> > >> > than development. Anyone have any suggestions/experience, say,
> setting
> > >> up a
> > >> > Mac or Linux box for developing and testing like this?
> > >> >
> > >> > Thanks,
> > >> >
> > >> > Mike
> > >>
> > >
>