SSLContextService Issue on Minifi

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

SSLContextService Issue on Minifi

Harrison Unruh
Hello,

I've been trying to configure Minifi to work with the SSLContextService,
and have been running into an error I can't find much documentation on:

ERROR [main] o.apache.nifi.controller.FlowController Unable to start
PostHTTP[id=80508d8d-015d-1000-0000-000000000000] due to
java.lang.IllegalStateException: Processor PostHTTP is not in a valid state
due to ['SSL Context Service' validated against
'854b66f7-015d-1000-0000-000000000000' is invalid because
StandardSSLContextService - 0.2.0 from org.apache.nifi.minifi -
minifi-ssl-context-service-nar is not compatible with SSLContextService -
unversioned from default - system]

I've tried quite a few configurations of the StandardSSLContextService I'm
using in my flow but haven't been able to avoid this. I was wondering if
you would have any ideas on what would cause this?

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: SSLContextService Issue on Minifi

Aldrin Piri
Hi Harrison,

Sorry to have overlooked this.  Would you be able to share the config you
are using for your attempt?

On Thu, Jul 27, 2017 at 3:24 PM, Harrison Unruh <[hidden email]>
wrote:

> Hello,
>
> I've been trying to configure Minifi to work with the SSLContextService,
> and have been running into an error I can't find much documentation on:
>
> ERROR [main] o.apache.nifi.controller.FlowController Unable to start
> PostHTTP[id=80508d8d-015d-1000-0000-000000000000] due to
> java.lang.IllegalStateException: Processor PostHTTP is not in a valid
> state
> due to ['SSL Context Service' validated against
> '854b66f7-015d-1000-0000-000000000000' is invalid because
> StandardSSLContextService - 0.2.0 from org.apache.nifi.minifi -
> minifi-ssl-context-service-nar is not compatible with SSLContextService -
> unversioned from default - system]
>
> I've tried quite a few configurations of the StandardSSLContextService I'm
> using in my flow but haven't been able to avoid this. I was wondering if
> you would have any ideas on what would cause this?
>
> Thanks!
>
Reply | Threaded
Open this post in threaded view
|

Re: SSLContextService Issue on Minifi

rpersaud
I have encountered a similar issue with ListenTCP, NiFi 1.3 and Minifi 0.2.0:

2017-10-10 20:24:55,184 ERROR [main] o.apache.nifi.controller.FlowController
Unable to start ListenTCP[id=f3feecbe-699d-34c5-0000-000000000000] due to
java.lang.IllegalStateException: Processor ListenTCP is not in a valid state
due to ['SSL Context Service' validated against
'71b53c95-b604-38c7-0000-000000000000' is invalid because
StandardSSLContextService - 0.2.0 from org.apache.nifi.minifi -
minifi-ssl-context-service-nar is not compatible with SSLContextService -
unversioned from default - system]

Looking at StandardSSLContextService in Github, both 1.3 and 1.2.0 (what
Minifi 0.2.0 is using) are the same version.  Here is my configuration with
the sensitive bits removed:

MiNiFi Config Version: 3
Flow Controller:
  name: LoadS3
  comment: ''
Core Properties:
  flow controller graceful shutdown period: 10 sec
  flow service write delay interval: 500 ms
  administrative yield duration: 30 sec
  bored yield duration: 10 millis
  max concurrent threads: 1
  variable registry properties: ''
FlowFile Repository:
  partitions: 256
  checkpoint interval: 2 mins
  always sync: false
  Swap:
    threshold: 20000
    in period: 5 sec
    in threads: 1
    out period: 5 sec
    out threads: 4
Content Repository:
  content claim max appendable size: 10 MB
  content claim max flow files: 100
  always sync: false
Provenance Repository:
  provenance rollover time: 1 min
Component Status Repository:
  buffer size: 1440
  snapshot frequency: 1 min
Security Properties:
  keystore: ''
  keystore type: ''
  keystore password: ''
  key password: ''
  truststore: ''
  truststore type: ''
  truststore password: ''
  ssl protocol: ''
  Sensitive Props:
    key:
    algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
    provider: BC
Processors:
- id: ec6fd775-8998-36f3-0000-000000000000
  name: CompressContent
  class: org.apache.nifi.processors.standard.CompressContent
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list:
  - failure
  Properties:
    Compression Format: gzip
    Compression Level: '9'
    Mode: compress
    Update Filename: 'true'
- id: f3feecbe-699d-34c5-0000-000000000000
  name: ListenTCP
  class: org.apache.nifi.processors.standard.ListenTCP
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list: []
  Properties:
    Character Set: UTF-8
    Client Auth: NONE
    Local Network Interface:
    Max Batch Size: '1'
    Max Number of TCP Connections: '2'
    Max Size of Message Queue: '10000'
    Max Size of Socket Buffer: 1 MB
    Message Delimiter: \n
    Port: '1515'
    Receive Buffer Size: 65507 B
    SSL Context Service: 71b53c95-b604-38c7-0000-000000000000
- id: 96b4f9d1-5d24-3386-0000-000000000000
  name: MergeContent
  class: org.apache.nifi.processors.standard.MergeContent
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list:
  - failure
  - original
  Properties:
    Attribute Strategy: Keep All Unique Attributes
    Compression Level: '1'
    Correlation Attribute Name:
    Delimiter Strategy: Text
    Demarcator File: |2+

    Footer File:
    Header File:
    Keep Path: 'false'
    Max Bin Age: 60 sec
    Maximum Group Size:
    Maximum Number of Entries: '100'
    Maximum number of Bins: '5'
    Merge Format: Binary Concatenation
    Merge Strategy: Bin-Packing Algorithm
    Minimum Group Size: 0 B
    Minimum Number of Entries: '100'
- id: d45e9378-054d-33fb-0000-000000000000
  name: PutS3Object
  class: org.apache.nifi.processors.aws.s3.PutS3Object
  max concurrent tasks: 1
  scheduling strategy: TIMER_DRIVEN
  scheduling period: 0 sec
  penalization period: 30 sec
  yield period: 1 sec
  run duration nanos: 0
  auto-terminated relationships list:
  - failure
  - success
  Properties:
    AWS Credentials Provider service:
    Access Key: REMOVED
    Bucket: REMOVED
    Communications Timeout: 30 secs
    Content Type:
    Credentials File:
    Endpoint Override URL:
    Expiration Time Rule:
    FullControl User List: ${s3.permissions.full.users}
    Multipart Part Size: 5 GB
    Multipart Threshold: 5 GB
    Multipart Upload AgeOff Interval: 60 min
    Multipart Upload Max Age Threshold: 7 days
    Object Key:
${now():format('yyyy-MM-dd'):prepend('dt='):append('/'):append(${filename}):prepend('logs/')}
    Owner: ${s3.owner}
    Proxy Host:
    Proxy Host Port:
    Read ACL User List: ${s3.permissions.readacl.users}
    Read Permission User List: ${s3.permissions.read.users}
    Region: us-east-1
    SSL Context Service:
    Secret Key: REMOVED
    Signer Override: Default Signature
    Storage Class: ReducedRedundancy
    Write ACL User List: ${s3.permissions.writeacl.users}
    Write Permission User List: ${s3.permissions.write.users}
    canned-acl: ${s3.permissions.cannedacl}
    server-side-encryption: None
Controller Services:
- id: 71b53c95-b604-38c7-0000-000000000000
  name: StandardSSLContextService
  type: org.apache.nifi.ssl.StandardSSLContextService
  Properties:
    Keystore Filename: REMOVED
    Keystore Password: REMOVED
    Keystore Type: JKS
    SSL Protocol: TLS
    Truststore Filename:
    Truststore Password:
    Truststore Type:
    key-password:
Process Groups: []
Input Ports: []
Output Ports: []
Funnels: []
Connections:
- id: 2da3e0cf-f0f2-3f98-0000-000000000000
  name: CompressContent/success/PutS3Object
  source id: ec6fd775-8998-36f3-0000-000000000000
  source relationship names:
  - success
  destination id: d45e9378-054d-33fb-0000-000000000000
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
- id: 079ecd28-60f0-3ac5-0000-000000000000
  name: ListenTCP/success/MergeContent
  source id: f3feecbe-699d-34c5-0000-000000000000
  source relationship names:
  - success
  destination id: 96b4f9d1-5d24-3386-0000-000000000000
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
- id: b766358d-1e9d-3da9-0000-000000000000
  name: MergeContent/merged/CompressContent
  source id: 96b4f9d1-5d24-3386-0000-000000000000
  source relationship names:
  - merged
  destination id: ec6fd775-8998-36f3-0000-000000000000
  max work queue size: 10000
  max work queue data size: 1 GB
  flowfile expiration: 0 sec
  queue prioritizer class: ''
Remote Process Groups: []
NiFi Properties Overrides: {}





--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: SSLContextService Issue on Minifi

Aldrin Piri
Hi Ryan,

I believe this is the same core issue as described in MINIFI-403 [1].  This
was addressed via PR #91 [2] where another user reported similar issues.
The root cause of this was duplicate libraries that were treated in the
system scope as a bundle and precluded the bundled versions from being
used.  Those incorrect and duplicate items were removed.

If you are in a position to do so and interested, I would encourage you to
perform a build and verify that it resolves your issue.  If not, this fix
will be in the next release which we are like due for in the near future.

Sorry for the hiccup and let us know if you decide to attempt a build.

--aldrin


[1] https://issues.apache.org/jira/browse/MINIFI-403
[2] https://github.com/apache/nifi-minifi/pull/91

On Tue, Oct 10, 2017 at 4:52 PM, rpersaud <[hidden email]> wrote:

> I have encountered a similar issue with ListenTCP, NiFi 1.3 and Minifi
> 0.2.0:
>
> 2017-10-10 20:24:55,184 ERROR [main] o.apache.nifi.controller.
> FlowController
> Unable to start ListenTCP[id=f3feecbe-699d-34c5-0000-000000000000] due to
> java.lang.IllegalStateException: Processor ListenTCP is not in a valid
> state
> due to ['SSL Context Service' validated against
> '71b53c95-b604-38c7-0000-000000000000' is invalid because
> StandardSSLContextService - 0.2.0 from org.apache.nifi.minifi -
> minifi-ssl-context-service-nar is not compatible with SSLContextService -
> unversioned from default - system]
>
> Looking at StandardSSLContextService in Github, both 1.3 and 1.2.0 (what
> Minifi 0.2.0 is using) are the same version.  Here is my configuration with
> the sensitive bits removed:
>
> MiNiFi Config Version: 3
> Flow Controller:
>   name: LoadS3
>   comment: ''
> Core Properties:
>   flow controller graceful shutdown period: 10 sec
>   flow service write delay interval: 500 ms
>   administrative yield duration: 30 sec
>   bored yield duration: 10 millis
>   max concurrent threads: 1
>   variable registry properties: ''
> FlowFile Repository:
>   partitions: 256
>   checkpoint interval: 2 mins
>   always sync: false
>   Swap:
>     threshold: 20000
>     in period: 5 sec
>     in threads: 1
>     out period: 5 sec
>     out threads: 4
> Content Repository:
>   content claim max appendable size: 10 MB
>   content claim max flow files: 100
>   always sync: false
> Provenance Repository:
>   provenance rollover time: 1 min
> Component Status Repository:
>   buffer size: 1440
>   snapshot frequency: 1 min
> Security Properties:
>   keystore: ''
>   keystore type: ''
>   keystore password: ''
>   key password: ''
>   truststore: ''
>   truststore type: ''
>   truststore password: ''
>   ssl protocol: ''
>   Sensitive Props:
>     key:
>     algorithm: PBEWITHMD5AND256BITAES-CBC-OPENSSL
>     provider: BC
> Processors:
> - id: ec6fd775-8998-36f3-0000-000000000000
>   name: CompressContent
>   class: org.apache.nifi.processors.standard.CompressContent
>   max concurrent tasks: 1
>   scheduling strategy: TIMER_DRIVEN
>   scheduling period: 0 sec
>   penalization period: 30 sec
>   yield period: 1 sec
>   run duration nanos: 0
>   auto-terminated relationships list:
>   - failure
>   Properties:
>     Compression Format: gzip
>     Compression Level: '9'
>     Mode: compress
>     Update Filename: 'true'
> - id: f3feecbe-699d-34c5-0000-000000000000
>   name: ListenTCP
>   class: org.apache.nifi.processors.standard.ListenTCP
>   max concurrent tasks: 1
>   scheduling strategy: TIMER_DRIVEN
>   scheduling period: 0 sec
>   penalization period: 30 sec
>   yield period: 1 sec
>   run duration nanos: 0
>   auto-terminated relationships list: []
>   Properties:
>     Character Set: UTF-8
>     Client Auth: NONE
>     Local Network Interface:
>     Max Batch Size: '1'
>     Max Number of TCP Connections: '2'
>     Max Size of Message Queue: '10000'
>     Max Size of Socket Buffer: 1 MB
>     Message Delimiter: \n
>     Port: '1515'
>     Receive Buffer Size: 65507 B
>     SSL Context Service: 71b53c95-b604-38c7-0000-000000000000
> - id: 96b4f9d1-5d24-3386-0000-000000000000
>   name: MergeContent
>   class: org.apache.nifi.processors.standard.MergeContent
>   max concurrent tasks: 1
>   scheduling strategy: TIMER_DRIVEN
>   scheduling period: 0 sec
>   penalization period: 30 sec
>   yield period: 1 sec
>   run duration nanos: 0
>   auto-terminated relationships list:
>   - failure
>   - original
>   Properties:
>     Attribute Strategy: Keep All Unique Attributes
>     Compression Level: '1'
>     Correlation Attribute Name:
>     Delimiter Strategy: Text
>     Demarcator File: |2+
>
>     Footer File:
>     Header File:
>     Keep Path: 'false'
>     Max Bin Age: 60 sec
>     Maximum Group Size:
>     Maximum Number of Entries: '100'
>     Maximum number of Bins: '5'
>     Merge Format: Binary Concatenation
>     Merge Strategy: Bin-Packing Algorithm
>     Minimum Group Size: 0 B
>     Minimum Number of Entries: '100'
> - id: d45e9378-054d-33fb-0000-000000000000
>   name: PutS3Object
>   class: org.apache.nifi.processors.aws.s3.PutS3Object
>   max concurrent tasks: 1
>   scheduling strategy: TIMER_DRIVEN
>   scheduling period: 0 sec
>   penalization period: 30 sec
>   yield period: 1 sec
>   run duration nanos: 0
>   auto-terminated relationships list:
>   - failure
>   - success
>   Properties:
>     AWS Credentials Provider service:
>     Access Key: REMOVED
>     Bucket: REMOVED
>     Communications Timeout: 30 secs
>     Content Type:
>     Credentials File:
>     Endpoint Override URL:
>     Expiration Time Rule:
>     FullControl User List: ${s3.permissions.full.users}
>     Multipart Part Size: 5 GB
>     Multipart Threshold: 5 GB
>     Multipart Upload AgeOff Interval: 60 min
>     Multipart Upload Max Age Threshold: 7 days
>     Object Key:
> ${now():format('yyyy-MM-dd'):prepend('dt='):append('/'):
> append(${filename}):prepend('logs/')}
>     Owner: ${s3.owner}
>     Proxy Host:
>     Proxy Host Port:
>     Read ACL User List: ${s3.permissions.readacl.users}
>     Read Permission User List: ${s3.permissions.read.users}
>     Region: us-east-1
>     SSL Context Service:
>     Secret Key: REMOVED
>     Signer Override: Default Signature
>     Storage Class: ReducedRedundancy
>     Write ACL User List: ${s3.permissions.writeacl.users}
>     Write Permission User List: ${s3.permissions.write.users}
>     canned-acl: ${s3.permissions.cannedacl}
>     server-side-encryption: None
> Controller Services:
> - id: 71b53c95-b604-38c7-0000-000000000000
>   name: StandardSSLContextService
>   type: org.apache.nifi.ssl.StandardSSLContextService
>   Properties:
>     Keystore Filename: REMOVED
>     Keystore Password: REMOVED
>     Keystore Type: JKS
>     SSL Protocol: TLS
>     Truststore Filename:
>     Truststore Password:
>     Truststore Type:
>     key-password:
> Process Groups: []
> Input Ports: []
> Output Ports: []
> Funnels: []
> Connections:
> - id: 2da3e0cf-f0f2-3f98-0000-000000000000
>   name: CompressContent/success/PutS3Object
>   source id: ec6fd775-8998-36f3-0000-000000000000
>   source relationship names:
>   - success
>   destination id: d45e9378-054d-33fb-0000-000000000000
>   max work queue size: 10000
>   max work queue data size: 1 GB
>   flowfile expiration: 0 sec
>   queue prioritizer class: ''
> - id: 079ecd28-60f0-3ac5-0000-000000000000
>   name: ListenTCP/success/MergeContent
>   source id: f3feecbe-699d-34c5-0000-000000000000
>   source relationship names:
>   - success
>   destination id: 96b4f9d1-5d24-3386-0000-000000000000
>   max work queue size: 10000
>   max work queue data size: 1 GB
>   flowfile expiration: 0 sec
>   queue prioritizer class: ''
> - id: b766358d-1e9d-3da9-0000-000000000000
>   name: MergeContent/merged/CompressContent
>   source id: 96b4f9d1-5d24-3386-0000-000000000000
>   source relationship names:
>   - merged
>   destination id: ec6fd775-8998-36f3-0000-000000000000
>   max work queue size: 10000
>   max work queue data size: 1 GB
>   flowfile expiration: 0 sec
>   queue prioritizer class: ''
> Remote Process Groups: []
> NiFi Properties Overrides: {}
>
>
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>