SOAP Service through InvokeHTTP

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

SOAP Service through InvokeHTTP

mliem90
This post was updated on .
Hello,

I am looking to do a SOAP Web service call and was told invokeHTTP may be able to work. Wonder how I would be able to do this. For example, using curl I could invoke SOAP like the below:

curl -X POST -H "Content-Type: text/xml" -H "SOAPAction: "http://www.webserviceX.NET/GetCitiesByCountry"" --data-binary @request.xml http://www.webservicex.net/globalweather.asmx

the request.xml:
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:web="http://www.webserviceX.NET">
   <soap:Header/>
   <soap:Body>
      <web:GetCitiesByCountry>
         
         <web:CountryName>CANADA</web:CountryName>
      </web:GetCitiesByCountry>
   </soap:Body>
</soap:Envelope>

Looking to understand how I'd be able to pass the request.xml in the invokeHTTP

I know a custom built processor is an option, but looking to use what's available in the current version. Any other recommendations aside from invokeHTTP?
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Jeff
Hello,

With InvokeHTTP, you can add dynamic properties, which will be sent in the
request as headers.  You can use dynamic properties to set values for the
Content-Type and SOAPAction headers, just use the header names for the
names of the dynamic properties.  InvokeHTTP lets you control the HTTP
method, so you can set that to POST.  The remaining step would be to get
the content of request.xml to be sent to the InvokeHTTP as a flowfile.  One
way to do this is to use a GetFile processor to fetch requeset.xml from
some location on the filesystem, and pass the success relationship of
GetFile to InvokeHTTP.

On Wed, Aug 24, 2016 at 7:06 PM mliem90 <[hidden email]> wrote:

> Hello,
>
> I am looking to do a SOAP Web service call and was told invokeHTTP may be
> able to work. Wonder how I would be able to do this. For example, using
> curl
> I could invoke SOAP like the below:
>
> curl -X POST -H "Content-Type: text/xml" -H "SOAPAction:
> "http://www.webserviceX.NET/GetCitiesByCountry"" --data-binary
> @request.xml
> http://www.webservicex.net/globalweather.asmx
>
> the request.xml:
> <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
> xmlns:web="http://www.webserviceX.NET">
>    <soap:Header/>
>    <soap:Body>
>       <web:GetCitiesByCountry>
>
>          <web:CountryName>CANADA</web:CountryName>
>       </web:GetCitiesByCountry>
>    </soap:Body>
> </soap:Envelope>
>
> I Appreciate the help!
>
>
>
> --
> View this message in context:
> http://apache-nifi-developer-list.39713.n7.nabble.com/SOAP-Service-through-InvokeHTTP-tp13129.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

mliem90
Awesome - did not know the contents of the flowfile would be passed as the result.xml.

worked perfectly.

Thanks!
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Andrew Psaltis
In reply to this post by Jeff
Hello,
I have written a SOAP processor [1] that you are free to use. It has gotten
a bit of testing under it's belt, however, it can always use more. I am
totally open to fixing issues you may find and/or accepting PR's. If it
makes enough sense and is complete enough I have no problem contributing
it.

Thanks,
Andrew

[1] https://github.com/apsaltis/nifi-soap

On Wed, Aug 24, 2016 at 7:40 PM, Jeff <[hidden email]> wrote:

> Hello,
>
> With InvokeHTTP, you can add dynamic properties, which will be sent in the
> request as headers.  You can use dynamic properties to set values for the
> Content-Type and SOAPAction headers, just use the header names for the
> names of the dynamic properties.  InvokeHTTP lets you control the HTTP
> method, so you can set that to POST.  The remaining step would be to get
> the content of request.xml to be sent to the InvokeHTTP as a flowfile.  One
> way to do this is to use a GetFile processor to fetch requeset.xml from
> some location on the filesystem, and pass the success relationship of
> GetFile to InvokeHTTP.
>
> On Wed, Aug 24, 2016 at 7:06 PM mliem90 <[hidden email]> wrote:
>
> > Hello,
> >
> > I am looking to do a SOAP Web service call and was told invokeHTTP may be
> > able to work. Wonder how I would be able to do this. For example, using
> > curl
> > I could invoke SOAP like the below:
> >
> > curl -X POST -H "Content-Type: text/xml" -H "SOAPAction:
> > "http://www.webserviceX.NET/GetCitiesByCountry"" --data-binary
> > @request.xml
> > http://www.webservicex.net/globalweather.asmx
> >
> > the request.xml:
> > <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
> > xmlns:web="http://www.webserviceX.NET">
> >    <soap:Header/>
> >    <soap:Body>
> >       <web:GetCitiesByCountry>
> >
> >          <web:CountryName>CANADA</web:CountryName>
> >       </web:GetCitiesByCountry>
> >    </soap:Body>
> > </soap:Envelope>
> >
> > I Appreciate the help!
> >
> >
> >
> > --
> > View this message in context:
> > http://apache-nifi-developer-list.39713.n7.nabble.com/SOAP-
> Service-through-InvokeHTTP-tp13129.html
> > Sent from the Apache NiFi Developer List mailing list archive at
> > Nabble.com.
> >
>



--
Thanks,
Andrew

Subscribe to my book: Streaming Data <http://manning.com/psaltis>
<https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Jeff
Glad it worked for you!

On Wed, Aug 24, 2016 at 8:10 PM Andrew Psaltis <[hidden email]>
wrote:

> Hello,
> I have written a SOAP processor [1] that you are free to use. It has gotten
> a bit of testing under it's belt, however, it can always use more. I am
> totally open to fixing issues you may find and/or accepting PR's. If it
> makes enough sense and is complete enough I have no problem contributing
> it.
>
> Thanks,
> Andrew
>
> [1] https://github.com/apsaltis/nifi-soap
>
> On Wed, Aug 24, 2016 at 7:40 PM, Jeff <[hidden email]> wrote:
>
> > Hello,
> >
> > With InvokeHTTP, you can add dynamic properties, which will be sent in
> the
> > request as headers.  You can use dynamic properties to set values for the
> > Content-Type and SOAPAction headers, just use the header names for the
> > names of the dynamic properties.  InvokeHTTP lets you control the HTTP
> > method, so you can set that to POST.  The remaining step would be to get
> > the content of request.xml to be sent to the InvokeHTTP as a flowfile.
> One
> > way to do this is to use a GetFile processor to fetch requeset.xml from
> > some location on the filesystem, and pass the success relationship of
> > GetFile to InvokeHTTP.
> >
> > On Wed, Aug 24, 2016 at 7:06 PM mliem90 <[hidden email]> wrote:
> >
> > > Hello,
> > >
> > > I am looking to do a SOAP Web service call and was told invokeHTTP may
> be
> > > able to work. Wonder how I would be able to do this. For example, using
> > > curl
> > > I could invoke SOAP like the below:
> > >
> > > curl -X POST -H "Content-Type: text/xml" -H "SOAPAction:
> > > "http://www.webserviceX.NET/GetCitiesByCountry"" --data-binary
> > > @request.xml
> > > http://www.webservicex.net/globalweather.asmx
> > >
> > > the request.xml:
> > > <soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
> > > xmlns:web="http://www.webserviceX.NET">
> > >    <soap:Header/>
> > >    <soap:Body>
> > >       <web:GetCitiesByCountry>
> > >
> > >          <web:CountryName>CANADA</web:CountryName>
> > >       </web:GetCitiesByCountry>
> > >    </soap:Body>
> > > </soap:Envelope>
> > >
> > > I Appreciate the help!
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > > http://apache-nifi-developer-list.39713.n7.nabble.com/SOAP-
> > Service-through-InvokeHTTP-tp13129.html
> > > Sent from the Apache NiFi Developer List mailing list archive at
> > > Nabble.com.
> > >
> >
>
>
>
> --
> Thanks,
> Andrew
>
> Subscribe to my book: Streaming Data <http://manning.com/psaltis>
> <https://www.linkedin.com/pub/andrew-psaltis/1/17b/306>
> twiiter: @itmdata <http://twitter.com/intent/user?screen_name=itmdata>
>
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Hitendra Kashyap
In reply to this post by Andrew Psaltis
Hi Jeff,

I am unable to use the NiFi-soap with the latest version of nifi (1.1.0).
after build and deploying nar file to lib when I restart nifi it doesn't
start. bootstrap log tail has following:

2016-12-15 01:25:05,216 INFO [main] org.apache.nifi.bootstrap.Command
Working Directory: /appl/idc/nifi-1.1.0
2016-12-15 01:25:05,217 INFO [main] org.apache.nifi.bootstrap.Command
Command: java -classpath
/appl/idc/nifi-1.1.0/./conf:/appl/idc/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar
-Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m
-Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true
-Djava.awt.headless=true -XX:+UseG1GC
-Djava.protocol.handler.pkgs=sun.net.www.protocol
-Dnifi.properties.file.path=/appl/idc/nifi-1.1.0/./conf/nifi.properties
-Dnifi.bootstrap.listen.port=55731 -Dapp=NiFi
-Dorg.apache.nifi.bootstrap.config.log.dir=/appl/idc/nifi-1.1.0/logs
org.apache.nifi.NiFi
2016-12-15 01:25:06,323 INFO [NiFi Bootstrap Command Listener]
org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for
Bootstrap requests on port 55732
2016-12-15 01:26:07,036 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,072 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,104 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,126 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,456 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,646 ERROR [NiFi logging handler] org.apache.nifi.StdErr
[Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting
with element 'template'. One of '{controllerService}' is expected.



--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/SOAP-Service-through-InvokeHTTP-tp13129p14237.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Hitendra Kashyap
In reply to this post by Andrew Psaltis
Hi Andrew,

I am unable to use the NiFi-soap with the latest version of nifi (1.1.0). after build and deploying nar file to lib when I restart nifi it doesn't start. bootstrap log tail has following:

2016-12-15 01:25:05,216 INFO [main] org.apache.nifi.bootstrap.Command Working Directory: /appl/idc/nifi-1.1.0
2016-12-15 01:25:05,217 INFO [main] org.apache.nifi.bootstrap.Command Command: java -classpath /appl/idc/nifi-1.1.0/./conf:/appl/idc/nifi-1.1.0/./lib/jcl-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/jul-to-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/log4j-over-slf4j-1.7.12.jar:/appl/idc/nifi-1.1.0/./lib/logback-classic-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/logback-core-1.1.3.jar:/appl/idc/nifi-1.1.0/./lib/nifi-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-documentation-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-framework-api-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-nar-utils-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-properties-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/nifi-runtime-1.1.0.jar:/appl/idc/nifi-1.1.0/./lib/slf4j-api-1.7.12.jar -Dorg.apache.jasper.compiler.disablejsr199=true -Xmx512m -Xms512m -Dsun.net.http.allowRestrictedHeaders=true -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseG1GC -Djava.protocol.handler.pkgs=sun.net.www.protocol -Dnifi.properties.file.path=/appl/idc/nifi-1.1.0/./conf/nifi.properties -Dnifi.bootstrap.listen.port=55731 -Dapp=NiFi -Dorg.apache.nifi.bootstrap.config.log.dir=/appl/idc/nifi-1.1.0/logs org.apache.nifi.NiFi
2016-12-15 01:25:06,323 INFO [NiFi Bootstrap Command Listener] org.apache.nifi.bootstrap.RunNiFi Apache NiFi now running and listening for Bootstrap requests on port 55732
2016-12-15 01:26:07,036 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,072 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,104 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,126 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,456 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{controllerService}' is expected.
2016-12-15 01:26:07,646 ERROR [NiFi logging handler] org.apache.nifi.StdErr [Error] :627:15: cvc-complex-type.2.4.a: Invalid content was found starting with element 'template'. One of '{controllerService}' is expected.
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Rashmi MR
In reply to this post by Jeff
how can i read data from soap service. What are the configuration properties
and where can i set the soap url and wsdl url uisng invokehttp?



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Code Novice
In reply to this post by mliem90
I've only just subscribed to the [hidden email] mailing list and today
was my first time even knowing that a thing like 'maling lists' existed.
Regardless, I'm posting my question in hopes to seek out how you eventually
managed to get your InvokeHTTP Processor to work with SOAP as I have not yet
been able to do so.

I posted my original question on this matter on Stackoverflow  here
<https://stackoverflow.com/questions/49457764/nifi-how-to-use-invokehttp-processor-with-soap>
. I also referenced this post as it is the only place I am seeing somebody
use the InvokeHTTP with SOAP.

I'm using SOAPUI to verify that everything works and it does as I am getting
the appropriate XML response back. However I am unable to accomplish this in
NiFi. My guess is that I don't know what to call my dynamic properties. I
also don't fully understand what data I would add to the Value of my dynamic
properties.

<http://apache-nifi-developer-list.39713.n7.nabble.com/file/t939/SOAP.png>



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: SOAP Service through InvokeHTTP

Mike Thomsen
Been a while since I've done anything with SOAP, but the way I remember it
is it's just a POST operation with a XML payload at the end of the day. If
you have configured the endpoint properly in the processor's URL property
and set it to do a POST, the *flowfile content* should be where you put the
SOAP request body. In the example of SO, it looked like you were using a
dynamic property to store the soap:Envelope segment, and I don't think
that's going to work. Let me know if you make any progress with the method
I suggested.

Mike

On Fri, Mar 23, 2018 at 5:55 PM, Code Novice <[hidden email]> wrote:

> I've only just subscribed to the [hidden email] mailing list and
> today
> was my first time even knowing that a thing like 'maling lists' existed.
> Regardless, I'm posting my question in hopes to seek out how you eventually
> managed to get your InvokeHTTP Processor to work with SOAP as I have not
> yet
> been able to do so.
>
> I posted my original question on this matter on Stackoverflow  here
> <https://stackoverflow.com/questions/49457764/nifi-how-
> to-use-invokehttp-processor-with-soap>
> . I also referenced this post as it is the only place I am seeing somebody
> use the InvokeHTTP with SOAP.
>
> I'm using SOAPUI to verify that everything works and it does as I am
> getting
> the appropriate XML response back. However I am unable to accomplish this
> in
> NiFi. My guess is that I don't know what to call my dynamic properties. I
> also don't fully understand what data I would add to the Value of my
> dynamic
> properties.
>
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/t939/SOAP.png>
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>
Reply | Threaded
Open this post in threaded view
|

RE: SOAP Service through InvokeHTTP

Bruce Jackman
Mike is correct, Put your SOAP XML in the body (I use a ReplaceText Processor), add your Attributes to Send and then Invoke using the correct URL:
Here is the SOAP section from one of my flows:

Step 1) Replace Text Processor
        Search Value (?s)(^.*$)
        Replacement Value <?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"> <soap:Header> <ApiServiceHeader xmlns="http://xxxxxxx"> <UserName>${UserName}</UserName> <Password>${Password}</Password> <ClientToken>${ClientToken}</ClientToken> <Client>${Client}</Client> </ApiServiceHeader> </soap:Header> <soap:Body> <GetBuildingsDataRequest xmlns="http://xxxxxxx"> <OrderNumber>${OrderNumber}</OrderNumber> <LastChangedDate>${now():format("yyyy-MM-dd'T'hh:mm:ss")}</LastChangedDate> </GetBuildingsDataRequest> </soap:Body> </soap:Envelope>
        Character Set UTF-8
        Maximum Buffer Size 1 MB
        Replacement Strategy Regex Replace
        Evaluation Mode Entire text

Step 2) Update Attribute: (may not be necessary for your flow)
        Store State Do not store state
        Stateful Variables Initial Value No value set
        SoapAction ${SoapActionRoot}SubmitGetBuildingsData

Step 3) Invoke HTTP
        HTTP Method POST
        Remote URL https://XXXXXX\WebDeliveryMembersServices/DataDeliveryServices.asmx
        SSL Context Service No value set
        Connection Timeout 30 min
        Read Timeout 30 min
        Include Date Header True
        Follow Redirects True
        Attributes to Send (Host|SoapAction)
        Basic Authentication Username No value set
        Basic Authentication Password No value set
        Proxy Host No value set
        Proxy Port No value set
        Proxy Username No value set
        Proxy Password No value set
        Put Response Body In Attribute No value set
        Max Length To Put In Attribute 2048
        Use Digest Authentication false
        Always Output Response false
        Trusted Hostname No value set
        Add Response Headers to Request false
        Content-Type text/xml; charset=utf-8
        Send Message Body true
        Use Chunked Encoding false
        Penalize on "No Retry" false

It was pretty easy once I got the Xml structure correct.

Hope this helps
Bruce

-----Original Message-----
From: Mike Thomsen [mailto:[hidden email]]
Sent: Saturday, March 24, 2018 4:17 AM
To: [hidden email]
Subject: Re: SOAP Service through InvokeHTTP

Been a while since I've done anything with SOAP, but the way I remember it is it's just a POST operation with a XML payload at the end of the day. If you have configured the endpoint properly in the processor's URL property and set it to do a POST, the *flowfile content* should be where you put the SOAP request body. In the example of SO, it looked like you were using a dynamic property to store the soap:Envelope segment, and I don't think that's going to work. Let me know if you make any progress with the method I suggested.

Mike

On Fri, Mar 23, 2018 at 5:55 PM, Code Novice <[hidden email]> wrote:

> I've only just subscribed to the [hidden email] mailing list and
> today was my first time even knowing that a thing like 'maling lists'
> existed.
> Regardless, I'm posting my question in hopes to seek out how you
> eventually managed to get your InvokeHTTP Processor to work with SOAP
> as I have not yet been able to do so.
>
> I posted my original question on this matter on Stackoverflow  here
> <https://stackoverflow.com/questions/49457764/nifi-how-
> to-use-invokehttp-processor-with-soap>
> . I also referenced this post as it is the only place I am seeing
> somebody use the InvokeHTTP with SOAP.
>
> I'm using SOAPUI to verify that everything works and it does as I am
> getting the appropriate XML response back. However I am unable to
> accomplish this in NiFi. My guess is that I don't know what to call my
> dynamic properties. I also don't fully understand what data I would
> add to the Value of my dynamic properties.
>
> <http://apache-nifi-developer-list.39713.n7.nabble.com/file/t939/SOAP.
> png>
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/
>