HTTP Request and x-www-form-urlencoded

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

HTTP Request and x-www-form-urlencoded

4D Tech mailing list
I have very little experience with HTTP Request. I’m calling an external REST service using 4D’s HTTP Request command. The REST service requires that the content-type be x-www-form-urlencoded. The “contents” parameter of the HTTP Request command supports sending a 4D Object (JSON) but that is failing (unknown parameter) when I submit it to the REST service. I suspect that is the REST service is expecting x-www-form-urlencoded rather than JSON.

I’ve successfully tested the REST service using Postman and I can copy the encoded URL from Postman and send it successfully from 4D HTTP Request instead or JSON. So I just need to build the encoded URL on the 4D side. It’s tedious to do though, and I think it’s convenient to use an object on the 4D side to manage the key value pairs. So I’m wondering is there an ‘easy’ way to convert a JSON object into an encoded URL?

Hope this makes sense,

Tom Benedict
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
Tom,

I'm not sure but maybe this will help you?
https://github.com/miyako/4d-tips-encode-uri

Regards,
 
Narinder Chandi,
ToolBox Systems Ltd.
 
I am available for new consulting opportunities…
http://4d.1045681.n5.nabble.com/ANN-4D-Developer-Available-td5765443.html
--

-----Original Message-----
From: 4D_Tech <[hidden email]> on behalf of 4D Tech Mailing List <[hidden email]>
Reply-To: 4D Tech Mailing List <[hidden email]>
Date: Saturday, 12 October 2019 at 20:54
To: 4D Tech Mailing List <[hidden email]>
Cc: Tom Benedict <[hidden email]>
Subject: HTTP Request and x-www-form-urlencoded

    I have very little experience with HTTP Request. I’m calling an external REST service using 4D’s HTTP Request command. The REST service requires that the content-type be x-www-form-urlencoded. The “contents” parameter of the HTTP Request command supports sending a 4D Object (JSON) but that is failing (unknown parameter) when I submit it to the REST service. I suspect that is the REST service is expecting x-www-form-urlencoded rather than JSON.
   
    I’ve successfully tested the REST service using Postman and I can copy the encoded URL from Postman and send it successfully from 4D HTTP Request instead or JSON. So I just need to build the encoded URL on the 4D side. It’s tedious to do though, and I think it’s convenient to use an object on the 4D side to manage the key value pairs. So I’m wondering is there an ‘easy’ way to convert a JSON object into an encoded URL?
   
    Hope this makes sense,
   
    Tom Benedict
    **********************************************************************
    4D Internet Users Group (4D iNUG)
    Archive:  http://lists.4d.com/archives.html
    Options: https://lists.4d.com/mailman/options/4d_tech
    Unsub:  mailto:[hidden email]
    **********************************************************************


**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
In reply to this post by 4D Tech mailing list
No easy way in 4D because there is no built in function to do url encoding. See feature request at the link below along with some code to implement it.

https://forums.4d.com/Post/EN/17578327/1/17591881

John DeSoi, Ph.D.



> On Oct 12, 2019, at 1:54 PM, Tom Benedict via 4D_Tech <[hidden email]> wrote:
>
> I’ve successfully tested the REST service using Postman and I can copy the encoded URL from Postman and send it successfully from 4D HTTP Request instead or JSON. So I just need to build the encoded URL on the 4D side. It’s tedious to do though, and I think it’s convenient to use an object on the 4D side to manage the key value pairs. So I’m wondering is there an ‘easy’ way to convert a JSON object into an encoded URL?

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
Thanks John,

Narinda Chandi points out Miyako’s URL_Encode component https://github.com/miyako/4d-tips-encode-uri <https://github.com/miyako/4d-tips-encode-uri>, which appears to be a product of the forum thread you referenced. In that component Miyako discusses and supports 5 ways to encode, each with different rules. At this point I just manually concatenate all the keys and values into a URL encoding each key or value as needed. The HTTP Request command docs state that the contents parameter is automatically encoded but evidently not in the way that I need.

I must confess I don’t understand any of this stuff. I recall struggling years ago trying to make an HTTP POST work with 4DIC. Hours of trial and error, because I simply didn’t (and still don’t) understand the underlying concepts. I figured it might be easier now that we have HTTP Request. I was kind hoping there’d be a nice ORDA way to populate a “content" object and send it on it’s way. If that’s possible, I’ve missed it.

Thanks again,

Tom Benedict

> On Oct 12, 2019, at 13:49, John DeSoi via 4D_Tech <[hidden email]> wrote:
>
> No easy way in 4D because there is no built in function to do url encoding. See feature request at the link below along with some code to implement it.
>
> https://forums.4d.com/Post/EN/17578327/1/17591881
>
> John DeSoi, Ph.D.
>
>
>
>> On Oct 12, 2019, at 1:54 PM, Tom Benedict via 4D_Tech <[hidden email]> wrote:
>>
>> I’ve successfully tested the REST service using Postman and I can copy the encoded URL from Postman and send it successfully from 4D HTTP Request instead or JSON. So I just need to build the encoded URL on the 4D side. It’s tedious to do though, and I think it’s convenient to use an object on the 4D side to manage the key value pairs. So I’m wondering is there an ‘easy’ way to convert a JSON object into an encoded URL?
>
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
In reply to this post by 4D Tech mailing list
Hi Narinder,

Thanks again for your quick response!

>I'm not sure but maybe this will help you?
>
> >https://github.com/miyako/4d-tips-encode-uri <https://github.com/miyako/4d-tips-encode-uri>
I found that component earlier today and it does help a bit. I was looking for a way to fill an object with key value pairs which HTTP Request would then convert into the right content-type.

Tom Benedict

> On Oct 12, 2019, at 13:45, Narinder Chandi via 4D_Tech <[hidden email]> wrote:
>
> Tom,
>
> I'm not sure but maybe this will help you?
> https://github.com/miyako/4d-tips-encode-uri
>
> Regards,
>
> Narinder Chandi,
> ToolBox Systems Ltd.
>
> I am available for new consulting opportunities…
> http://4d.1045681.n5.nabble.com/ANN-4D-Developer-Available-td5765443.html
> --
>
> -----Original Message-----
> From: 4D_Tech <[hidden email]> on behalf of 4D Tech Mailing List <[hidden email]>
> Reply-To: 4D Tech Mailing List <[hidden email]>
> Date: Saturday, 12 October 2019 at 20:54
> To: 4D Tech Mailing List <[hidden email]>
> Cc: Tom Benedict <[hidden email]>
> Subject: HTTP Request and x-www-form-urlencoded
>
>    I have very little experience with HTTP Request. I’m calling an external REST service using 4D’s HTTP Request command. The REST service requires that the content-type be x-www-form-urlencoded. The “contents” parameter of the HTTP Request command supports sending a 4D Object (JSON) but that is failing (unknown parameter) when I submit it to the REST service. I suspect that is the REST service is expecting x-www-form-urlencoded rather than JSON.
>
>    I’ve successfully tested the REST service using Postman and I can copy the encoded URL from Postman and send it successfully from 4D HTTP Request instead or JSON. So I just need to build the encoded URL on the 4D side. It’s tedious to do though, and I think it’s convenient to use an object on the 4D side to manage the key value pairs. So I’m wondering is there an ‘easy’ way to convert a JSON object into an encoded URL?
>
>    Hope this makes sense,
>
>    Tom Benedict
>    **********************************************************************
>    4D Internet Users Group (4D iNUG)
>    Archive:  http://lists.4d.com/archives.html
>    Options: https://lists.4d.com/mailman/options/4d_tech
>    Unsub:  mailto:[hidden email]
>    **********************************************************************
>
>
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
Tom,

    >https://github.com/miyako/4d-tips-encode-uri       
    >I found that component earlier today and it does help a bit. I was looking for a way to fill an object with key value pairs which HTTP Request would then convert into the right content-type.

Unfortunately HTTP Request is sadly still lacking this capability and, to the best of my knowledge, it is not a part of the HTTP client command set or the Internet Command set which is rather a glaring oversight IMO. It is pretty standard stuff as far as HTTP requests go. 4D really need to address this.

You could perhaps create your own wrapper around HTTP Request and combine that with Miyako's code if that might solve your problem?

Regards,
 
Narinder Chandi,
ToolBox Systems Ltd.
 
I am available for new consulting opportunities…
http://4d.1045681.n5.nabble.com/ANN-4D-Developer-Available-td5765443.html
--



**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
Tom,

This may have been mentioned already, but I just remembered that it is possible to do this encoding natively with 4D's XML and XSLT Transformation commands and I wrote a pair of encode/decode functions several years ago that worked no problem.

However, XSLT has been deprecated in v17. So, unless you are using a prior version of 4D (and I get the impression that you are not as you mention ORDA in a previous post) then  you are out of luck. However, if you want the encode/decoded functions let me know and I can send them to privately.

Regards,
 
Narinder Chandi,
ToolBox Systems Ltd.
 
I am available for new consulting opportunities…
http://4d.1045681.n5.nabble.com/ANN-4D-Developer-Available-td5765443.html
--

-----Original Message-----
From: 4D_Tech <[hidden email]> on behalf of 4D Tech Mailing List <[hidden email]>
Reply-To: 4D Tech Mailing List <[hidden email]>
Date: Saturday, 12 October 2019 at 23:12
To: 4D Tech Mailing List <[hidden email]>
Cc: Narinder Chandi <[hidden email]>
Subject: Re: HTTP Request and x-www-form-urlencoded

    Tom,
   
        >https://github.com/miyako/4d-tips-encode-uri       
        >I found that component earlier today and it does help a bit. I was looking for a way to fill an object with key value pairs which HTTP Request would then convert into the right content-type.
   
    Unfortunately HTTP Request is sadly still lacking this capability and, to the best of my knowledge, it is not a part of the HTTP client command set or the Internet Command set which is rather a glaring oversight IMO. It is pretty standard stuff as far as HTTP requests go. 4D really need to address this.
   
    You could perhaps create your own wrapper around HTTP Request and combine that with Miyako's code if that might solve your problem?
   
    Regards,
     
    Narinder Chandi,
    ToolBox Systems Ltd.
     
    I am available for new consulting opportunities…
    http://4d.1045681.n5.nabble.com/ANN-4D-Developer-Available-td5765443.html
    --
   
   
   
    **********************************************************************
    4D Internet Users Group (4D iNUG)
    Archive:  http://lists.4d.com/archives.html
    Options: https://lists.4d.com/mailman/options/4d_tech
    Unsub:  mailto:[hidden email]
    **********************************************************************


**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
In reply to this post by 4D Tech mailing list
http://forums.4d.fr/Post/FR/18982177/1/18982372#18982372

or

https://github.com/miyako/4d-plugin-curl-v2
https://github.com/miyako/4d-plugin-houdini

in 4D code it would be something like:

C_TEXT:C284($1;$0;$escaped)

C_LONGINT:C283($i)
C_BOOLEAN:C305($shouldEscape)
C_BLOB:C604($data)

For ($i;1;Length:C16($1))
$char:=Substring:C12($1;$i;1)
$code:=Character code:C91($char)
$shouldEscape:=False:C215
Case of
: ($code=45)
: ($code=46)
: ($code>47) & ($code<58)
: ($code>63) & ($code<91)
: ($code=95)
: ($code>96) & ($code<123)
: ($code=126)
Else
$shouldEscape:=True:C214
End case
If ($shouldEscape)
CONVERT FROM TEXT:C1011($char;"utf-8";$data)
For ($j;0;BLOB size:C605($data)-1)
$hex:=String:C10($data{$j};"&x")
$escaped:=$escaped+"%"+Substring:C12($hex;Length:C16($hex)-1)
End for
Else
$escaped:=$escaped+$char
End if
End for

$0:=$escape

---

there is no native mapping to/from objects.



**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
In reply to this post by 4D Tech mailing list
Thanks again Narinder.

Indeed, XSLT is going away.

I think what I’ve put together with Miyako’s encoding component and some hard-coded concatenation will be sufficient for this one time, non-scalable, semi-hack. ;) If I need to do it more than once I’ll look at doing it the “right” way.

I appreciate all the suggestions you’ve offered.

Tom Benedict


> On Oct 12, 2019, at 15:50, Narinder Chandi via 4D_Tech <[hidden email]> wrote:
>
> Tom,
>
> This may have been mentioned already, but I just remembered that it is possible to do this encoding natively with 4D's XML and XSLT Transformation commands and I wrote a pair of encode/decode functions several years ago that worked no problem.
>
> However, XSLT has been deprecated in v17. So, unless you are using a prior version of 4D (and I get the impression that you are not as you mention ORDA in a previous post) then  you are out of luck. However, if you want the encode/decoded functions let me know and I can send them to privately.
>
> Regards,
>
> Narinder Chandi,
> ToolBox Systems Ltd.
>
> I am available for new consulting opportunities…
> http://4d.1045681.n5.nabble.com/ANN-4D-Developer-Available-td5765443.html
> --
>
> -----Original Message-----
> From: 4D_Tech <[hidden email]> on behalf of 4D Tech Mailing List <[hidden email]>
> Reply-To: 4D Tech Mailing List <[hidden email]>
> Date: Saturday, 12 October 2019 at 23:12
> To: 4D Tech Mailing List <[hidden email]>
> Cc: Narinder Chandi <[hidden email]>
> Subject: Re: HTTP Request and x-www-form-urlencoded
>
>    Tom,
>
>> https://github.com/miyako/4d-tips-encode-uri       
>> I found that component earlier today and it does help a bit. I was looking for a way to fill an object with key value pairs which HTTP Request would then convert into the right content-type.
>
>    Unfortunately HTTP Request is sadly still lacking this capability and, to the best of my knowledge, it is not a part of the HTTP client command set or the Internet Command set which is rather a glaring oversight IMO. It is pretty standard stuff as far as HTTP requests go. 4D really need to address this.
>
>    You could perhaps create your own wrapper around HTTP Request and combine that with Miyako's code if that might solve your problem?
>
>    Regards,
>
>    Narinder Chandi,
>    ToolBox Systems Ltd.
>
>    I am available for new consulting opportunities…
>    http://4d.1045681.n5.nabble.com/ANN-4D-Developer-Available-td5765443.html
>    --
>
>
>
>    **********************************************************************
>    4D Internet Users Group (4D iNUG)
>    Archive:  http://lists.4d.com/archives.html
>    Options: https://lists.4d.com/mailman/options/4d_tech
>    Unsub:  mailto:[hidden email]
>    **********************************************************************
>
>
> **********************************************************************
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:[hidden email]
> **********************************************************************

**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
In reply to this post by 4D Tech mailing list
Hi,

Can you post the code you are using ?

If you send JSON data, you probably have to send the right content type (content type "application/json").

>
> I’ve successfully tested the REST service using Postman and I can copy the encoded URL from Postman and send it successfully from 4D HTTP Request instead or JSON. So I just need to build the encoded URL on the 4D side. It’s tedious to do though, and I think it’s convenient to use an object on the 4D side to manage the key value pairs. So I’m wondering is there an ‘easy’ way to convert a JSON object into an encoded URL?

Was Postman successful to send a json request ?
Or did it only work with x-www-form-urlencoded (content type "application/x-www-form-urlencoded").

In the AWS component available here
        https://github.com/blegay/aws_component

There is a method
        AWS__arrToCanonicalQueryString

Which takes two text array pointers (key and values) and output a url encoded query string.

Bruno LEGAY
A&C Consulting




**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: HTTP Request and x-www-form-urlencoded

4D Tech mailing list
Bruno writes:

>If you send JSON data, you probably have to send the right content type (content type "application/json”).
>Was Postman successful to send a json request ?
>Or did it only work with x-www-form-urlencoded (content type "application/x-www-form-urlencoded").

Postman only works with x-www-form-urlencoded. Now that I understand a bit more about how all of this works, I think this explains everything.

>In the AWS component available here
> https://github.com/blegay/aws_component
>
>There is a method
> AWS__arrToCanonicalQueryString
>
>Which takes two text array pointers (key and values) and output a url encoded query string.

That’s sounds like what I’m looking for. It can help me make scaleable, reusable code.

Many thanks,

Tom Benedict
**********************************************************************
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:[hidden email]
**********************************************************************