URL failed to parse when redirecting, and contains [] characters

[patrixl]

And possibly other characters...

I realize this is an issue with the puri library, however it might be faster to work around this in the radiance code rather than wait for puri to get updated (if ever).

When calling merge-url with the referrer:

Because radiance uses form-encoded post/get values, and I need some arrays, I have some form elements containing the [] characters.

I also end up having some URLs that are built using the same variables and arrays, and from these URLs there are forms that POST to the API. So far so good.

When POSTing to the API, and using the recommended method of checking (post/get "browser"), which should redirect back to the page containing the form when there's an error, I instead get an error message ("an unexpected error has occured").

Looking at the logs on the REPL , I see that puri:parse-uri had trouble parsing the url, probably during the merge-url call that generates the redirect url here:

radiance/api.lisp

Line 155 in a72e293

(redirect (merge-url (referer) :parameters `(("error" . ,(princ-to-string message)))))

Simple example:
(puri:parse-uri "http://localhost:9090/test?a=b&c[]=d")

Error:

Parse error:URI "http://localhost:9090/test?a=b&c[]=d" contains illegal character #\[ at position 32.

With URL-encoding:

(puri:parse-uri "http://localhost:9090/test?a=b&c%5b%5d=d")

It works fine but my browser will of course re-write %5b%5d back to [].

[Shinmera]

Puri is being strict with the URL representation, so it's not wrong in complaining about the illegal characters. Technically the browser is in the wrong for not encoding these characters.

Unfortunately, the referrer is a header that the client could potentially stuff anything into, so I'm not 100% clear on what should be done here.

[patrixl]

You're right that puri is strict. I think the http standard doesn't mandate arg[] as meaning "the arg[] parameter is an array", but that is how PHP was doing it, seems like that's how Hunchentoot is doing it, which is inherited by Radiance (through post/get) when using Hunchentoot as backend.

I suppose I'll have to rethink my approach (i.e. use POST instead of GET) and avoid having URLs with these characters, as I don't know any modern browser that doesn't change the URL-encoded strings back to the actual chars, how times have changed..

[Shinmera]

That's not a Hunchentoot thing, it's a Radiance thing.

One solution would be to include a way to turn a lenient URL into a strict one, meaning to destructure the URL without caring about what should be encoded or not.