Accept urls with lengths larger than 2040 characters

[eoss-tw]

Hi, I know due to compability the limitation makes sense but we want to use kutt for service urls which encode their config in the urls (e.g. kibana). Hence urls can grow quite large and with modern browsers this is fine. I made a quick change for v1 resp v2 of your api but got an error resolving the urls. I can post the urls into the db but got a 500 with the new link. I am using nginx as proxy, maybe this could be the bottleneck. Is it possible to increase logging to dig any further?

Thanks
Thilo

[eoss-tw]

Ok, nginx was indeed the bottle neck. Maybe you can exclude the limit to the config for the table generation/validator.
Thilo

[poeti8]

The problem with making it configurable is that the table needs to be updated every time. What's the ideal max limit for you?

[eoss-tw]

It’s difficult to say because the url encodes json structures which increase over time. It started with ca. 3k and reaches easily > 20k. In the database var char can be easily replaced as text - the overhead should be very small. The validator can have a length compatibility tag which can be per default set to enabled. Especially with the api post the url length should not be of importance - I think. Am 16.02.2020 um 12:49 schrieb Pouria Ezzati <[email protected]<mailto:[email protected]>>: The problem with making it configurable is that the table needs to be updated every time. What's the ideal max limit for you? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#279?email_source=notifications&email_token=AFMKTOLJ6H56S2ZQE5BTOJTRDER4JA5CNFSM4KTWJ4L2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEL4ETMY#issuecomment-586697139>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFMKTOL42FMUEUT3G5HTUATRDER4JANCNFSM4KTWJ4LQ>.

[trgwii]

This idea sounds terrible, I don't think I would want it to support > 2 000 chars in URLs, and for sure not > 8 000.

Even the nodejs core HTTP implementation supports at most 50 000 chars in total in the entire request header payload, so if they can increase even more you'll eventually run into platform problems anyway (meaning you would need to find a service NOT written in node).

[eoss-tw]

I know that the proposed mechanism isnt an elegant one but I guess there are more use cases outside which would benefit from extra long urls. An idea would be to use compatible strict mode and experimental mode to provide stability for the one group and extensibility for the other one. On the backend side it should not matter/restrict anything. The app I am using (globus) is using node which stores its state in the url and kibana uses the url, too. Hacking the project did work for me. I hoped that it would make sense for more people, too.

…

________________________________ Von: trgwii <[email protected]> Gesendet: Thursday, September 24, 2020 2:44:34 PM An: thedevs-network/kutt <[email protected]> Cc: Thilo Wehrmann <[email protected]>; Author <[email protected]> Betreff: Re: [thedevs-network/kutt] Accept urls with lengths larger than 2040 characters (#279) This idea sounds terrible, I don't think I would want it to support > 2 000 chars in URLs, and for sure not > 8 000. Even the nodejs core HTTP implementation supports at most 50 000 chars in total in the entire request header payload, so if they can increase even more you'll eventually run into platform problems anyway (meaning you would need to find a service NOT written in node). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#279 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFMKTONF6IOGWQIFTTGLIGLSHM5LFANCNFSM4KTWJ4LQ>.

[trgwii]

On the backend side it should not matter/restrict anything.

I disagree, because max data lengths matter for relational databases, they affect the overall size / size per row, as well as query speeds.

[eoss-tw]

Most modern db systems support TEXT type to support dynamic strings. Because url is just stored/not indexed, speed should not be an issue, too – but I assume discussion is floating a little bit away. Von: trgwii <[email protected]> Antworten an: thedevs-network/kutt <[email protected]> Datum: Freitag, 25. September 2020 um 14:13 An: thedevs-network/kutt <[email protected]> Cc: Thilo Wehrmann <[email protected]>, Author <[email protected]> Betreff: Re: [thedevs-network/kutt] Accept urls with lengths larger than 2040 characters (#279) On the backend side it should not matter/restrict anything. I disagree, because max data lengths matter for relational databases, they affect the overall size / size per row, as well as query speeds. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#279 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AFMKTOMBGMEGMLHVXENSSMTSHSCPTANCNFSM4KTWJ4LQ>.

[chq-matteo]

I was wondering if we could revisit this

The problem with making it configurable is that the table needs to be updated every time. What's the ideal max limit for you?

I see, that's probably not great, we probably don't want to update this often

However, based on rfc9110, maybe it's worth bumping the limit to 8000?

It is RECOMMENDED that all senders and recipients support, at a minimum, URIs with lengths of 8000 octets in protocol elements. Note that this implies some structures and on-wire representations (for example, the request line in HTTP/1.1) will necessarily be larger in some cases.

refs:

• https://www.rfc-editor.org/rfc/rfc9110#section-4.1-5
• https://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-a-url-in-different-browsers