Limit the Limit

[MarinPostma]

There are actually no limitation on the limit param for the search. It is a vulnerability that needs to be addressed, since it can be used to perform DOS attack, and scrap the entirety of the database in one call. I propose to limit the limit to 1000 documents per query.

[curquiza]

This problem is also present with the offset parameter.
Since we have to define the behavior (returning error or not?) this might involve a spec. I let @gmourier see when we can integrate it into the roadmap.

[gmourier]

The offset parameter is a bit different because it is not really limited, it could go outside the window of possible documents and therefore return an empty answer without giving an error.

I think I'll have to update the paginated search specification because I don't seem to have informed the behavior of these parameters in the case of an unwanted value.

Like you said @curquiza, it could be silent and force the limit to 1000 🤔

Thanks!

[MarinPostma]

I think silent is the way to go too, we just clamp the number of documents that can be retrieved.

[Kerollmops]

The offset and limit with high values can trigger the same kind of behavior and can easily be used by an attacker to DDOS the server.

It could be great to clamp the limit to 1000 but what if an attacker decide to fetch documents at offset 1000 too? The engine will simply have to compute 2000 documents. A clamp on both values can, indeed, do the job.

[MarinPostma]

I gave that some thought, and I think that what we need to clamp is the maximum retrievable documents. This takes the limit and the offset into account. i.e, if the maximum retrievable documents is set to 1000, then you can either query with limit=1000 and offset=0 or with ,limit =1 and offset=999

[gmourier]

Following the v0.28 release, this discussion is now locked.

We encourage you to create a new thread if needed.

Thank you ✨

Ps: La bise @MarinPostma 😎