Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add basic ESearch support #333

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

nevans
Copy link
Collaborator

@nevans nevans commented Sep 26, 2024

No description provided.

@nevans
Copy link
Collaborator Author

nevans commented Oct 8, 2024

FWIW: I discovered in testing that Yahoo does not return ESEARCH results when RETURN (PARTIAL 1:500) is used, contrary to the PARTIAL RFC that was written by Yahoo engineers!

@nevans nevans added this to the v0.6 milestone Oct 8, 2024
@nevans nevans added the IMAP4rev2 Requirement for IMAP4rev2, RFC9051 label Oct 14, 2024
@nevans nevans force-pushed the basic-esearch-support branch 4 times, most recently from 59137e9 to 7dc2006 Compare October 25, 2024 21:21
nevans added a commit to nevans/net-imap that referenced this pull request Nov 8, 2024
This affects `#search`, `#uid_search`, `#sort`, `#uid_sort`, `#thread`,
and `#uid_thread`.

Prior to this, sending a parenthesized list in the search criteria for
any of these commands required the use of strings, which are converted
to `RawData`, which has security implications with untrusted inputs.

With this change, arrays will only be converted into SequenceSet when
_every_ element in the array is a valid SequenceSet input.  Otherwise,
the array will be left alone, which allows us to send parenthesized
lists without using strings and RawData.

For example, some searches this change enables:

* Combining criteria to pass into `OR`, `NOT`, `FUZZY`, etc.
  * `search(["not", %w(flagged unread)])`
    converts to: `SEARCH not (flagged unread)`
* Adding return options (we should also add a return kwarg).
  * `uid_search(["RETURN", ["PARTIAL", 1..50], "UID", 12345..67890])`
    converts to: `UID SEARCH RETURN (PARTIAL 1:50) UID 12345:67890`
  * Note that `PARTIAL` supports negative ranges, which can't be coerced
    to SequenceSet.  They'll need to be sent as strings, for now.
  * Note that searches with return options should return ESEARCH
    results, which are currently unsupported.  See ruby#333.

This _should_ be backward compatible: previously these inputs would
raise an exception.
@nevans nevans force-pushed the basic-esearch-support branch 2 times, most recently from d86e849 to efd2760 Compare November 8, 2024 23:02
nevans and others added 8 commits November 11, 2024 11:01
For new data structs, I'd prefer frozen by default and I don't want to
support the entire Struct API.  Data is perfect, but it's not available
until ruby 3.2.

So this adds a DataLite class that closely matches ruby 3.2's Data
class, and it can be a drop-in replacement for Data.  Net::IMAP::Data
is an alias for Net::IMAP::DataLite, so when we remove our
implementation, the constant will resolve to ruby's ::Data.

Ideally, we wouldn't define this on newer ruby versions at all, but that
breaks the YAML serialization for our test fixtures.

The tests (and some of the implementation) have been copied from the
polyfill-data gem and updated so that they use "Data" as it is resolved
inside the "Net::IMAP" namespace.  Copyright notices have been added to
the appropriate files to satisfy the MIT license terms.

Co-authored-by: Jim Gay <[email protected]>
Parses +ESEARCH+ into ESearchResult, with support for:
* RFC4466 syntax
* RFC4731 `ESEARCH`
* RFC5267 `CONTEXT=SEARCH`
* RFC6203 `SEARCH=FUZZY`
* RFC9394 `PARTIAL`

For compatibility, `ESearchResult#to_a` returns an array of integers
(sequence numbers or UIDs) whenever any `ALL` or `PARTIAL` result is
available.
If the server returns both `ESEARCH` and `SEARCH`, both are cleared from
the responses hash, but only the `ESEARCH` is returned.

When the server doesn't send any search responses:  If return options
are passed, return an empty ESearchResult.  It will have the appropriate
`tag` and `uid` values, but no `data`.  Otherwise return an empty
`SearchResult` (changed from empty array).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IMAP4rev2 Requirement for IMAP4rev2, RFC9051
Development

Successfully merging this pull request may close these issues.

1 participant