Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Add basic ESearch support #333

Draft
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

nevans
Copy link
Collaborator

@nevans nevans commented Sep 26, 2024

No description provided.

@nevans
Copy link
Collaborator Author

nevans commented Oct 8, 2024

FWIW: I discovered in testing that Yahoo does not return ESEARCH results when RETURN (PARTIAL 1:500) is used, contrary to the PARTIAL RFC that was written by Yahoo engineers!

@nevans nevans added this to the v0.6 milestone Oct 8, 2024
@nevans nevans added the IMAP4rev2 Requirement for IMAP4rev2, RFC9051 label Oct 14, 2024
@nevans nevans force-pushed the basic-esearch-support branch 4 times, most recently from 59137e9 to 7dc2006 Compare October 25, 2024 21:21
nevans added a commit to nevans/net-imap that referenced this pull request Nov 8, 2024
This affects `#search`, `#uid_search`, `#sort`, `#uid_sort`, `#thread`,
and `#uid_thread`.

Prior to this, sending a parenthesized list in the search criteria for
any of these commands required the use of strings, which are converted
to `RawData`, which has security implications with untrusted inputs.

With this change, arrays will only be converted into SequenceSet when
_every_ element in the array is a valid SequenceSet input.  Otherwise,
the array will be left alone, which allows us to send parenthesized
lists without using strings and RawData.

For example, some searches this change enables:

* Combining criteria to pass into `OR`, `NOT`, `FUZZY`, etc.
  * `search(["not", %w(flagged unread)])`
    converts to: `SEARCH not (flagged unread)`
* Adding return options (we should also add a return kwarg).
  * `uid_search(["RETURN", ["PARTIAL", 1..50], "UID", 12345..67890])`
    converts to: `UID SEARCH RETURN (PARTIAL 1:50) UID 12345:67890`
  * Note that `PARTIAL` supports negative ranges, which can't be coerced
    to SequenceSet.  They'll need to be sent as strings, for now.
  * Note that searches with return options should return ESEARCH
    results, which are currently unsupported.  See ruby#333.

This _should_ be backward compatible: previously these inputs would
raise an exception.
@nevans nevans force-pushed the basic-esearch-support branch 2 times, most recently from d86e849 to efd2760 Compare November 8, 2024 23:02
@nevans nevans force-pushed the basic-esearch-support branch 5 times, most recently from 26211ba to fd7b0f9 Compare November 25, 2024 17:02
nevans and others added 5 commits November 25, 2024 12:08
For new data structs, I don't want to commit to supporting the entire
Struct API and I'd prefer frozen by default.  `Data` is exactly what I
want but it's not available until ruby 3.2.

So this adds a DataLite class that closely matches ruby 3.2's Data
class and can be a drop-in replacement for Data.  Net::IMAP::Data is an
alias for Net::IMAP::DataLite, so when we remove this implementation,
the constant will resolve to ruby's ::Data.  The most noticable
incompatibility is that member names must be valid local variable names.

Ideally, we wouldn't define this on newer ruby versions at all, but that
breaks the YAML serialization for our test fixtures.  So, on newer
versions of ruby, this class inherits from `Data` and only reimplements
the two methods that are needed for YAML serialization.  This serves the
additional purpose of ensuring that the same tests pass for both the
core `Data` class and our reimplementation.

Most of the test code and some of the implementation code has been
copied from the polyfill-data gem and updated so that they use "Data" as
it is resolved inside the "Net::IMAP" namespace.  Copyright notices have
been added to the appropriate files to satisfy the MIT license terms.

Co-authored-by: Jim Gay <[email protected]>
The new `CommandData` superclass uses `Data` to add the pattern matching
and equality methods while also simplifying the implementation.

Specifically, I wanted RawData#deconstruct for the basic `ESEARCH`
support branch.  It seemed reasonable to apply the same change to all of
the internal command data classes.

Please note: this does change these objects to be frozen.  However,
these classes are explicitly undocumented and considered "internal", so
this will _not_ be treated as a "breaking change".
Parses +ESEARCH+ into ESearchResult, with support for:
* RFC4466 syntax
* RFC4731 `ESEARCH`
* RFC5267 `CONTEXT=SEARCH`
* RFC6203 `SEARCH=FUZZY`
* RFC9394 `PARTIAL`

For compatibility, `ESearchResult#to_a` returns an array of integers
(sequence numbers or UIDs) whenever any `ALL` or `PARTIAL` result is
available.
If the server returns both `ESEARCH` and `SEARCH`, both are cleared from
the responses hash, but only the `ESEARCH` is returned.

When the server doesn't send any search responses:  If return options
are passed, return an empty ESearchResult.  It will have the appropriate
`tag` and `uid` values, but no `data`.  Otherwise return an empty
`SearchResult` (changed from empty array).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IMAP4rev2 Requirement for IMAP4rev2, RFC9051
Development

Successfully merging this pull request may close these issues.

1 participant