Skip to content

feat: deliver Local Email App V2 with agent-first MCP and CLI contracts - #212

Merged
Wh1isper merged 18 commits into
mainfrom
refactor/local-mcp-app-v2
Jul 25, 2026
Merged

feat: deliver Local Email App V2 with agent-first MCP and CLI contracts#212
Wh1isper merged 18 commits into
mainfrom
refactor/local-mcp-app-v2

Conversation

@Wh1isper

@Wh1isper Wh1isper commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Summary

Deliver the Local Email App V2 architecture and finish its agent-first MCP and CLI contracts.

  • Separate the bounded mail workflow plane from user-operated account and credential management.
  • Make managed SQLite state and SecretStore the authorities for non-secret configuration and secrets respectively.
  • Preserve legacy TOML/environment/keyring behavior as an explicit compatibility mode and import source.
  • Route reads and mutations through application services with centralized limits, authority checks, deadlines, and provider-effect handling.
  • Ship and validate the loopback-only local management UI, managed CLI, metadata projection, packaging, and migration lifecycle.

Agent-first MCP surface

  • Keep a static MCP catalog with no account or credential management tools.
  • Add reviewed readOnlyHint, destructiveHint, idempotentHint, and openWorldHint annotations to every tool. These are planning hints, not authorization or safe-retry guarantees.
  • Expose stable, non-secret list_available_accounts discovery with bounded presentation DTOs instead of leaking internal configuration models.
  • Report the installed application version in MCP serverInfo.version.
  • Add schema-level and application/provider defense-in-depth validation for canonical positive decimal ASCII IMAP UIDs.
  • Keep tool descriptions, schemas, catalog fixtures, plugin guidance, published docs, and normative specs synchronized.

Machine-readable management CLI

Every finite management command now supports a leaf --json result mode: 10 config commands, 11 account commands, reset, and migrate-credentials.

  • Success emits one versioned JSON document with a stable command identifier, explicit data, and warnings.
  • Dispatched failures preserve nonzero exit status and emit bounded stable error codes rather than provider exceptions or local paths.
  • Presentation DTOs omit secrets, keyring locators, preview tokens, configuration paths, and database paths.
  • Credential migration reports bounded cleanup counts, completion state, and warning codes in JSON; exact keyring entries remain available only in human-facing text diagnostics.
  • Secret-writing JSON commands require stdin, while interactive import apply remains intentionally non-JSON.
  • Endpoint/state preflight runs before secret input, and connectivity checks expose typed, bounded remediation categories.
  • Destructive legacy reset now requires exact --confirm RESET confirmation.

Security and correctness hardening

  • Resolve only the selected account/role secret immediately before provider construction.
  • Revalidate account enablement and policy across long-running MCP sessions.
  • Centralize request, result, account-discovery, text, path, attachment, and provider-work limits.
  • Repeat UID validation at low-level IMAP read and mutation sinks before connection establishment.
  • Preserve inferred outbound attachment MIME main types and subtypes instead of coercing all attachments to application/*.
  • Retain exact caller-selected attachment destinations under the documented trusted-caller compatibility contract, with private-target, no-follow, race-resistant, and atomic replacement defenses.
  • Keep management JSON and diagnostics path-free and secret/locator-free while retaining actionable user-operated text flows.

Compatibility and migration

  • Historical MCP account setup is removed rather than renamed; agents hand setup back to the user-operated CLI or UI.
  • Managed mode uses an explicit bootstrap, revisioned catalog writes, soft account removal, secret lifecycle state, and deterministic legacy import.
  • Legacy mode remains available for compatibility and recovery, but MCP exposes no management surface in either mode.
  • Specifications and user documentation describe accepted boundaries separately from shipped compatibility behavior.

Related PR audit

Reviewed #209, #210, #184, and #211 for issues relevant to this refactor.

  • Absorbed low-risk shared defenses where they fit the accepted architecture, including canonical UID validation at multiple boundaries and MIME type correctness; existing centralized pagination, body, attachment, and filesystem protections are broader than the overlapping patches.
  • Did not import dynamic MCP permission scopes/read-only defaults from feat: capability-scope permission model (read-only default) #209 because V2 deliberately uses a static catalog with call-time policy enforcement.
  • Did not add forward_email from feat: add forward_email tool #184 or apply the dependency-only Ruff update from [pre-commit.ci] pre-commit autoupdate #211 as part of this contract change.
  • Did not replace the accepted exact-destination attachment contract with a new download-root policy; that would require an explicit policy/configuration migration rather than an incidental patch.

Validation

  • make check
  • make test — 1,056 passed, 5 deselected; 86.05% coverage
  • make docs-test
  • make test-e2e — 5 GreenMail tests passed
  • git diff --check
  • Independent code review after fixes: no remaining P0, P1, or P2 findings

Assisted-by: YAAI 261597362+yaai-bot@users.noreply.github.com

Wh1isper added 5 commits July 22, 2026 11:56
Finish managed lifecycle, explicit legacy import, application read boundaries, and end-to-end validation.

Assisted-by: YAAI <261597362+yaai-bot@users.noreply.github.com>
@Wh1isper Wh1isper changed the title refactor/local mcp app v2 feat: complete local email app lifecycle Jul 22, 2026
Run test and type-check jobs with the requested matrix interpreter. Replace terminal-width-dependent CLI assertions and isolate the non-POSIX test from process-global platform state.

Assisted-by: YAAI <261597362+yaai-bot@users.noreply.github.com>
@Wh1isper

Copy link
Copy Markdown
Owner Author

@codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 38c2941750

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread mcp_email_server/application/management.py Outdated
Comment thread mcp_email_server/application/management.py Outdated
@Wh1isper Wh1isper linked an issue Jul 22, 2026 that may be closed by this pull request
@Wh1isper
Wh1isper marked this pull request as draft July 22, 2026 13:03
@codecov

codecov Bot commented Jul 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.50258% with 1094 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.1%. Comparing base (81dd623) to head (a761d91).

Files with missing lines Patch % Lines
mcp_email_server/managed.py 73.2% 172 Missing and 85 partials ⚠️
mcp_email_server/emails/classic.py 73.9% 151 Missing and 46 partials ⚠️
mcp_email_server/cli.py 76.5% 91 Missing and 26 partials ⚠️
mcp_email_server/application/management.py 87.2% 55 Missing and 43 partials ⚠️
mcp_email_server/application/mutations.py 81.3% 58 Missing and 24 partials ⚠️
mcp_email_server/metadata_index.py 71.8% 51 Missing and 21 partials ⚠️
mcp_email_server/adapters/management.py 76.6% 52 Missing and 19 partials ⚠️
mcp_email_server/bootstrap.py 80.2% 30 Missing and 18 partials ⚠️
mcp_email_server/adapters/reads.py 80.2% 23 Missing and 15 partials ⚠️
mcp_email_server/adapters/mutations.py 77.0% 22 Missing ⚠️
... and 7 more
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #212     +/-   ##
=======================================
- Coverage   92.5%   83.1%   -9.5%     
=======================================
  Files         10      31     +21     
  Lines       2013    7980   +5967     
  Branches     310    1101    +791     
=======================================
+ Hits        1863    6635   +4772     
- Misses       107     956    +849     
- Partials      43     389    +346     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Wh1isper
Wh1isper marked this pull request as ready for review July 23, 2026 12:39
@Wh1isper Wh1isper changed the title feat: complete local email app lifecycle feat: deliver Local Email App V2 with agent-first MCP and CLI contracts Jul 23, 2026
@Wh1isper
Wh1isper force-pushed the refactor/local-mcp-app-v2 branch 2 times, most recently from ea317f3 to d436481 Compare July 25, 2026 06:13
- Refine the account-first UI, policy editing, and typed CLI/Web feedback.
- Make Linux managed credential updates atomic and remove repair-only states.
- Expand security documentation, regression coverage, and staged UI assets.

Assisted-by: YAAI <261597362+yaai-bot@users.noreply.github.com>
@Wh1isper
Wh1isper force-pushed the refactor/local-mcp-app-v2 branch from d436481 to 44d56c0 Compare July 25, 2026 09:00
Wh1isper added 2 commits July 25, 2026 19:11
- Remove catalog staging and activation so complete accounts are immediately usable.
- Preserve v1 TOML through a private bootstrap sidecar and guarded credential-aware cutover.
- Align CLI, local UI, specifications, documentation, and regression coverage.

Assisted-by: YAAI <261597362+yaai-bot@users.noreply.github.com>
@Wh1isper
Wh1isper merged commit 19e3b03 into main Jul 25, 2026
2 checks passed
@Wh1isper
Wh1isper deleted the refactor/local-mcp-app-v2 branch July 25, 2026 13:16
jbkjr pushed a commit to jbkjr/mcp-email-server that referenced this pull request Aug 21, 2026
Port upstream PR Wh1isper#224 (ai-zerolab/mcp-email-server b2032b0) by hand: the
upstream V2 architecture rewrite (Wh1isper#212) makes its commits uncherry-pickable,
so this reimplements the same behavior against the fork's pre-V2 code.

`save_path` is now optional on `download_attachment` (MCP tool, abstract
`EmailHandler`, `ClassicEmailHandler`, and `EmailClient`). When omitted, the
destination resolves under the current user's `Downloads/mcp-email-server`
directory using a sanitized, length-bounded attachment basename plus a
cryptographically random suffix. Path separators, traversal segments, and
Unicode control/format characters (RTL-override extension spoofing included)
are reduced to `_`, so a provider-supplied attachment name can never steer
the write outside that directory.

The new `emails/attachment_paths.py` holds the resolution and sanitization
helpers. On POSIX the default directory is created owner-only (0700) and
validated on reuse (symlink, group/world-writable, foreign ownership all
fail closed), and default downloads are written through
`O_CREAT|O_EXCL|O_NOFOLLOW` at mode 0600. The `~/Downloads` parent's own
permissions are left untouched.

An explicit `save_path` keeps its exact prior behavior: expanded, made
absolute, never rewritten, and written with the previous overwrite
semantics. Destination resolution happens before the IMAP connection, so an
unusable target fails before any message is fetched or decoded. The
`enable_attachment_download` gate and the sender allowlist on the read path
are both unchanged.

Skipped from upstream: the Windows Known Folder registry lookup and the
Windows filesystem-security plumbing, since this fork does not carry
upstream's Windows support work.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jbkjr pushed a commit to jbkjr/mcp-email-server that referenced this pull request Aug 21, 2026
Hand-port two upstream fixes into the fork's pre-V2 `classic.py`. Upstream
replaced its whole architecture in Wh1isper#212, so neither commit cherry-picks.

FETCH literal parsing (upstream Wh1isper#227, cbcd9b0):
`_check_email_content` / `_extract_raw_email` identified an email body by
`len(item) > 100`, so short messages were silently dropped into `failed_ids`
while a long line of protocol metadata could be mistaken for content. Identify
the body structurally instead: walk the response with `itertools.pairwise`,
match the FETCH `BODY[]`/`RFC822` marker, and validate the `{N}` literal
declaration against the payload length. `_check_email_content` is now a thin
wrapper over `_extract_raw_email`.

RFC 2971 ID compatibility (upstream Wh1isper#218, 296f360):
`_send_imap_id` sent the ID command unconditionally via aioimaplib's `id()`,
whose `ID ( "name" "value" )` spacing violates the RFC 2971 grammar, then
retried with a raw command on failure. Now it gates on the server advertising
the `ID` capability (reusing the fork's `_imap_capabilities`), sends only the
conformant compact parameter-list form, keeps a tagged non-OK reply non-fatal,
and adds `_discard_imap_after_id_failure` to abort the stream when a timeout,
cancellation or transport failure makes the response boundary unknown. Those
three cases now propagate (`TimeoutError` / new `ImapTransportError`) rather
than handing back a desynchronized connection; the awaitable-`new_tag()`
compat shim is gone since aioimaplib is pinned >=2.0.1.

Tests: upstream's regression tests adapted to the fork's mocking idioms
(`_imap_connect` / cached `imap_class`, OK-shaped `login`/`select`), plus
coverage for a bytearray literal without a size declaration, a non-write
transport being closed rather than aborted, and `_imap_connection` still
emitting exactly one ID on an ID-capable server. 675 tests pass.

Not ported: upstream's `docs/tools.md` and `docs/troubleshooting.md` edits
(V2-era docs that do not exist in this fork), and upstream's privacy-scrubbed
log messages (the fork logs response/exception detail throughout).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jbkjr pushed a commit to jbkjr/mcp-email-server that referenced this pull request Aug 21, 2026
Hand-port of two upstream fixes into the fork's pre-V2 EmailClient design
(upstream's V2 rewrite in Wh1isper#212 makes the commits uncherry-pickable).

Upstream PR Wh1isper#229 (cfaeece), issue Wh1isper#228 — separate header and envelope senders.
The bug was present here and worse than upstream's report: ClassicEmailHandler
interpolated `f"{full_name} <{email_address}>"`, so a full_name that is itself
an email address produced `a@b.com <a@b.com>`, which parseaddr reads back as
('', ''). compose_message then emitted that raw malformed string as the From
header, and aiosmtplib — deriving MAIL FROM by re-parsing From — sent an empty
`MAIL FROM:<>` null reverse-path.

The identity is now kept structured end to end. EmailClient takes optional
sender_name/sender_address keywords, formats self.sender once via formataddr
(which quotes display names containing specials), and exposes an
envelope_sender property returning the RFC 5321 addr-spec alone.
compose_message no longer re-parses its own From header, and derives the
Message-Id domain from the structured address; send_email passes
sender=envelope_sender explicitly. A non-ASCII addr-spec now yields a quoted
RFC 6532 mailbox instead of raising UnicodeEncodeError out of formataddr.

Upstream PR Wh1isper#219 (f6c05f9) — provider compatibility headers. web.de, 1&1 and
GMX answer 554 to messages carrying no sender-software identification. Adds
User-Agent and X-Mailer to composed messages; MIME-Version: 1.0 already comes
from the MIME constructors and is asserted to appear exactly once. Unlike
upstream, which hardcoded the values, these are configurable per account via
the new EmailServer.smtp_user_agent / smtp_x_mailer fields (and
MCP_EMAIL_SERVER_SMTP_USER_AGENT / _SMTP_X_MAILER), defaulting to
"mcp-email-server"; an empty string omits the header. A field validator
rejects control characters so a configured value cannot inject headers —
the injection concern upstream cited as its reason to keep them fixed.

Both fields default, so existing config files and the keyring migration path
load unchanged. The incoming server carries the same values because
save_to_mailbox composes drafts through the incoming client.

The fork's Markdown->HTML conversion, quote_reply, and forward_email
extra_parts paths are unaffected; coverage added for the extra_parts case.

Deliberately not ported: upstream's _as_modern_smtp_message() /
SMTPUTF8_POLICY machinery and its structured delivery diagnostics, both of
which serve V2 plumbing this fork does not have (send_message here takes the
legacy compat32 branch, so SMTPUTF8 can never be negotiated regardless).

658 -> 687 tests, all passing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jbkjr pushed a commit to jbkjr/mcp-email-server that referenced this pull request Aug 21, 2026
Wh1isper#230)

Hand-ports the reachable subset of upstream ai-zerolab/mcp-email-server
commit a1668a9 (PR Wh1isper#230, "close RFC 5322, SMTPUTF8, MIME, and IMAP
interoperability gaps") into this fork's pre-V2 classic.py. Upstream
rewrote its architecture in Wh1isper#212, so the commit cannot be cherry-picked;
each item below was re-implemented against the fork's own idioms.

Ported (all in mcp_email_server/emails/classic.py):

1. Unknown MIME charset. The three decode sites caught only
   UnicodeDecodeError, but bytes.decode() raises LookupError for an unknown
   codec (unknown-8bit, x-unknown, iso-8859-8-i, windows-874). Since
   get_email_body_by_id wraps parsing in a broad except, such emails
   vanished silently — indistinguishable from a missing UID or an allowlist
   block. Added _decode_text_part, which catches both.

2. message/rfc822 subtree isolation. _is_attachment_part returned False for
   an encapsulated message with no filename, so walk() descended into it and
   spliced the inner body into the outer one. Added the content-type check
   plus _iter_content_parts, which yields body leaves and attachment roots
   without entering attachments, and replaced the walk() loop.

3. Structural RFC 5322 recipient parsing. _parse_recipients split the raw
   header on ",", mangling '"Doe, John" <john@example.com>' and group
   syntax. Both callers already parse with policy=default, so
   _addresses_for_header reads AddressHeader.addresses instead.

4. IMAP LIST framing. aioimaplib appends the tagged completion text to
   response.lines (parsed into a phantom mailbox), and a literal-form
   mailbox name arrives as two entries (parsed as "{7}", real name dropped).
   The latter matters: _find_special_folder feeds folder.name to
   archive_emails and trash-first delete_emails, which would COPY into a
   nonexistent mailbox. Added _IMAP_LITERAL_MARKER + _parse_list_responses,
   required the mandatory attribute list in _parse_list_response, and
   switched both call sites (list_mailboxes, sent-folder lookup).

5. APPEND keyword atom grammar. _VALID_IMAP_FLAG rejected legal IANA
   keywords ($Forwarded, $Junk, $NotJunk, $Phishing, $MDNSent) and atoms
   like 123flag / project.name. Replaced with _IMAP_ATOM_SPECIALS +
   _is_imap_atom + _is_valid_imap_flag; still rejects "a b", "\Seen)",
   "bad{flag" and "\\Seen".

6. Locale-independent SEARCH dates. strftime("%d-%b-%Y") emits 05-MÄRZ-2026
   under de_DE, an invalid IMAP date-text that breaks every before/since
   search. Replaced with an _IMAP_MONTHS lookup.

7. Special-use flag case. _find_sent_folder_by_flag compared r"\Sent"
   case-sensitively. Now casefold()-equal, matching \sent / \Sent alike.
   _find_special_folder also moved from a substring match (which would match
   a hypothetical \NoArchive) to the same exact comparison.

8. IMAP SEARCH astring encoding, ASCII half. _sanitize_imap_value passed
   single words through unescaped, so ", \, (, ), {, %, * reached the wire
   raw and broke framing, and it silently stripped the user's own quotes
   ('say "hi"' -> '"say hi"'). Now escapes instead of stripping via
   _is_imap_astring, and raises on control characters.

Deliberately not ported:
- SMTPUTF8 envelope selection — unreachable here; it needs the modern
  EmailMessage conversion this fork lacks.
- RFC 6855 ENABLE / UTF8=ACCEPT APPEND, and the utf8= parameter threaded
  through upstream's LIST helpers, which only serves that work.
- _LiteralSearchCommand / _ImapSearchLiteral / synchronizing UTF-8 SEARCH
  literals — deep aioimaplib private-API coupling for no benefit here.
  _sanitize_imap_value still returns a quoted str for non-ASCII, and the
  fork's conditional CHARSET logic is unchanged.
- app.py's _PUBLIC_APPEND_DETAILS — V2-only.

Two upstream details were intentionally not copied verbatim: the unified
loop collects html_body unconditionally (upstream guards it with
"and not body", but quote_reply and forward_email consume html_body), and
the single-part fallback is kept so a lone non-text body (e.g.
text/calendar) is not dropped.

Tests: new tests/test_rfc_interoperability.py (37 cases adapted from
upstream's suite of the same name) plus four existing tests updated to the
corrected contracts. 747 pass (was 710); ruff check and ruff format debt
unchanged (7 errors, 6 files).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Email App

1 participant