feat(gmail): add gmail sync-all for concurrent multi-account sync - #1508
Merged
Conversation
CoverageTotal: 97.32% 🔴 -0.01 pp vs Comparing
Patch coveragePatch: 95.96% (879/916 new lines covered)
Uncovered new lines (37)
|
newhoggy
force-pushed
the
issue-1504-gmail-sync-all
branch
from
August 6, 2026 10:17
c8d338b to
646f841
Compare
…ync engine Adds SyncOptions.shared_pool: Option<Arc<Semaphore>>, acquired immediately before each message fetch in both fan-out paths (fetch_and_archive_messages / _streaming), layered underneath each account's existing local --concurrency clamp. None for every current caller, so gmail sync's behavior and test suite are unchanged — this is groundwork for gmail sync-all (#1504, ADR-0067), which will be the first caller to actually construct Some(pool) and share it across concurrent per-account syncs. A dedicated test proves a pool sized to 1 never lets a second fetch request reach the server while the first is still in flight.
Replaces the external wrapper-script pattern (looping `gmail sync --account ...` serially over a list of mailboxes) with a first-class `omni-dev gmail sync-all` subcommand, driven by a new `.omni-dev/gmail-sync.yaml` config file (src/cli/gmail/sync_all.rs). - gmail-sync.yaml is discovered via the existing Chain A machinery (local/ shadow, --context-dir/OMNI_DEV_CONFIG_DIR, walk-up to repo root) and loaded strictly: a missing file, an empty `accounts` list, malformed YAML, or an account name absent from ~/.omni-dev/settings.json's gmail.accounts map are all hard errors before any client is built, with unknown account names batched into one message. - Each account resolves its own client via helpers::create_client_for, never GmailCommand::execute's env-var-mutating path, so concurrent per-account tasks can't race on OMNI_DEV_GMAIL_ACCOUNT. --account is rejected outright when combined with sync-all. - Accounts run concurrently via tokio::spawn, sharing one Arc<Semaphore> (sized by --concurrency / gmail-sync.yaml's concurrency / the same default as `gmail sync --concurrency`) built on the shared_pool support just added to SyncOptions. - No live per-message progress bars in this release (several concurrent MultiProgress instances would fight over one terminal): one summary line prints per account as it finishes, plus a trailing combined total. -o json/yaml/yamls/jsonl emit one structured record per account plus a combined_summary. Exits non-zero if any account fails or reports per-message errors, without aborting the others. See ADR-0067 for the full design rationale. Updates the help_all_output golden snapshot for the new subcommand.
Records the design rationale for gmail sync-all's config file, the shared-semaphore concurrency model, and the v1 decision to skip live multi-account progress bars (ADR-0068). - docs/gmail.md gains a "Sync all accounts" section alongside the existing single-account Sync docs, cross-referencing Multiple accounts and Rate limits and retry behaviour. - docs/omni-dev-directory.md adds gmail-sync.yaml to the recognised files table, a File specs entry, and a Validation behaviour entry noting it's the one Chain A file that never silently falls back. - docs/adrs/README.md's inventory gains the ADR-0068 row. - CHANGELOG.md's Unreleased/Added section gains an entry for #1504, matching the existing Gmail feature entries.
gmail sync-all shipped calling engine::run_sync (no progress channel) for every account instead of run_sync_with_progress, so a sync-all run showed no live feedback at all — just each account's summary line once it finished, unlike gmail sync's own listing-spinner + fetch-bar pair. ADR-0068 recorded this as a deliberate v1 gap: running one indicatif::MultiProgress per account concurrently would mean N independent renderers fighting over one stderr. Closes that gap with a single shared MultiProgress instead of one per account: - SyncProgressBars::new_in(&multi, label) (src/cli/gmail/sync/progress.rs) registers a listing-spinner + fetch-bar pair on a caller-supplied MultiProgress rather than constructing its own, prefixing both bars with an account label so accounts stay visually distinguishable. SyncProgressBars::new keeps its existing single-instance behavior and plain (unprefixed) template byte-identical to before. - should_show_progress (src/cli/gmail/sync.rs) is now pub(crate) so sync-all gates its bars on the exact same condition gmail sync uses (-o table, not --quiet, an interactive stderr) rather than risking a second, drifting copy of that logic. - run_sync_all (src/cli/gmail/sync_all.rs) stands up one MultiProgress when that gate passes, gives each account a bar pair registered on it, and threads the resulting progress sender through to run_sync_with_progress per account. The per-account summary line still prints to stdout as each account finishes, now routed through MultiProgress::suspend so it doesn't land mid-redraw of the still-live bars below it; the trailing combined-total line prints only after every render task has drained. - --quiet's help text now notes it also suppresses the live bars. Closes #1504 (follow-up)
Updates ADR-0068 and docs/gmail.md to describe the shared-MultiProgress design that closed Decision 4's v1 gap (live per-account bars for gmail sync-all). - ADR-0068's Status line records the same-day amendment; Decision 4 is rewritten from "no live multi-account progress bars in v1" to describe SyncProgressBars::new_in, the shared should_show_progress gate, and the MultiProgress::suspend-wrapped summary print. Consequences' "deferred, not solved here" bullet drops the now-shipped item, keeping only the still-deferred per-account --concurrency override. - docs/gmail.md's sync-all "Progress and output" section replaces "shows no live progress bars" with the shared-bars behavior and notes --quiet now suppresses them too. - CHANGELOG.md's still-unreleased sync-all entry (#1504) is updated in place to describe the shipped behavior rather than the interim limitation, per Keep a Changelog practice for pre-release amendments.
newhoggy
force-pushed
the
issue-1504-gmail-sync-all
branch
from
August 6, 2026 11:08
65eb9bf to
87bafd2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1504.
omni-dev gmail sync-all, driven by a new.omni-dev/gmail-sync.yamlconfig file, replacing the external wrapper-script pattern of loopinggmail sync --account ...serially over a list of mailboxes.SyncOptionsgains an optionalshared_pool: Option<Arc<Semaphore>>(Nonefor every existinggmail synccall — behavior/tests unchanged), whichsync-alluses to bound in-flight fetch requests across every configured account, layered underneath each account's own local--concurrency. Each account still runs its own independentTokenBucketagainst its own Gmail quota.helpers::create_client_for, never the--account-driven env var (unsafe across concurrent tasks);--accountis rejected outright when combined withsync-all.gmail-sync.yamlis loaded strictly (missing file / emptyaccounts/ malformed YAML / unknown account names are all hard errors before any client is built), unlikescopes.yaml/commit-rules.yaml's warn-and-fall-back loaders.MultiProgressinstances would fight over one terminal) — a summary line prints per account as it finishes, plus a trailing combined total; structured-oformats emit one record per account plus acombined_summary.docs/gmail.md#sync-all-accountsfor the operator-facing docs.Commit-by-commit:
feat(gmail): thread an optional shared concurrency pool through the sync engine— the engine-level primitive, isolated and behavior-preserving.feat(gmail): add gmail sync-all for concurrent multi-account sync— the CLI feature itself.docs(docs,release): add ADR-0068 and document gmail sync-all— ADR, docs, changelog.Test plan
cargo buildcargo fmt --checkcargo clippy --all-targets --all-features -- -D warningscargo test(full suite) — all pass except the two pre-existing, unrelateddaemon_test.rslifecycle-flake tests (zero references togmailin that file)cargo insta test --test integration_test—help_all_outputsnapshot updated and accepted for the new subcommandomni-dev git commit message lintagainst the three new commits — 0 errors, 0 warnings