Skip to content

feat(drive): Google Drive queue tunnel mode (FlowDriver port)#359

Open
dazzling-no-more wants to merge 13 commits intotherealaleph:mainfrom
dazzling-no-more:feature/google-api
Open

feat(drive): Google Drive queue tunnel mode (FlowDriver port)#359
dazzling-no-more wants to merge 13 commits intotherealaleph:mainfrom
dazzling-no-more:feature/google-api

Conversation

@dazzling-no-more
Copy link
Copy Markdown
Contributor

End-to-end working on Android in my own setup. Opening for community testing before merge — please give it a spin and report.

Summary

Adds a new tunnel mode google_drive that uses Google Drive as a shared file
queue for SOCKS5 traffic. Ported from FlowDriver
by @NullLatency — the original Go
implementation. Same protocol idea (multiplexed binary envelopes as short-lived
files in a Drive folder, polled by both peers), same drive.file OAuth scope,
same architectural choices (HTTP/2 connection reuse, 8-way parallel storage
ops). No Apps Script involved.

This sits alongside the existing apps_script / google_only / full modes;
none of them are touched by this PR.

What's new

Transport / CLI

  • New mode: "google_drive" in config
  • New binary mhrv-drive-node — the server side (polls Drive folder, opens TCP
    connections on the client's behalf, sends responses back via Drive)
  • Dockerfile.drive-node for one-line VPS deployment
  • config.drive.example.json covering all the new knobs

Desktop UI (egui)

  • "Google Drive queue" entry in the mode dropdown
  • New section: credentials JSON picker (Browse… via OS file dialog), Authorize
    Google Drive dialog (paste redirect URL → token cached), folder ID/name,
    client ID, poll/flush/idle drag-values

Android UI (Compose)

  • Same mode entry + section
  • One-tap onboarding for non-technical users: sharer's "Share Drive setup"
    dialog produces a QR + base64 text payload bundling credentials + refresh
    token + folder ID. Recipient gets it via any messenger and imports through
    any of:
    • Tapping the mhrv-rs-setup:// deep link (auto-linkified by Telegram /
      WhatsApp)
    • Pasting the text — clipboard banner auto-detects the prefix
    • Scanning the QR with the in-app camera
    • Picking the QR image from the gallery (handles the "WhatsApp received
      an image, save to gallery" flow without needing two phones face-to-face)

Architecture

  • GoogleApiClient uses hyper HTTP/2 with one persistent multiplexed
    connection — matches FlowDriver's http.Transport{ForceAttemptHTTP2: true, MaxIdleConns: 100} shape
  • Storage ops in flush_all / poll_once parallelized with
    buffer_unordered(8) — matches FlowDriver's sem chan struct{} of cap 8
  • Wire format: magic byte + version byte + length-prefixed envelope fields,
    multiplexed within one mux file per (direction, client_id)

Performance

Initial v1 was ~10× slower than FlowDriver. Found the gap: hand-rolled
HTTP/1.1 + Connection: close per call meant 300-500 ms of pure TLS handshake
per Drive API call. Switching to hyper's HTTP/2 + parallelizing storage ops
closed it. On my Brave-on-Android-VPN-TUN test, first cold page load went from
~94 s → ~5-10 s.

Testing

Tested end-to-end on Android against an Ubuntu VPS (real hardware, real
network):

  • Phone in SOCKS5 (PROXY_ONLY) via adb forward + curl → returns VPS public
    IP ✅
  • Phone in VPN_TUN with app-split = Brave + WhatsApp → Brave loads pages ✅
  • HTTP/2 perf fix: confirmed faster, multiple parallel curls finish in ~the
    same time as one ✅
  • All four import paths (link / clipboard / camera / gallery) ✅

cargo test --lib: 156 / 156 ✅
cargo build: mhrv-rs, mhrv-rs-ui, mhrv-drive-node all green
./gradlew :app:assembleDebug: green for arm64-v8a, armeabi-v7a, x86_64,
x86 + universal

How to test this PR

1. Google Cloud OAuth (one-time, ~5 min)

  • Cloud Console → create / pick a project → enable Google Drive API
  • OAuth consent screen → External, add yourself as a test user
  • Credentials → Create OAuth client ID → application type Desktop app
    download the JSON

2. VPS-side mhrv-drive-node

git clone -b <this-branch> <fork-or-upstream-url> mhrv-drive-src
cd mhrv-drive-src
docker build -t mhrv-drive-node -f Dockerfile.drive-node .

mkdir -p /opt/mhrv-drive
# scp credentials.json into /opt/mhrv-drive/
cp config.drive.example.json /opt/mhrv-drive/config.drive.json
# Edit drive_credentials_path to /data/credentials.json

# First run, interactive — prints OAuth URL, paste consent redirect URL back
docker run -it --rm -v /opt/mhrv-drive:/data mhrv-drive-node

# Note the folder_id from the log, pin it into config.drive.json,
# then daemonize:
docker run -d --name mhrv-drive-node --restart unless-stopped \
    -v /opt/mhrv-drive:/data mhrv-drive-node

@github-actions github-actions Bot added the type: feature feat: PR — auto-applied by release-drafter label Apr 27, 2026
Resolved conflict in android/app/src/main/java/com/therealaleph/mhrv/ui/ConfigSharing.kt:
took upstream's manual Paste button (Android 13+ blocks background clipboard
reads) and extended its onClick to dispatch Drive-setup blobs as well — same
prefix-based routing the QR scanner uses, so all three import paths (Paste /
QR / deep link) end up in the same DriveSetupConfirmDialog.

Dropped the unused banner_drive_setup_clipboard string from values/ and
values-fa/ since the auto-detect banner went with the upstream change.

Cargo.lock + Cargo.toml auto-merged cleanly; version bumped to 1.7.10 with
hyper / hyper-util / http-body-util preserved.

Verified:
- cargo test --lib: 161/161 passing
- cargo build --bin mhrv-rs --bin mhrv-rs-ui --bin mhrv-drive-node --features ui: green
- ./gradlew :app:compileDebugKotlin: green
@dazzling-no-more
Copy link
Copy Markdown
Contributor Author

Copy link
Copy Markdown
Owner

@therealaleph therealaleph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dazzling-no-more — read the description + browsed the diff. Significant feature, well-designed, glad you ported FlowDriver. Honoring your "community testing first" request — not merging yet, but giving you initial review notes for parallel iteration.

Architecture is clean:

  • New mode sits alongside apps_script/google_only/full rather than replacing them — exactly the right call. Users can pick which front to use based on their ISP situation, and each is independently usable.
  • drive.file OAuth scope (vs full drive) is the right minimum — only files this app creates, no risk of touching unrelated user content.
  • HTTP/2 connection reuse via hyper matches FlowDriver's transport choice and avoids the per-fetch TLS handshake overhead that was killing Apps Script throughput.
  • 8-way parallel storage ops in the queue, matches FlowDriver's parallel.
  • Onboarding flow (sharer's QR + mhrv-rs-setup:// deep link + paste banner + gallery image picker) is genuinely the strongest part of this PR — solves the "non-technical Iran user receiving config from a friend over WhatsApp" path much better than what we have for Apps Script.

Concerns I'd want to surface in testing:

  1. Quota visibility for users. Drive API quota (free tier: 1k req/100s/user, 10k req/100s/project) is tighter than Apps Script's per-account quota for sustained chatter. The 8-way parallel + polling cadence could blow the 1k/100s/user ceiling under load. Worth surfacing the consumed quota in UI like we do for Apps Script.

  2. Drive folder cleanup. Short-lived files in a Drive folder that polled by both peers — what happens if mhrv-drive-node dies mid-batch? Orphan files accumulate. FlowDriver had a TTL-based reaper IIRC; would good to confirm that's preserved.

  3. drive.file scope still asks for OAuth consent that triggers the "unverified app" warning. Same workflow as Apps Script's deploy flow but on a different surface. Documentation will need to walk through this since it's new for users who haven't done Apps Script setup.

  4. Domain fronting still applies. Drive API endpoints (www.googleapis.com, oauth2.googleapis.com) are at Google IPs — the SNI-rewrite path through google_ip should work the same. But: the OAuth flow opens a browser to accounts.google.com directly, which doesn't go through mhrv-rs by default. For Iran users where accounts.google.com is filtered, the auth flow itself becomes the chicken-and-egg problem. Sharer-side onboarding (you bake the refresh token into the share payload) sidesteps this for the recipient — good — but the sharer still needs an unfiltered path for the initial OAuth. Worth calling out in the docs.

  5. Iran ISP behavior on Drive API. If the same DPI logic that's RST-injecting Apps Script outbound applies to googleapis.com too, this mode would have the same issue. We don't have data on whether Drive API is RST'd from Iran ISPs the same way. Useful test: an Iran user runs the curl test (similar to the one in #313) against www.googleapis.com/drive/v3/files — if that gets RST too, this mode shares the underlying ISP-filter weakness.

  6. Code organization. 4737 LOC is a lot — I'd find it easier to review if the Drive client / API layer is in its own module hierarchy (src/drive/{queue,api,oauth}.rs) rather than mixed into existing ones. If it already is, ignore.

Testing path I'd suggest:

  • Iran user with v1.7.x apps_script broken from #313 → install this branch → does Drive mode reach origin? (Tests whether Drive API is filter-equivalent to Apps Script)
  • US user → install this branch → measure throughput vs apps_script mode (Tests whether Drive's per-poll latency is competitive)
  • Onboarding: a non-technical recipient gets the QR via WhatsApp, scans/pastes/imports — does setup work first try?

If we get green data on (1) Iran reachability, this becomes a real alternative to Apps Script for users hit by the ISP RST pattern.

I'll spin up the branch on a test box from my end too. Will reply with findings within a day or two.

For PR mechanics: keep it open for community testing. Once we have a couple of independent positive reports + Drive cleanup is confirmed, we merge. Don't squash — this is feature-substantial enough to deserve its own merge commit in main's history.

Thanks @dazzling-no-more, this is great work.


[reply via Anthropic Claude | reviewed by @therealaleph]

@dazzling-no-more
Copy link
Copy Markdown
Contributor Author

Thanks for the review. Addressed in latest commit:

#1 (quota visibility): Added a QuotaTracker that bumps on every Drive REST call. Logs WARN at 80% and ERROR past 100% of the 1000/100s/user free-tier ceiling. The full UI quota-meter (desktop + Android) is a real feature on its own — I've left a public quota_snapshot() hook for it and want to do that as a follow-up so this PR doesn't grow further.
#2 (cleanup): We do have the TTL reaper. I extended it: phone-side cleanup_loop now also reaps res-{my_client_id}-mux- orphans older than 5 min, which closes the "drive-node died mid-batch" gap.
#3, #4 (OAuth caveats): Documented in the README. Sharer needs unfiltered Google access for the initial OAuth; recipient gets the refresh token via the QR-share path so they don't.
#5 (Iran/Drive reachability): README has a curl --resolve recipe users can run before deploying to verify their network reaches Drive's edge and completes a TLS handshake. Same shape as the issue #313 test for Apps Script.
#6 (code organization): Agreed the file split would help. I'd rather do that as a separate "pure rename" PR after this merges so the diff is reviewable as moves-not-changes; happy to push it immediately on merge if you'd prefer.

@Enqvy
Copy link
Copy Markdown

Enqvy commented Apr 28, 2026

@dazzling-no-more
Copy link
Copy Markdown
Contributor Author

@therealaleph Any update on your test? I want to move on to more feature / speed improvement., based on this implementation.

@Feiabyte
Copy link
Copy Markdown

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP

curl --resolve www.googleapis.com:443:216.239.38.120
-I https://www.googleapis.com/drive/v3/files
curl: (35) Recv failure: Connection reset by peer

2 similar comments
@Feiabyte
Copy link
Copy Markdown

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP

curl --resolve www.googleapis.com:443:216.239.38.120
-I https://www.googleapis.com/drive/v3/files
curl: (35) Recv failure: Connection reset by peer

@Feiabyte
Copy link
Copy Markdown

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP

curl --resolve www.googleapis.com:443:216.239.38.120
-I https://www.googleapis.com/drive/v3/files
curl: (35) Recv failure: Connection reset by peer

@dazzling-no-more
Copy link
Copy Markdown
Contributor Author

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP

curl --resolve www.googleapis.com:443:216.239.38.120 -I https://www.googleapis.com/drive/v3/files curl: (35) Recv failure: Connection reset by peer

can you run this instead:

curl --resolve www.google.com:443:216.239.38.120
-H "Host: www.googleapis.com"
-k -I https://www.google.com/drive/v3/files

@Feiabyte
Copy link
Copy Markdown

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP
curl --resolve www.googleapis.com:443:216.239.38.120 -I https://www.googleapis.com/drive/v3/files curl: (35) Recv failure: Connection reset by peer

can you run this instead:

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120
-H "Host: www.googleapis.com"
-k -I https://www.google.com/drive/v3/files
curl: (2) no URL specified
curl: try 'curl --help' or 'curl --manual' for more information
-H: command not found
-k: command not found

I ran it on wsl

@dazzling-no-more
Copy link
Copy Markdown
Contributor Author

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP
curl --resolve www.googleapis.com:443:216.239.38.120 -I https://www.googleapis.com/drive/v3/files curl: (35) Recv failure: Connection reset by peer

can you run this instead:
curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files curl: (2) no URL specified curl: try 'curl --help' or 'curl --manual' for more information -H: command not found -k: command not found

I ran it on wsl

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

@Feiabyte
Copy link
Copy Markdown

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP
curl --resolve www.googleapis.com:443:216.239.38.120 -I https://www.googleapis.com/drive/v3/files curl: (35) Recv failure: Connection reset by peer

can you run this instead:
curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files curl: (2) no URL specified curl: try 'curl --help' or 'curl --manual' for more information -H: command not found -k: command not found
I ran it on wsl

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files
HTTP/2 403
content-type: application/json; charset=UTF-8
date: Tue, 28 Apr 2026 21:22:20 GMT
server: ESF
x-xss-protection: 0
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
accept-ranges: none
vary: Accept-Encoding

@dazzling-no-more
Copy link
Copy Markdown
Contributor Author

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP
curl --resolve www.googleapis.com:443:216.239.38.120 -I https://www.googleapis.com/drive/v3/files curl: (35) Recv failure: Connection reset by peer

can you run this instead:
curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files curl: (2) no URL specified curl: try 'curl --help' or 'curl --manual' for more information -H: command not found -k: command not found
I ran it on wsl

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files HTTP/2 403 content-type: application/json; charset=UTF-8 date: Tue, 28 Apr 2026 21:22:20 GMT server: ESF x-xss-protection: 0 x-frame-options: SAMEORIGIN x-content-type-options: nosniff alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 accept-ranges: none vary: Accept-Encoding

Perfect. This means this method will work for you. The 403 is expected since we there is no auth token in the request. If you have time go ahead and try it.

@Feiabyte
Copy link
Copy Markdown

@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP
curl --resolve www.googleapis.com:443:216.239.38.120 -I https://www.googleapis.com/drive/v3/files curl: (35) Recv failure: Connection reset by peer

can you run this instead:
curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files curl: (2) no URL specified curl: try 'curl --help' or 'curl --manual' for more information -H: command not found -k: command not found
I ran it on wsl

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files

curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files HTTP/2 403 content-type: application/json; charset=UTF-8 date: Tue, 28 Apr 2026 21:22:20 GMT server: ESF x-xss-protection: 0 x-frame-options: SAMEORIGIN x-content-type-options: nosniff alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 accept-ranges: none vary: Accept-Encoding

Perfect. This means this method will work for you. The 403 is expected since we there is no auth token in the request. If you have time go ahead and try it.

nice thanks if I'll try it tomorrow I'll let you know

@dazzling-no-more
Copy link
Copy Markdown
Contributor Author

@therealaleph Is there a plan to merge this? If yes, I resolve merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feature feat: PR — auto-applied by release-drafter

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants