feat(drive): Google Drive queue tunnel mode (FlowDriver port)#359
feat(drive): Google Drive queue tunnel mode (FlowDriver port)#359dazzling-no-more wants to merge 13 commits intotherealaleph:mainfrom
Conversation
Resolved conflict in android/app/src/main/java/com/therealaleph/mhrv/ui/ConfigSharing.kt: took upstream's manual Paste button (Android 13+ blocks background clipboard reads) and extended its onClick to dispatch Drive-setup blobs as well — same prefix-based routing the QR scanner uses, so all three import paths (Paste / QR / deep link) end up in the same DriveSetupConfirmDialog. Dropped the unused banner_drive_setup_clipboard string from values/ and values-fa/ since the auto-detect banner went with the upstream change. Cargo.lock + Cargo.toml auto-merged cleanly; version bumped to 1.7.10 with hyper / hyper-util / http-body-util preserved. Verified: - cargo test --lib: 161/161 passing - cargo build --bin mhrv-rs --bin mhrv-rs-ui --bin mhrv-drive-node --features ui: green - ./gradlew :app:compileDebugKotlin: green
therealaleph
left a comment
There was a problem hiding this comment.
@dazzling-no-more — read the description + browsed the diff. Significant feature, well-designed, glad you ported FlowDriver. Honoring your "community testing first" request — not merging yet, but giving you initial review notes for parallel iteration.
Architecture is clean:
- New mode sits alongside apps_script/google_only/full rather than replacing them — exactly the right call. Users can pick which front to use based on their ISP situation, and each is independently usable.
drive.fileOAuth scope (vs fulldrive) is the right minimum — only files this app creates, no risk of touching unrelated user content.- HTTP/2 connection reuse via hyper matches FlowDriver's transport choice and avoids the per-fetch TLS handshake overhead that was killing Apps Script throughput.
- 8-way parallel storage ops in the queue, matches FlowDriver's
parallel. - Onboarding flow (sharer's QR +
mhrv-rs-setup://deep link + paste banner + gallery image picker) is genuinely the strongest part of this PR — solves the "non-technical Iran user receiving config from a friend over WhatsApp" path much better than what we have for Apps Script.
Concerns I'd want to surface in testing:
-
Quota visibility for users. Drive API quota (free tier: 1k req/100s/user, 10k req/100s/project) is tighter than Apps Script's per-account quota for sustained chatter. The 8-way parallel + polling cadence could blow the 1k/100s/user ceiling under load. Worth surfacing the consumed quota in UI like we do for Apps Script.
-
Drive folder cleanup. Short-lived files in a Drive folder that polled by both peers — what happens if mhrv-drive-node dies mid-batch? Orphan files accumulate. FlowDriver had a TTL-based reaper IIRC; would good to confirm that's preserved.
-
drive.filescope still asks for OAuth consent that triggers the "unverified app" warning. Same workflow as Apps Script's deploy flow but on a different surface. Documentation will need to walk through this since it's new for users who haven't done Apps Script setup. -
Domain fronting still applies. Drive API endpoints (
www.googleapis.com,oauth2.googleapis.com) are at Google IPs — the SNI-rewrite path throughgoogle_ipshould work the same. But: the OAuth flow opens a browser toaccounts.google.comdirectly, which doesn't go through mhrv-rs by default. For Iran users whereaccounts.google.comis filtered, the auth flow itself becomes the chicken-and-egg problem. Sharer-side onboarding (you bake the refresh token into the share payload) sidesteps this for the recipient — good — but the sharer still needs an unfiltered path for the initial OAuth. Worth calling out in the docs. -
Iran ISP behavior on Drive API. If the same DPI logic that's RST-injecting Apps Script outbound applies to
googleapis.comtoo, this mode would have the same issue. We don't have data on whether Drive API is RST'd from Iran ISPs the same way. Useful test: an Iran user runs the curl test (similar to the one in #313) againstwww.googleapis.com/drive/v3/files— if that gets RST too, this mode shares the underlying ISP-filter weakness. -
Code organization. 4737 LOC is a lot — I'd find it easier to review if the Drive client / API layer is in its own module hierarchy (
src/drive/{queue,api,oauth}.rs) rather than mixed into existing ones. If it already is, ignore.
Testing path I'd suggest:
- Iran user with v1.7.x apps_script broken from #313 → install this branch → does Drive mode reach origin? (Tests whether Drive API is filter-equivalent to Apps Script)
- US user → install this branch → measure throughput vs apps_script mode (Tests whether Drive's per-poll latency is competitive)
- Onboarding: a non-technical recipient gets the QR via WhatsApp, scans/pastes/imports — does setup work first try?
If we get green data on (1) Iran reachability, this becomes a real alternative to Apps Script for users hit by the ISP RST pattern.
I'll spin up the branch on a test box from my end too. Will reply with findings within a day or two.
For PR mechanics: keep it open for community testing. Once we have a couple of independent positive reports + Drive cleanup is confirmed, we merge. Don't squash — this is feature-substantial enough to deserve its own merge commit in main's history.
Thanks @dazzling-no-more, this is great work.
[reply via Anthropic Claude | reviewed by @therealaleph]
|
Thanks for the review. Addressed in latest commit: #1 (quota visibility): Added a QuotaTracker that bumps on every Drive REST call. Logs WARN at 80% and ERROR past 100% of the 1000/100s/user free-tier ceiling. The full UI quota-meter (desktop + Android) is a real feature on its own — I've left a public quota_snapshot() hook for it and want to do that as a follow-up so this PR doesn't grow further. |
|
@therealaleph Any update on your test? I want to move on to more feature / speed improvement., based on this implementation. |
|
@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP curl --resolve www.googleapis.com:443:216.239.38.120 |
2 similar comments
|
@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP curl --resolve www.googleapis.com:443:216.239.38.120 |
|
@therealaleph @dazzling-no-more I ran the curl test from Iran and it seems that google drive is black listed at least on my ISP curl --resolve www.googleapis.com:443:216.239.38.120 |
can you run this instead: curl --resolve www.google.com:443:216.239.38.120 |
curl --resolve www.google.com:443:216.239.38.120 I ran it on wsl |
curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files |
curl --resolve www.google.com:443:216.239.38.120 -H "Host: www.googleapis.com" -k -I https://www.google.com/drive/v3/files |
Perfect. This means this method will work for you. The 403 is expected since we there is no auth token in the request. If you have time go ahead and try it. |
nice thanks if I'll try it tomorrow I'll let you know |
|
@therealaleph Is there a plan to merge this? If yes, I resolve merge conflicts. |
End-to-end working on Android in my own setup. Opening for community testing before merge — please give it a spin and report.
Summary
Adds a new tunnel mode
google_drivethat uses Google Drive as a shared filequeue for SOCKS5 traffic. Ported from FlowDriver
by @NullLatency — the original Go
implementation. Same protocol idea (multiplexed binary envelopes as short-lived
files in a Drive folder, polled by both peers), same
drive.fileOAuth scope,same architectural choices (HTTP/2 connection reuse, 8-way parallel storage
ops). No Apps Script involved.
This sits alongside the existing
apps_script/google_only/fullmodes;none of them are touched by this PR.
What's new
Transport / CLI
mode: "google_drive"in configmhrv-drive-node— the server side (polls Drive folder, opens TCPconnections on the client's behalf, sends responses back via Drive)
Dockerfile.drive-nodefor one-line VPS deploymentconfig.drive.example.jsoncovering all the new knobsDesktop UI (egui)
Google Drive dialog (paste redirect URL → token cached), folder ID/name,
client ID, poll/flush/idle drag-values
Android UI (Compose)
dialog produces a QR + base64 text payload bundling credentials + refresh
token + folder ID. Recipient gets it via any messenger and imports through
any of:
mhrv-rs-setup://deep link (auto-linkified by Telegram /WhatsApp)
an image, save to gallery" flow without needing two phones face-to-face)
Architecture
GoogleApiClientuses hyper HTTP/2 with one persistent multiplexedconnection — matches FlowDriver's
http.Transport{ForceAttemptHTTP2: true, MaxIdleConns: 100}shapeflush_all/poll_onceparallelized withbuffer_unordered(8)— matches FlowDriver'ssem chan struct{}of cap 8multiplexed within one mux file per (direction, client_id)
Performance
Initial v1 was ~10× slower than FlowDriver. Found the gap: hand-rolled
HTTP/1.1 +
Connection: closeper call meant 300-500 ms of pure TLS handshakeper Drive API call. Switching to hyper's HTTP/2 + parallelizing storage ops
closed it. On my Brave-on-Android-VPN-TUN test, first cold page load went from
~94 s → ~5-10 s.
Testing
Tested end-to-end on Android against an Ubuntu VPS (real hardware, real
network):
adb forward+curl→ returns VPS publicIP ✅
same time as one ✅
cargo test --lib: 156 / 156 ✅cargo build:mhrv-rs,mhrv-rs-ui,mhrv-drive-nodeall green./gradlew :app:assembleDebug: green for arm64-v8a, armeabi-v7a, x86_64,x86 + universal
How to test this PR
1. Google Cloud OAuth (one-time, ~5 min)
download the JSON
2. VPS-side
mhrv-drive-node