[viridale]

Fixes #1466. Follows #1476, which read the runtime's settings from the environment. It was closed in favor of "a solution that is more similar to what haskell does" (comment). This PR is that design: a compiled program reads its argv the way GHC's RTS does. It also finishes what #934 and #988 started for --help, for the remaining runtime options.

The rule

• The bracket: the runtime's options are the words between +RTS and -RTS. A trailing -RTS can be left off.
• Everything else is the program's (IO.args), in order. Outside a bracket, --threads, --gpu, --gpu-build and --bend-help are ordinary arguments.
• -- stops the runtime from reading options and is passed to the program. So runtime options are still taken only until the first --, and the program now sees that --.
• --RTS does the same as -- but is dropped.
• BENDRTS holds options too. It's read before the command line, so ./main +RTS --threads 4 overrides BENDRTS="--threads 8".
• Spans: BENDRTS and each bracket are read separately, so an option's value must be in the same span. BENDRTS="--threads" ./main +RTS 2 fails, and so does ./main +RTS --threads -RTS x +RTS 2.
• Unknown options: an unknown word inside a bracket fails with bend: unknown runtime option --bogus. So does an empty word.
• Values: --threads takes digits, at least 1. --gpu takes on, off, or digits with at most one point followed by GB or MB, above zero. main's C parser (strtol, strtod) also took a sign, leading spaces, an exponent or hex, so --gpu 1e2GB and --threads +4 worked there; they fail now, on C and JS alike.

GHC does the same in its argv loop and with GHCRTS.

Before and after

args prints IO.args(), #1466's program; the head is left out below. These rows are from the C binary. The JS build of this branch prints the same lists.

./args …	main	this PR
grep -e --threads notes.txt	bend: expected a thread count of 1 or more after --threads, exit 1	[<grep>, <-e>, <--threads>, <notes.txt>]
compress --threads auto	the same error, exit 1	[<compress>, <--threads>, <auto>]
build --threads 4	[<build>]	[<build>, <--threads>, <4>]
make --gpu-build	no output, exit 0; main never runs	[<make>, <--gpu-build>]
grep -- -v	[<grep>, <-v>]	[<grep>, <-->, <-v>]
grep +RTS --threads 4 -RTS -v	[<grep>, <+RTS>, <-RTS>, <-v>]	[<grep>, <-v>]
--bend-help	the runtime's usage	[<--bend-help>]

Migration

before	after
./main --threads 8	./main +RTS --threads 8, or BENDRTS="--threads 8" ./main
./main -- -v	./main -v
./main --bend-help	./main +RTS --bend-help
./main --gpu-build	./main +RTS --gpu-build (bend -o passes it itself)

A binary keeps its old behavior until it's rebuilt.

What changes

• bend2/comp.ts:
  • The C main collects BENDRTS's words and the bracketed words, then parses them with the existing option code. A NULL closes each span, so a value can't come from the next one and an empty argument stays an argument.
  • CLI_HELP describes the bracket.
  • The out-of-memory error's hint reads +RTS --gpu 8GB, since a bare --gpu 8GB is now the program's argument.
  • The JS cli() drops the bracket the same way. The JS runtime has no options to set, but it checks a bracket's option names and values with C's grammar and messages, so a typo fails on both. +RTS --gpu-build exits 0 with no output. JS ignores BENDRTS.
  • let cli_args = []; is unchanged, since bend-lint's seam matches that line.
  • Size: comp.ts goes from 63,495 to 63,953 tokens, under its 64k cap. Rebased as is, it came to 64,151, so the last commit makes room without changing behavior. It drops main's and cli's comments, since CLI_HELP states the rule. It also drops three comments that repeat another: JS show_val's and io_list's (the C ones say the same) and ring_rewind's. An unknown option prints and returns 1, as --gpu-build's failure does, and JS's size pattern is shorter but accepts the same set.
  • BENDRTS is 90 of those tokens, so it can be dropped if you'd rather not have it.
• bend2/main.ts: bend -o runs +RTS --gpu-build, with BENDRTS cleared so the user's options can't change or break the build.
• gates/perf.ts: FLAGS gets the bracket. gen_pins.ts reads the same constant.
• gates/ping.ts: adds two checks beside "a plain run gets its arguments":
  • A built program's argv on C and JS: runtime option names, --, an empty argument and a space are kept, and a mid-line bracket is dropped.
  • --RTS is dropped and ends the options, a valid BENDRTS is accepted, and BENDRTS="--threads 0" exits 1, as does a value in the span after its option.
  • +RTS --thread 2, +RTS --gpu of, +RTS --gpu 1e2GB, +RTS '' -RTS and a value across brackets fail with exit 1 and the same stderr on C and JS.
  • The test gate passes no arguments, so it can't cover this.
• tests/io/args.bend: only its comment changes.

Not in this PR

• bend file.bend … is unchanged. The driver accepts its options after the file (bend <file.bend> -o <out>), so the program's arguments still follow its --, as with cargo run -- …. The interpreter has no runtime options, so it passes the words after -- through unchanged, +RTS included.
• Build-time defaults are left out. GHC's -with-rtsopts and -rtsopts could come later.
• The BendRT paper doesn't change. It names --threads and --gpu, and those names stay.

Suggested doc changes (not committed)

guide/GUIDE.md: it's human-edited and sits at 7,998 of its 8,000 tokens, so the two GPU lines become one. With this change it's at 7,993:

-answers the command line, less the runtime's own options (a `--` ends them):
-its head is the program as invoked, like C's `argv[0]`. A handle (`File`,
+answers the command line, less the runtime's options (`+RTS` to `-RTS`): its
+head is the program as invoked, like C's `argv[0]`. A handle (`File`,

-./file --threads 8        # run a native binary on 8 CPU threads
-./file --gpu off          # run ! calls on the CPU (the GPU is on by default)
-./file --gpu 4GB          # cap the GPU's heap at 4GB
+./file +RTS --threads 8   # run a native binary on 8 CPU threads
+./file +RTS --gpu 4GB     # cap the GPU heap at 4GB; off runs ! calls on the CPU

guide/SHADERS.md:24: `--gpu off` becomes `+RTS --gpu off`.

Spelling

+RTS and -RTS match GHC exactly, so GHC's documentation of them applies here as written. If you'd prefer a Bend-specific spelling like +bend and -bend, it's a rename in the C and JS loops.

What I ran

All on an M3 Max Mac, rebased onto main at d2aeefc.

• The table above: built with main's compiler and with this branch. I also checked both builds against the bracket mid-line and at the end, --RTS, -- inside a bracket, BENDRTS with extra whitespace, and an empty, a spaced and a Unicode argument.
• Options take effect:
  • IO.thread_count prints 3 with +RTS --threads 3, and 5 with BENDRTS="--threads 5".
  • It prints 3 when given both, so the command line wins.
  • Without a bracket, ./tc --threads 3 prints the CPU count.
• Metal: bend -o on tests/run/array_atomic_fadd.bend writes fadd.gpu, and the binary prints 68 and 2448.25, also with +RTS --gpu 512MB, --gpu .5GB and --gpu off. It still does with BENDRTS="--bend-help" or BENDRTS="--threads 0" set. ./fadd +RTS --gpu-build exits 0 with no output. I didn't test a host with no GPU; that code path is unchanged.
• The new ping checks: the full ping gate needs the site repo, which I don't have. I ran the same checks with bun bend2/main.ts as bend. All 7 pass on this branch, and 6 fail on its first commit.
• C and JS agree on 41 bracket inputs, with the same exit code and stderr. Accepted: --gpu 4GB, 512MB, 0.5GB, .5GB, 5.GB, on, off, and --threads 3, 007. Rejected: --gpu +1GB, 1e2GB, " 4GB", 0x10GB, infGB, 0GB, 4gb; --threads " 3", +3, 0, 3x; an empty word; constructor; and --threads -RTS 2.
• Tests: gates/test.ts's shard script, run locally on this Mac in 4 shards, passes 1633 / 1635.
  • The two failures are gfx_clicks and gfx_window. They expect "no display", but this Mac has a desktop session, so a window opens.
  • main fails the same two here.
• Repo gate: gates/repo.ts passes, 54 / 54.
• tsc: no new errors.
• bend-lint: loads and checks a file.
• Not run: the cluster gates, the perf gate (only its FLAGS change), Linux and CUDA.

[viridale]

@nicolas-abril is this more in line of how you'd like this fix to land? I'm working on a CLI library for bend hence the motivation for this

[nicolas-abril]

The bracket parsing looks good. The ping checks pass on C and JS for me, with the same output on both. I found three issues, inline below.

[nicolas-abril]

This can overflow when argc == 0. rts holds the BENDRTS words, a NULL, and at most argc - 1 more entries from argv, so with argc >= 1 it fits. With argc == 0 (an execve with argv = {NULL}) and BENDRTS unset or one character long, malloc gets 0 or 1 pointers and rts[nr++] = NULL writes past the end. Running it that way, the binary dies with bend: memory fault.

Suggested change

	char** rts = malloc((argc + strlen(env)) * sizeof(char*));
	char** rts = malloc((argc + strlen(env) + 1) * sizeof(char*));

[nicolas-abril]

n > 0 is checked before the size is converted to bytes, so a tiny size passes the check but ends up as 0 bytes. (u64)(n * mul) truncates, and the reservation is later rounded down by & ~16383 (line 5245). On Metal with tests/run/array_atomic_fadd.bend:

• +RTS --gpu 0.0000001MB makes mem 0, so it silently falls back to the default size.
• +RTS --gpu 0.001MB rounds to a 0-byte span and fails with bend: reservation failed instead of the size error.

Could the check run on the byte count after rounding (or require at least one 16 KB page)? JS's size regex accepts the same inputs, so it would need the matching rule to keep the C/JS errors identical.

[nicolas-abril]

run() goes through lib.exec, which spreads process.env into the child's environment (_lib.ts:67), so the operator's BENDRTS reaches every program these checks run. With BENDRTS="--threads 0" exported on the gate host, this check exits 1. With BENDRTS=--gpu-build, every built binary run by ping or test.ts exits 0 with no output. main.ts already clears it for bend -o. Could run() default to BENDRTS: "" (before ...env, so the checks that set it still work), and test.ts do the same?

[nicolas-abril]

I pushed d322836 with fixes for the three issues from my review:

• rts gets room for one more pointer, so an empty argv no longer overflows it (ASan flagged the old size).
• --gpu now checks the size after converting it to bytes and requires at least one 16 KB page. 0.0000001MB and 0.001MB give the size error on C and JS, and ping checks 0.001MB.
• ping's run() and test.ts's shard script clear BENDRTS. With BENDRTS="--threads 0" exported, ping went from 60/65 to passing.

ping 66/66, repo 54/54, test 1635/1635. comp.ts is at 63,992 tokens. The branch conflicts with main in bend2/comp.ts, so it still needs a rebase or merge, and that may push comp.ts over its cap.

[nicolas-abril]

I merged main and pushed cce9d84. It's mergeable again.

• Conflict: one comment above ring_rewind, which I took from main. comp.ts's cap is 66k on main, so I restored JS io_list's comment that had been dropped to fit the old 64k cap. comp.ts is now at 65,804.
• IO.args with an empty argv: io_argc started at 1, so a program exec'd with argv = {NULL} read the NULL argv[0] and crashed. It now starts at argc > 0, and IO.args returns an empty list.
• Docs: GUIDE.md and SHADERS.md now show +RTS --threads 8, +RTS --gpu 4GB and +RTS --gpu off, and the IO.args paragraph names the bracket. GUIDE.md is at 7,993 of its 8,000 tokens. The PR description's "Docs" section and token counts are now out of date.

ping 70/70, repo 54/54, bend-lint 134/134. The cluster test gate kept failing on shards whose working directory was deleted mid-run ("Module not found bend2/main.ts"), probably a concurrent run. The 110 tests it reported all match their expected output when run locally, apart from 12 expected-error tests that give the same output on main.

[nicolas-abril]

Follow-up: I reran the gates on cce9d84 with the cluster quiet. test passes 1691/1691, ping 70/70 and repo 54/54.

[nicolas-abril]

Updated description (as of 8507661)

The description above no longer matches the branch. This is what the branch does now.

Fixes #1466. A compiled program (C or JS) reads runtime options only between +RTS and -RTS. Every other word is the program's.

The rule

• The bracket: the words between +RTS and -RTS are the runtime's options. A trailing -RTS can be left off.
• Everything else goes to the program (IO.args), in order. Outside a bracket, --threads, --gpu, --gpu-build and --help are ordinary arguments.
• --RTS ends the runtime's options and is dropped. Every word after it is the program's.
• -- is an ordinary argument. It reaches the program, and a bracket after it is still read. Inside a bracket it's an unknown runtime option. GHC differs here: its -- is passed through but stops runtime parsing.
• BENDRTS holds options too. It's read before the command line, so ./main +RTS --threads 4 overrides BENDRTS="--threads 8". JS ignores it.
• Spans: BENDRTS and each bracket are parsed separately, so an option's value must be in the same span as the option.
• Unknown options: an unknown or empty word inside a bracket fails with bend: unknown runtime option … and exit 1, on C and JS alike.
• Values: --threads takes digits, at least 1. --gpu takes on, off, or digits with at most one point followed by GB or MB, and at least one 16 KB page. A sign, leading spaces, an exponent or hex are no longer accepted.
• Help: +RTS --help prints the runtime's usage (it was --bend-help).

./args …	the program gets
grep -e --threads notes.txt	grep -e --threads notes.txt
grep -- -v	grep -- -v
a -- +RTS --threads 2 -RTS b	a -- b
a --RTS +RTS b	a +RTS b
+RTS -- -RTS	bend: unknown runtime option --, exit 1

Migration

before	after
./main --threads 8	./main +RTS --threads 8, or BENDRTS="--threads 8" ./main
./main --gpu off	./main +RTS --gpu off
./main -- -v	./main -v (-- -v now passes -- too)
./main --bend-help	./main +RTS --help
./main --gpu-build	./main +RTS --gpu-build (bend -o passes it itself)

Old flags outside a bracket aren't rejected. They silently become program arguments. A binary keeps its old behavior until it's rebuilt.

What changes

• bend2/comp.ts (65,706 of 66,000 tokens):
  • The C main collects BENDRTS's words and the bracketed words, then parses them with the existing option code. A NULL closes each span.
  • IO.args answers an empty list when the program is exec'd with no argv[0].
  • CLI_HELP changes only its usage line, to usage: %s [+RTS options -RTS] [arguments]. --bend-help becomes --help, and the -- line is gone.
  • The out-of-memory hint reads +RTS --gpu 8GB.
  • The JS cli() drops the bracket the same way. It checks options with C's grammar and messages, so a typo fails the same way on both.
• bend2/main.ts: bend -o runs +RTS --gpu-build, with BENDRTS cleared.
• gates/perf.ts: FLAGS use the bracket.
• gates/ping.ts:
  • run() clears BENDRTS, so the operator's environment can't change the checks.
  • New checks: a built program's argv on C and JS, --RTS, BENDRTS and its spans, and +RTS --help.
  • Malformed options fail alike on C and JS, including --gpu 0.001MB and +RTS -- -RTS.
• gates/test.ts: the shard script unsets BENDRTS.
• guide/GUIDE.md, guide/SHADERS.md: show +RTS --threads 8, +RTS --gpu 4GB and +RTS --gpu off. GUIDE.md is at 7,993 of its 8,000 tokens.
• tests/io/args.bend: only its comment changes.

Not in this PR

• bend file.bend … is unchanged. The program's arguments follow its -- and are passed through as they are, +RTS included, since the interpreter has no runtime options.
• Build-time defaults (GHC's -with-rtsopts, -rtsopts) are left out.
• The BendRT paper doesn't change. It names --threads and --gpu, and those names stay.

Checks at 8507661

• Gates: test 1691/1691, ping 72/72, repo 54/54, repo-gate CI green.
• By hand:
  • C and JS print the same output for the examples above.
  • AddressSanitizer is clean on an exec with argv = {NULL}.
  • On Metal, --gpu 0.001MB gives the size error, and 512MB still runs.
• Not run: Linux, CUDA, the perf gate.