General purpose web IDE for MLscript by chengluyu · Pull Request #367 · hkust-taco/mlscript

chengluyu · 2025-12-16T11:12:12Z

Note

The web IDE is (manually) deployed and accessible at https://mlscript.fun.

Preview: https://web-ide-mlscript-web-ide.luyu.workers.dev/

Planned Tasks

Support importing standard libraries through import "std/Stack.mls" instead of using relative paths.
Rewrite as many web components as possible with MLscript.
Generate deployable compiled project source files.

Bugs to be fixed

The file explorer doesn't navigate when using Firefox and Safari.

CAG2Mark · 2025-12-17T07:19:22Z

FYI: pressing tab does not insert whitespace, but instead navigates to the next element

LPTK · 2025-12-18T05:26:20Z

+  val pwd = os.pwd
+  val workingDir = pwd
+
+  val mainTestDir = workingDir/"hkmc2"/"shared"/"src"/"test"
+  val stdPath = mainTestDir / "mlscript-compile"
+
+  val compilerPaths = new MLsCompiler.Paths:
+    val preludeFile = mainTestDir / "mlscript" / "decls" / "Prelude.mls"
+    val runtimeFile = mainTestDir / "mlscript-compile" / "Runtime.mjs"
+    val termFile = mainTestDir / "mlscript-compile" / "Term.mjs"
+
+  val nodeModulesPath = workingDir / "node_modules"


Any reason to move all of these in the companion? Doing so means they can no longer be overridden by other users of the test suite. (There is just one example with BenchTestState for now.)

LPTK · 2025-12-18T05:27:03Z

 import hkmc2.semantics.*
 import hkmc2.syntax.Keyword.`override`
 import semantics.Elaborator.{Ctx, State}
+import hkmc2.io.Path


LPTK · 2025-12-18T05:28:21Z

-
+      raise:
+        ErrorReport(msg"Cannot resolve the import path ${actualFile.toString}" -> rawPath.toLoc :: Nil)
+      Import(sym, rawPath.value, actualFile)


Suggested change

Import(sym, rawPath.value, actualFile)

Import(sym, rawPath.value, actualFile)

LPTK · 2025-12-18T05:28:43Z

Beautiful! :^)

LPTK · 2025-12-18T05:30:34Z

      catch ex =>
        // System.err.println("Unexpected error in watcher: " + ex)
-        // ex.printStackTrace()
+        ex.printStackTrace()


Isn't the stack trace already shown when throw ex propagates to the top-level?

FlandiaYingman · 2025-12-27T15:52:00Z

How about having a special syntax to denote that std is a predefined shortcut instead of an actual place? Say something similar to TypeScript's @ prefix for predefined paths and Typst's @preview or @local. We can have @std.

LPTK · 2025-12-27T16:09:57Z

I didn't know TypeScript did that. It's not strictly necessary (since paths not starting with . are treated as absolute and the others are for libs/predefined folders), but it could be a nice way of getting some more disambiguation. WDYT @chengluyu?

chengluyu · 2025-12-28T07:00:44Z

How about having a special syntax to denote that std is a predefined shortcut instead of an actual place? Say something similar to TypeScript's @ prefix for predefined paths and Typst's @preview or @local. We can have @std.

@ usually represents the scope name. You can register an organization in a package registry like npm, and the packages published within that organization must include this prefix when importing, for example @mlscript/ucs-demo-build (the MLscript compiler package I published for the UCS web demo).

The practice you mentioned should be more or less like, in TypeScript projects, some people use paths to shorten long path names. For example, the usage here: https://github.com/Netflix/metaflow-ui/blob/ed4c2d34f12e7c4ac0020978bcfb8c8c2d938c6b/tsconfig.json#L28-L37

I plan to implement path remapping options like this. So that projects in mlscript-apps can resolve to the std in other directories.

I didn't know TypeScript did that. It's not strictly necessary (since paths not starting with . are treated as absolute and the others are for libs/predefined folders), but it could be a nice way of getting some more disambiguation. WDYT @chengluyu?

As to whether we should use @std, I think this is not bad.

Resolve merge fallout around module import aliases, split test runners, and test-folder ownership.

# Conflicts: # hkmc2/shared/src/test/mlscript-compile/Predef.mjs

# Conflicts: # hkmc2/shared/src/main/scala/hkmc2/codegen/js/JSBuilder.scala # hkmc2/shared/src/test/mlscript-compile/Predef.mjs # hkmc2/shared/src/test/mlscript-compile/Runtime.mjs

Introduces a project registry as a foundation for switching between multiple projects. Each project owns its slice of localStorage so the existing global file tree can later be swapped wholesale. - New filesystem/projects.mls owns the registry, current id, key helpers, and a one-shot legacy migration that rewrites mlscript-fs:* and mlscript-fs-paths into mlscript-fs:proj:default:* and mlscript-fs-paths:proj:default. A schema marker (mlscript.projects.schema = v1) prevents re-running migration. - filesystem/persistent.mls now resolves storage keys through projects.fileKey / projects.pathsKey using the current project id, so every write/read is project-scoped without touching the existing event flow. - main.mls calls projects.bootstrap() at the top of start() so the registry is ready before any persisted file is loaded. No UI changes; existing users keep their files seamlessly under the new default project.

Introduces a workspace-style project switcher accessible from a pill in the toolbar and via Cmd/Ctrl+O. Users can create, delete, and open projects; each project owns its own slice of the in-browser filesystem. - New components/ProjectSwitcher.mls defines a <project-switcher> custom element built on native <dialog> elements: list of project cards on the left, detail pane with stats and actions on the right, nested sub-dialogs for create and delete confirmations. - components/ToolbarPanel.mls replaces the static title with a pill that shows the current project glyph + name and dispatches workspace-switcher-requested on click. - components/IdeWorkbench.mls renders <project-switcher> once. - main.mls owns switchProject: empty the in-memory tree via fs.resetAll, clear diagnostics, re-inject the standard library, commit the new current id, then call persistent.loadPersistedFiles for the new namespace. Seeds /main.mls when the project has no files yet. - filesystem/fs.mls adds resetAll, which atomically clears fileTree and notifies subscribers with a single "reset" event. Persistence ignores it, so the outgoing project's files stay safe in localStorage. - components/FileExplorer.mls and components/EditorPanel.mls handle "reset" by clearing their derived state (tree rows, open tabs). - filesystem/projects.mls grows registry mutations (createProject, deleteProject, purgeProjectStorage, fileCount, findProject) plus commitCurrent used by the switch routine.

- Add projects.renameProject and a Rename action with its own sub-dialog in the switcher. Renaming the current project re-dispatches current-project-changed so the toolbar pill updates immediately. - Lock the modal to a fixed height (min(560px, calc(100vh - 48px))) instead of max-height, so the dialog no longer jumps as projects are added or removed; the project list scrolls inside. - File count now includes std files: a small fs.getAllFiles predicate counts std nodes once and is added to the per-project persisted count. - Visual pass on the modal: - Delete action button takes a soft red palette to signal danger. - Rename confirm button shares the accent palette with Create. - Close (×) button becomes a 28×28 inline-flex box so the glyph is centered. - Project cards get user-select: none. - Restore the MLscript wordmark in the toolbar's leftmost slot: "ML" in Rubik 600 sans, "script" in Instrument Serif italic at the accent color. The brand sits in a new .toolbar-leading flex container next to the project pill. - index.html loads Instrument Serif from Google Fonts alongside Rubik and Google Sans Code.

Each project can now be exported as a self-contained JSON file and imported back into another browser instance. The switcher modal also gained a read-only file preview with a clickable file list. Storage layer: - filesystem/persistent.mls grows snapshotProject(id) (reads every file for a project out of localStorage as { path, payload } records) and writeProjectFile(id, path, payload) used by import to seed the new project's namespace without going through the in-memory tree. - filesystem/projects.mls adds adoptProject(meta) that allocates a fresh id while preserving the imported project's descriptive metadata (description, color, icon, createdAt, activeFile). Switcher UI: - Export button in the detail action row downloads <project>.mls-project.json shaped as { schema: "mlscript-project/v1", exportedAt, project, files }. - Import opens a dedicated sub-dialog that explains the accepted format, shows the practical localStorage size limit, and surfaces parse errors inline. After a successful import the new project is selected in the list but not auto-opened. - A two-column preview pane replaces the old "coming soon" note. The left column lists every file the project ships with (user files merged with std lib files, sorted, depth-indented) and the right column renders that file's content as plain monospaced text. - Clicking a file routes through a setPreviewPath setter method so the cross-scope write reaches the private class field; preview content switches accordingly. - Modal sized to 1100x720 to give the preview real estate. The detail pane is a flex column so the preview grows to fill remaining vertical space. - Detail header carries only glyph + name + description; the four action buttons (Rename, Export, Delete, Open) sit on their own row so the project name no longer competes with them. - Stats grid is now a single row of four inline label/value pairs.

After fs.resetAll() and persistent.loadPersistedFiles() restore the new project's tree, the analysis worker has received a flood of incremental events ("reset" plus a "create" per file) and its symbol index is in a mixed state. Send it a fresh init snapshot instead so the workspace symbols reflect the new project cleanly. - analysis/index.mls exposes reinitialize(): cancels any pending change flush, drops queued changes, and resends a full "init" message via the existing sendInitialSnapshot path. A no-op if start() has not run yet. - main.mls's switchProject calls analysis.reinitialize() between persistent.loadPersistedFiles() and dispatchProjectChange(), only when the analysis module reference is available. The compiler worker is intentionally not restarted: each compile request already sends the full file set, so it has no per-project cache to invalidate.

The preview tree used to render a flat depth-indented file list, so paths under /std/ appeared as bare filenames with no visible parent. Rebuilt the renderer to emit folder rows between files, matching how the left sidebar's file explorer shows structure. - buildTreeRows walks the sorted path list once and emits a folder row the first time it enters a new directory. A Set of emitted folder paths guards against duplicates so multiple files in /std/ produce exactly one /std/ folder row. - File rows remain clickable <button> elements; folder rows are <div>s (no preview content to switch to). The querySelectorAll for click wiring already targeted "button.ps-preview-row", so it ignores folders without changes. - File icon picks among file-code (.mls, .md), file-json (.mjs), and file as a generic fallback. - CSS scopes cursor/hover to .ps-preview-file so folder rows don't look clickable, gives folder rows muted bold text with an accent-colored folder-open glyph, and adds a small chevron-down to match the file explorer's expanded-folder affordance.

# Conflicts: # hkmc2/shared/src/main/scala/hkmc2/codegen/Block.scala # hkmc2/shared/src/main/scala/hkmc2/codegen/js/JSBuilder.scala # hkmc2/shared/src/test/mlscript-compile/Predef.mjs

Upstream commit 984bf90 (Remove implicit Return flag) removed the distinction between explicit and implicit returns from the IR. It updated Lowering.term_nonTail to use Assign(noSymbol, ...) for non-tail results, but missed Lowering.program (used by MLsCompiler when emitting .mjs files). The result was that every module's last top-level expression was wrapped in Return and JSBuilder dutifully emitted it as `return EXPR;` at module scope — illegal JS that broke every Web IDE component with `customElements.define(...)` as its last statement ("Uncaught SyntaxError: Illegal return statement" in the console). Fixing it in Lowering.program is tempting but breaks the diff-test worksheet path: JSBackendDiffMaker explicitly maps top-level Return into an Assign so it can display the result value. Both paths emit the same IR but want different JS for the trailing Return. So fix this in JSBuilder.programBody — the module-emission entry point. Apply Block.mapReturn there to rewrite Return(res) → Assign(noSymbol, res, End()) before handing the block to `block`. The worksheet path (jsb.worksheet) is untouched and keeps its existing semantics. The bundled sjs diff-test goldens have been updated mechanically: every `return EXPR` at the end of a module-mode sanitized JS dump becomes `EXPR;`, which is what would actually be emitted for a real module.

- Folder rows are now <button>s with click handlers that toggle a collapsed/expanded state stored in a Set<path> on the component. The chevron flips between chevron-down (expanded) and chevron-right (collapsed), the folder glyph swaps between folder-open and folder, and the row also carries aria-expanded for accessibility. - renderPreview filters out any row whose ancestor folder is currently collapsed (isHiddenByCollapse walks the path prefixes). The file click-to-preview wiring still targets button.ps-preview-file, so folder clicks no longer trigger preview switches. - .mjs (and .js) rows now render the icon-file-code glyph used by the left-sidebar file explorer for code files. The previously assigned icon-file-json class is not part of the loaded Lucide font, which is why those rows displayed with no glyph at all. - CSS gives folder buttons cursor + hover styling matching file rows and ensures the folder button stretches full width.

The preview body now hosts a real read-only CodeMirror EditorView with the same syntax-highlight extensions as the editor (MLscript for .mls via Highlight.mlscript; JavaScript for .mjs / .js). The plain <pre><code> text dump is gone. - ProjectSwitcher imports the editor module and stores the live EditorView on the component. renderPreviewBody mounts a new view only when the visible file actually changes (snapshot of dataset.previewPath on the body element + identity check on the view's DOM root); folder collapse / expand re-renders no longer thrash the editor instance. - previewEntry replaces previewLookup + lookupFromFs, returning both content and mtime so the header can show when the file was last written. Snapshot wins for user files; fs.findNode is the fallback source for std files which live in the in-memory tree. - The header is now a flex row: file path on the left (monospace, muted, ellipsized when long), formatted mtime on the right (monospace, subtle, never wrapping). When no file is selected both spans go empty. - CSS sizes the body container as flex 1 with `.cm-editor { height: 100% }` so the CodeMirror view fills available space and scrolls inside itself.

LPTK · 2026-05-27T07:12:28Z

 Infinity
 //│ ————————————| JS (unsanitized) |————————————————————————————————————————————————————————————————————
-//│ return Infinity
+//│ 


We should not apply mapReturn for diff-test JS pretty-printed output: producing an discarding assignment with a pure value will just delete the assignment, resulting in no pretty-printed JS.

LPTK · 2026-05-27T07:15:50Z

  fun emptyDetailHtml() =
    """<div class="project-switcher-empty"><p>No project selected.</p></div>"""

  fun detailHtml(project) =


By the way, why not use the XML module to make generating this nicer?

LPTK · 2026-05-27T07:25:21Z

I can't actually reproduce what 525d1a8 is talking about in the commit message. Compiled files already seem fine on the main branch. As evidence of that, the commit did not produce any mjs changes. What's going on?

chengluyu added 2 commits December 16, 2025 16:45

Prototype the Web IDE

cfee4a0

Move web IDE to apps and create a runner for apps

6fd2eb9

chengluyu changed the base branch from mlscript to hkmc2 December 16, 2025 11:12

chengluyu added 3 commits December 18, 2025 10:18

Add path resolution logic

1630ee5

Support module resolution

f88984a

Commit the unstable uid changes to make CI happy

7e14585

LPTK reviewed Dec 18, 2025

View reviewed changes

Specify files that need to be vendored

a98cf1c

LPTK force-pushed the hkmc2 branch 2 times, most recently from 46c5ce1 to 414cf0c Compare March 10, 2026 15:42

LPTK force-pushed the hkmc2 branch 2 times, most recently from f5b9c17 to 323afbd Compare April 13, 2026 10:20

chengluyu added 6 commits May 9, 2026 20:26

Integrate hkmc2 updates into web-ide

c2d564a

Resolve merge fallout around module import aliases, split test runners, and test-folder ownership.

Split package tests from app tests

479e8f6

Implement package vendoring prototype

ed2c319

Validate package graph vendoring

1204cb9

Display relative paths when building packages

9f8b64a

Get rid of unnecessary imports

abcef1e

LPTK reviewed May 10, 2026

View reviewed changes

Comment thread hkmc2/shared/src/test/mlscript/codegen/QQImport.mls Outdated

chengluyu added 3 commits May 10, 2026 17:32

Make parser preprocessing stack-safe for Scala.js

12b3f7c

Harden Web IDE browser compiler path

0827dff

Rewrite Web IDE filesystem in MLscript

fbf47aa

chengluyu added 2 commits May 21, 2026 19:02

Surface runtime execution errors

d293da1

Document Web IDE error surfacing

5be4cfa

chengluyu force-pushed the web-ide branch from 88d36cc to 5be4cfa Compare May 21, 2026 11:43

chengluyu added 3 commits May 21, 2026 19:49

Skip Web IDE preview deploy without secrets

1723333

Trigger Web IDE preview

e03d00a

Document fork preview secrets

e7ac2d3

chengluyu marked this pull request as ready for review May 21, 2026 12:38

chengluyu added 8 commits May 21, 2026 20:58

Use full optimized Web IDE build

ad5b12b

Merge remote-tracking branch 'origin/hkmc2' into web-ide

17b8c21

# Conflicts: # hkmc2/shared/src/test/mlscript-compile/Predef.mjs

Update golden outputs after hkmc2 merge

9b12870

Compile active import closure in Web IDE

3e7175a

Merge remote-tracking branch 'origin/hkmc2' into web-ide

8e8502e

# Conflicts: # hkmc2/shared/src/main/scala/hkmc2/codegen/js/JSBuilder.scala # hkmc2/shared/src/test/mlscript-compile/Predef.mjs # hkmc2/shared/src/test/mlscript-compile/Runtime.mjs

Fix tokenizer end-of-line handling

f107700

Keep tab indentation inside editor

45249c4

Fix compile completion status text

60bd62b

LPTK added the slopfest label May 22, 2026

chengluyu added 11 commits May 27, 2026 02:00

Merge remote-tracking branch 'origin/hkmc2' into web-ide

5173c82

# Conflicts: # hkmc2/shared/src/main/scala/hkmc2/codegen/Block.scala # hkmc2/shared/src/main/scala/hkmc2/codegen/js/JSBuilder.scala # hkmc2/shared/src/test/mlscript-compile/Predef.mjs

Update golden outputs after hkmc2 merge

dcf012e

LPTK reviewed May 27, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

General purpose web IDE for MLscript#367

General purpose web IDE for MLscript#367
chengluyu wants to merge 148 commits into
hkust-taco:hkmc2from
chengluyu:web-ide

chengluyu commented Dec 16, 2025 •

edited by LPTK

Loading

Uh oh!

CAG2Mark commented Dec 17, 2025

Uh oh!

LPTK Dec 18, 2025

Uh oh!

LPTK Dec 18, 2025

Uh oh!

LPTK Dec 18, 2025

Uh oh!

LPTK Dec 18, 2025

Uh oh!

LPTK Dec 18, 2025

Uh oh!

FlandiaYingman commented Dec 27, 2025

Uh oh!

LPTK commented Dec 27, 2025

Uh oh!

chengluyu commented Dec 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

LPTK May 27, 2026

Uh oh!

LPTK May 27, 2026

Uh oh!

LPTK commented May 27, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	Import(sym, rawPath.value, actualFile)
	Import(sym, rawPath.value, actualFile)

Conversation

chengluyu commented Dec 16, 2025 • edited by LPTK Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Planned Tasks

Bugs to be fixed

Uh oh!

CAG2Mark commented Dec 17, 2025

Uh oh!

LPTK Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

LPTK Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

LPTK Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

LPTK Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

LPTK Dec 18, 2025

Choose a reason for hiding this comment

Uh oh!

FlandiaYingman commented Dec 27, 2025

Uh oh!

LPTK commented Dec 27, 2025

Uh oh!

chengluyu commented Dec 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

LPTK May 27, 2026

Choose a reason for hiding this comment

Uh oh!

LPTK May 27, 2026

Choose a reason for hiding this comment

Uh oh!

LPTK commented May 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

chengluyu commented Dec 16, 2025 •

edited by LPTK

Loading

chengluyu commented Dec 28, 2025 •

edited

Loading

LPTK commented May 27, 2026 •

edited

Loading