v0.8.25: Sim Search, SCIM, Sim OAuth - #7594
Merged
Merged
Conversation
* feat(helm): publish the chart to GHCR as a signed OCI artifact
The chart has only ever existed inside the repo, so self-hosters whose
security process requires an officially published artifact could not use
it at all — a chart pulled from a git checkout is not something they can
pin, mirror, or admit.
Publishes helm/sim to oci://ghcr.io/simstudioai/charts/sim on push to
main, signed with Sigstore keyless signing and carrying a SLSA build
provenance attestation, both stored in the registry so they survive a
mirror into an internal registry.
The publish job lives in helm.yml rather than its own publish-*.yml so it
can gate on the existing jobs: nothing ships unless the chart linted,
unit-tested, rendered clean under kubeconform, and installed on kind.
Publishing is idempotent — a version already in the registry is skipped
rather than overwritten, since a published chart version is immutable.
Also pushes helm/artifacthub-repo.yml to the reserved artifacthub.io tag
for the Artifact Hub listing, and switches the chart README and the
Kubernetes docs page off the clone-and-install path.
* fix(helm): fail closed on registry lookup errors and pin documented installs
The publish guard treated every non-zero `helm show chart` exit as proof the
version was unpublished, so a transient 5xx, an expired token, or a DNS blip
would have enabled a push that moves an already-published version tag. Runs
where the version already exists are routine — the path filter also fires on
package.json and workflow edits — so this was reachable.
Now only an explicit `: not found` counts as absent; anything else stops the
job. Verified against the pinned Helm v3.16.4 that an absent version and an
absent repository both report `: not found`, so a first publish still
proceeds, while denied/unauthorized/dial-tcp failures abort instead.
Also pins the two documented install paths that were still reproducible only
by accident: the cloud-specific command paired a pinned chart with values
fetched from the moving main branch, and the README quick start omitted
--version while the same README warns against exactly that.
* chore(helm): harden the chart workflow against zizmor and document cosign v3
Ran the two standard workflow linters over helm.yml. actionlint reported one
shellcheck style issue; zizmor reported 22 findings, including two High.
Both High findings predate this PR: the version gate expanded
github.base_ref straight into a run block. The three Medium findings were
checkouts leaving the token in .git/config. Fixed all of them here rather
than leaving known findings in a file this PR already rewrites -- the gate
only reads history and fetches a public branch, so it never needed the
credential.
The 16 informational findings were mine: every ${{ }} in a run block is
template expansion before the shell sees it, so the documented fix is to
pass values through env and reference them as shell variables. Done for the
whole publish job, which also removes the shellcheck nit.
zizmor now reports zero findings on the file in pedantic mode, with no
suppressions, and actionlint is clean.
Separately, cosign v3 writes the Sigstore protobuf bundle format by default
and cosign v2 cannot read it, so the verification instructions now state the
required version. The in-workflow verify uses the same binary it signs with,
so it could never have caught this.
Re-verified the existence guard's four branches after the env refactor, and
confirmed it also fails closed when the registry login has not happened.
* fix(helm): push Artifact Hub metadata from helm/ so the layer title matches
Dry-ran the publish sequence against a local OCI registry, which surfaced one
divergence from Artifact Hub's documented command: passing a path-qualified
argument to oras records the layer's title annotation as
`helm/artifacthub-repo.yml` rather than the bare `artifacthub-repo.yml` the
docs produce. Artifact Hub selects the layer by media type, so this was
probably harmless, but there is no reason to diverge from the documented
form. Running the step from helm/ reproduces it exactly.
Everything else in the sequence behaved as intended against a real registry:
helm push derives charts/sim:1.9.1 from the chart name and version, the
digest grep extracts it, the metadata lands with both documented media types
under the artifacthub.io tag, the chart and metadata tags coexist without
disturbing each other, and helm pull round-trips to an identical sha256.
Also confirmed the overwrite hazard behind the fail-closed guard is real
rather than theoretical: re-pushing modified content under an existing
version moved the tag, so a pinned consumer would have silently received
different bytes.
* fix(helm): categorise the chart for Artifact Hub
The chart carried `category: developer-tools`, which is a different
convention entirely -- Artifact Hub reads `artifacthub.io/category` and
accepts only one value from a fixed list that has no developer-tools entry.
The listing this PR sets up would therefore have published uncategorised.
Adds `artifacthub.io/category: ai-machine-learning`, the closest fit in that
list, and keeps the bare annotation for whatever else consumes it. Bumps to
1.9.2 per the chart's own gate and moves the documented versions with it.
Deliberately not adding `artifacthub.io/images`: Artifact Hub already
extracts images from a dry-run render, and a hand-listed copy would be a
second source of truth next to the generated images.yaml.
* feat(helm): also publish the chart to a classic HTTP repo
I claimed earlier that OCI-only was defensible in 2026. That was wrong, and
checking rather than reasoning settled it: Bitnami, cert-manager,
ingress-nginx, prometheus-community, Grafana, Argo and external-secrets all
still serve a live index.yaml. Dual-publish is the actual convention, and
external-secrets -- the workflow this one is modelled on -- does both.
Adds a chart-releaser job that maintains index.yaml on a pages branch and
attaches each packaged chart to a GitHub release. It is a separate job from
the OCI publish because it needs contents: write to cut that release, and
that permission has no business sitting in the job that holds the signing
identity.
Three details worth stating:
- mark_as_latest is false. A chart release must never take the "Latest"
badge from the application release it packages.
- Releases are named helm-chart-<version> so they stay distinguishable from
the vX.Y.Z app releases in the same list.
- skip_existing mirrors the OCI job: re-running on a published version is a
no-op rather than an overwrite.
The job no-ops with a warning until a gh-pages branch exists, because
creating that branch and enabling Pages are one-time manual steps no
workflow can perform for itself, and main's CI must not fail on their
absence.
No documentation yet. index.yaml is only reachable once the branch, Pages,
and DNS are in place, and pointing users at a hostname that does not resolve
is the exact failure this whole PR exists to remove.
* docs(helm): document the Helm repository now that charts.sim.ai serves
Held these back until the hostname actually resolved, because shipping an
install command that points at a dead domain is the exact failure this work
exists to remove. It resolves now: DNS is in place, GitHub Pages serves the
branch, and Let's Encrypt has issued the certificate.
Verified end to end rather than assumed:
helm repo add sim https://charts.sim.ai
helm repo update # Successfully got an update
Presents OCI as the preferred path in both places and the repository as the
compatibility path, and says plainly that the signature and provenance are
attached to the OCI artifact only -- `helm repo add` has no equivalent, and
leaving that implicit would let someone believe they had verified something
they had not.
Bumps to 1.9.3 per the chart's own gate, since this touches helm/sim/.
* docs(self-hosting): install from the published chart, not a checkout
The Terraform page still told readers the chart "is not published to a Helm
repository or an OCI registry, so there is no repository to point at", and
had them vendor the repo as a submodule or git clone. That is now false, and
it is the same claim that makes a self-hoster conclude Sim is not packaged
for them, so it is the most important line in the docs to correct.
It now points repository at the OCI registry with a pinned version, notes
the classic repository as the alternative for tooling that does not speak
OCI, and replaces the "because the chart is local, version does nothing"
paragraph -- pinning the chart version is now both possible and the thing
that stops an unplanned apply pulling new migrations. Syntax checked against
the provider docs; the page's existing set_sensitive list form is already v3.
Also switches the two upgrade commands and the env render check off ./helm/sim,
which no longer exists for anyone following the install path.
Deliberately unchanged, because these are not install references:
- The chart README links on the Kubernetes page. The README lives in the
repo; charts.sim.ai serves index.yaml, not documentation.
- The cosign --certificate-identity-regexp. That is a Sigstore OIDC identity
that happens to look like a URL, and rewriting it would break verification.
- The releases page and issue tracker links, which point where they should.
- The git clone in the quick start, which is the Docker Compose path and
genuinely does need the repo.
* docs(helm): audit the chart documentation against the chart
Swept the remaining checkout-era references and audited the Helm docs by
checking every values path they mention against values.yaml rather than
reading them.
That found a real error: the chart has no `copilot.env`. The map is
`copilot.server.env`, so anyone following the env-validation note would have
set a key that silently does nothing. Fixed.
It also flagged `app.envDefaults.FREE_TABLES_LIMIT` and
`app.secrets.existingSecret.keys`, which are NOT bugs -- both appear only in
upgrade notes describing keys older releases shipped and this one removed.
Left exactly as they are; "fixing" them would delete the history an upgrader
needs.
The chart README was the bigger problem. It led with registry install after
the earlier commits, then used `./helm/sim` in eleven subsequent commands --
upgrade, dry-run, examples, parameters, secrets, troubleshooting. Someone
following the documented install had no such path. They now use the OCI
reference with a pinned version, with one note establishing that convention
so the checkout form does not have to be repeated everywhere.
Two related corrections while in there: the example values files are not part
of the packaged chart, so the examples section now fetches one at a release
tag instead of assuming it locally; and values.schema.json cannot be `cat`ed
without a checkout, so it links to the repo or `helm pull --untar`.
Left pointing at GitHub on purpose: the chart README deep links (that is
where the README lives, and an extra hop through the repository root would
be worse), the releases page, the issue tracker, the cosign identity regexp,
and the Docker Compose quick start's git clone.
Bumps to 1.9.4 per the chart's own gate.
* fix(helm): track the app release in appVersion, and enforce it
appVersion sat at v0.8.18 while the app shipped through v0.8.24 -- six
releases. Because the image tags default to Chart.AppVersion, a default
`helm install` deployed the older Sim, and regenerating images.yaml here
moved eleven lines, so the mirror inventory an air-gapped operator works
from named the wrong images too.
Publishing is what makes this serious. Cloning main got you whatever was
there; a published chart version is immutable, so every stale appVersion
would be frozen and installable forever.
It was bumped by hand and nothing checked it, which is why it drifted. The
publish job now refuses to publish when appVersion does not match the latest
GitHub release, comparing against the release API rather than a hardcoded
value so the check cannot go stale itself. Confirmed it fires on exactly the
drift that existed (v0.8.18 vs v0.8.24) and passes now that it is fixed.
Bumps appVersion to v0.8.24, regenerates images.yaml, and takes the chart to
1.9.5.
* fix(helm): fail the appVersion check only when it lags, not on any mismatch
The check compared appVersion to /releases/latest for equality, which races
the release it is meant to protect. Version tags are cut by the main-branch
merge commit that releases them, so on that commit appVersion legitimately
names a release that does not exist yet while ci.yml is still building it.
Equality would have rejected the bump and blocked the very publish it was
for. helm/sim/ci/kind-values.yaml already documents this circularity, and
records it as the reason appVersion went unbumped from chart 1.2.0 to 1.6.3.
Lagging is the failure; being ahead is a normal transient. The comparison is
`sort -V`, so it orders versions rather than strings -- v0.9.9 against
v0.10.0 is exactly where a string compare silently inverts.
Verified all five cases: behind fails, equal passes, ahead passes with a
notice, and both multi-digit orderings resolve correctly.
* fix(slack): paginate v2 conversation listing * fix(slack): refresh pagination response fixtures * fix(slack): preserve optional pagination defaults * fix(slack): clarify shared pagination output * fix(slack): drain conversation pagination safely
* fix(slack): harden native webhook configuration * fix(slack): complete setup remediation * fix(slack): fail closed without webhook secret * fix(slack): align setup checks with runtime
* docs(library): update what-is-an-mcp-server * Pi Babysit: address PR #7570 feedback --------- Co-authored-by: Sim Pi Agent <pi@sim.ai>
…ation Explained (#7571) Co-authored-by: Sim Pi Agent <pi@sim.ai>
* fix(redis): bound the three unbudgeted stream writers by bytes
The copilot stream buffer, the Tables event log and the realtime file-doc
streams were each bounded by entry count and nothing else. An entry cap bounds
how many entries a key holds and says nothing about how large each one is, so a
single writer emitting large entries reaches gigabytes well inside its cap —
which is how one file-edit stream filled a shared Redis under a 100,000-entry
cap and evicted the whole keyspace.
Each writer gets the bound its read semantics allow:
- Copilot's replay buffer is read from a cursor and must stay contiguous, so it
now reserves bytes against per-stream and per-user ceilings inside the same
Lua that appends, and the writer soft-stops persistence on refusal rather
than failing the live stream.
- The Tables event log is a live feed whose readers already refetch on a prune,
so it drops oldest-first once past a byte ceiling — the existing `pruned`
path carries it, with the running total kept in meta under the same TTL as
the bytes it counts.
- The file-doc streams are Yjs deltas replayed in full by a task attaching
later, so dropping the oldest would lose edits and a native MAXLEN bound is
unsafe. Compaction, which folds deltas into a snapshot first, is lossless —
it now triggers on appended bytes as well as entry count.
Also folds `lib/execution/redis-budget.server.ts` into the shared module rather
than leaving two definitions of the same prefix and ceilings writing the same
Redis keys. Key layout and every execution limit are unchanged, and pinned by
test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(realtime): count only deltas toward the compaction byte threshold
Re-seeding the counter with the snapshot's own size left any document larger
than the ceiling permanently over it, forcing a full snapshot append on every
subsequent keystroke — the write amplification the threshold exists to prevent.
The counter measures edit churn since the last fold, so a stream settles at one
snapshot plus that much churn.
Also self-corrects the Tables byte counter whenever its buffer trims to a single
entry, so an independently evicted events key cannot leave the accumulator
over-reporting and pin the buffer at one entry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(redis): address review findings on the byte bounds
- Measure ceilings in UTF-8 bytes on both the copilot and Tables paths, so the
TypeScript checks bound a stream the same way the Lua's `string.len` does
rather than under-reporting every non-ASCII frame.
- Split an oversized copilot batch on the per-write ceiling instead of refusing
it. A flush carries whatever accumulated since the last one, so a run of large
frames can exceed the ceiling collectively while each frame is individually
writable; refusing that stopped replay for the rest of the stream over a
batching artefact. A single frame past the ceiling is still refused.
- Re-check the copilot soft stop when an in-flight append resolves, not only at
enqueue, so a batch queued behind a refusal cannot land and leave replay
holding later events but not the refused ones.
- Deduct rather than zero the file-doc compaction counter, and only once the
trim succeeds, so a failed fold leaves the trigger armed and a concurrent
publish's bytes survive.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(redis): account for deltas a fold retains, and release cleared counters
The compaction counter was a single total, so a fold deducted bytes for entries
`XTRIM MINID` had retained — anything published past the fold boundary the
tailer had not yet integrated. Those bytes are still in Redis, so the trigger
disarmed while the stream kept growing. Deltas are now tracked as `{id, bytes}`
and dropped only once a trim provably removed them.
Arming the trigger on retained bytes would be the opposite fault: a fold that
reclaims nothing would re-arm immediately and force a full snapshot append per
publish. Only bytes at or before the fold boundary arm it, and because that
boundary moves in the tailer rather than on publish, the tailer re-checks it —
otherwise a burst of large edits followed by silence would sit unfolded until
the next keystroke.
Also releases the copilot owner counter when the buffer is cleared, crediting
the user counter by exactly what the owner held. Those keys are deleted rather
than expired, so the counter otherwise outlived its data and a retry reusing the
streamId would be refused against bytes that no longer exist.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(redis): make buffer cleanup atomic and match MINID's inclusive boundary
Deleting a copilot buffer and releasing its reservation were two round trips, so
a concurrent append landing between them kept its events stored with its
reservation already erased. Both now run in one script, composed from a rendered
release fragment the same way the reservation is.
`XTRIM MINID upTo` is inclusive — it keeps the entry whose id equals the
boundary. Accounting treated that entry as folded, so its bytes stopped counting
while they were still in Redis, and a large paste landing exactly on the
boundary could leave the trigger disarmed. Both directions now use the same
strict/inclusive split: only entries strictly before the boundary arm the
trigger, and only those are dropped once a trim removes them.
Replaces the tests' `any` casts with a typed accessor, per the repository's
TypeScript rule. This immediately caught injected test rooms missing
`pendingDeltas`, which made compaction throw into its catch while the assertions
still passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(redis): never credit the shared user counter from a buffer delete
An owner id is not proof of who wrote the bytes, so crediting the user counter
on clear let anyone able to name a stream decrement a ceiling they never
charged. That is the one direction that must not be possible: a counter driven
down grants writes rather than denying them.
The clear now drops the owner counter only. The user counter's fixed window
settles it instead — it already tolerates accruing bytes Redis has dropped, and
this is the same over-count bounded by the same window. The scope threading that
existed only to credit it is removed with it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(redis): keep counters outliving their data, and stop a failed fold retrying hot
The copilot owner counter used a fixed one-hour window while the stream TTL is
configurable and defaults to exactly that. Raising COPILOT_STREAM_TTL_SECONDS
would have let the counter expire under live data, and the next write would see
zero reserved and grant another full ceiling. The window is now the larger of
the two, so a counter can never expire before what it accounts for.
A failed fold deliberately leaves the trigger armed, but the snapshot XADD lands
before the XTRIM — so a persistent trim failure retried immediately, appending a
full-document snapshot every time and turning a Redis blip into the write
amplification the threshold exists to prevent. A forced fold now waits out a
cooldown after a failure. The entry-count path is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(realtime): adopt byte accounting for a stream taken over
A room attaching to an existing stream started from an empty ledger, so a
multi-megabyte stream under the entry threshold stayed unfolded while that
room's own heartbeat kept refreshing its TTL — a restart or handoff could hold
one open indefinitely.
`catchUp` already reads every entry to rebuild the doc, so adopting their bytes
costs no extra work. Plain deltas only: a compaction snapshot is the result of a
fold rather than something a fold can reclaim, so counting one would arm the
trigger against itself. The trigger is re-checked once, after catch-up, since
nothing else re-checks until the next local publish and a read-only participant
never makes one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(realtime): mark a fold's output explicitly, and account in the tailer
A fold of an agent-only stream is stamped with the agent marker to preserve the
no-persist guarantee, which made it indistinguishable from an ordinary agent
preview frame. Excluding that marker from accounting therefore dropped preview
deltas — the largest payloads there are, and the ones that caused the incident —
while including it would let a snapshot arm the trigger against its own output.
A dedicated field settles it without touching origin selection.
With the ambiguity gone, accounting moves from the publish path to `applyEntry`,
which observes every entry the room tails: this task's appends, a peer task's,
and one published with no room attached anywhere. Publish-side accounting could
only ever see local writes, and contributed nothing to the trigger before the
tailer caught up regardless, since only entries at or before the fold boundary
arm it. The ledger is now a Map keyed by entry id, so an entry observed twice is
recorded once.
Entries written before this field carry no marker and count as deltas, which
over-arms by at most one fold that then trims them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* refactor(redis): drop the clear-buffer script and the unused error class
A variadic DEL is already a single atomic command, so the Lua script and the
render function that built it achieved nothing a plain `del(events, seq, abort,
ownerBudget)` does not. The keys carry no hash tag either, so the script had the
same cluster-slot constraint it appeared to avoid.
Also deletes `RedisBudgetExceededError`, which was defined and never thrown, and
consolidates four rounds of stacked comments in the fold down to the one that
still describes the code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(files): keep file extensions on extensionless downloads * fix(files): detect extensions without a length cap
* feat(files): improve editor recovery and editing controls * fix(files): harden recovery and editor edge cases * fix(files): simplify recovery UI and restore canceled link carets * fix(files): simplify replay and isolate invalid recovery * fix(files): close editor recovery and rendering gaps * fix(files): preserve native non-pixel image heights * chore(files): remove redundant find-bar comment * fix(files): preserve collaboration through reconnects and peer edits * fix(files): authorize joins before subscribing to document frames * fix(files): unify image controls and guard collaborative edits * fix(files): address editor battle-test regressions * fix(files): fence cache writes and preserve native snapshots * fix(files): close recovery and seed review gaps * fix(files): keep acknowledgements responsive and fence admission * fix(files): recheck access after content-room admission * chore(files): type the Redis test client
Add scoped OAuth authorization-code login with PKCE, rotating token families, consent and revocation, and operation-level API scope enforcement. Preserve existing API-key, session, connector OAuth, and browserless CLI authentication. Use the existing global AppConfig oauth-provider flag for request-time rollout control, with the environment variable as its standard fallback. Share the database lifecycle between migration and db:push, bound token cleanup, and document rollout sequencing. Validation includes the full app, CLI, and database suites; both PostgreSQL provisioning paths; live browser and CLI authentication; API-key regression checks; and AppConfig off/on/off changes without restarting the app.
Sim-native SCIM 2.0 provisioning for Okta, Microsoft Entra ID, OneLogin, and JumpCloud: Users and Groups CRUD with filters and paging, RFC 7644 discovery and error envelope, provider payload tolerances, bearer tokens with rotation, reversible deactivation, removal with ownership handoff, group mappings onto permission groups, workspace access, and the organization admin role, hourly reconciliation, an activity log, and the settings UI under SSO. Ships with the Enterprise plan on the hosted product; SCIM_ENABLED / ENTERPRISE_ENABLED on self-hosted. Migration 0324_scim_provisioning (expand-only). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DGXcwsHzYGp6pUDWRtJKpz
* fix(cli): clarify login method selection * fix(cli): correct API-key authentication guidance
* fix(workflows): keep long-running calls active * chore(helm): bump chart version * fix(workflows): preserve stream compatibility * refactor(streaming): import SSE helpers directly
* fix(api): clarify and standardize endpoint descriptions * fix(api): qualify run recovery and trace duration guidance * fix(api): align descriptions across resource families
…7581) * fix(identity): streamline SSO settings and harden SCIM provisioning * fix(identity): reuse settings controls and tighten SCIM projections
…7582) * fix(files): preserve editor content and isolate shared image reads * fix(files): preserve source drafts and image export fidelity
* fix(provenance): close durable writer and reader gaps * fix(provenance): preserve mixed files and repeated durable bindings
* feat(auth): enable OAuth with permission-group controls * fix(auth): serialize OAuth refresh with permission updates * docs(auth): clarify CLI permission scope * fix(auth): recheck OAuth entitlement under organization lock
* improvement(docs): record Entra directory disablement validation * improvement(docs): clarify Entra deprovisioning lifecycle
…7590) * fix(cli): expose pagination cursors and fetch complete inventories * fix(cli): reject pagination cursor cycles
* feat(knowledge): resolve connector tokens through one service-account-aware resolver
A connector's access token was resolved three different ways — once in the sync
engine and twice in the connector application layer — each calling
`refreshAccessTokenIfNeeded` with no scopes. Google's service-account resolver
throws `Scopes are required for service account credentials` without them, so a
service-account credential could not authenticate a connector at all: creation
failed config validation, and a connector that got past it failed at its first
mint. The string wrapper also discarded the `cloudId` an Atlassian service
account resolves with, which is the only way such a credential can name its
site — its API token cannot call `accessible-resources` to discover one.
`resolveConnectorAccessToken` now owns that resolution for all three call sites,
turning a connector's declared `ConnectorAuthConfig` plus its credential or key
into a token bundle. `serviceAccountScopes` lets a connector declare the scopes
its provider accepts in a two-legged grant where those differ from the ones its
consent screen asks for, defaulting to `requiredScopes` where they coincide. A
resolved cloud id seeds the same `syncContext` slot the connector would memoise
it into, so no connector needs a service-account branch of its own.
`getMissingRequiredScopes` now reports nothing missing for a service account: it
names its scopes per request and has no granted-scope list, so measuring it
against the required set flagged every scope missing and offered a reconnect
that would have granted nothing. With both fixed, the add-connector modal stops
filtering service accounts out of its credential list.
* feat(knowledge): map Google Drive permissions to document access tokens
The mirrored-ACL model needs one place that decides who a Drive file is
readable by, and it is the piece with the least margin for error: a wrong arm
here publishes a document rather than failing a request.
`driveFileAcl` maps a file's `permissions[]` to the token vocabulary the
document ACL already speaks — `u:` for a named person, `g:` for a group, `pub`
for a genuinely public file — with two rules worth stating outright.
An open share grants nothing unless an admin has opted in for that connector.
A file shared to a whole domain, or to anyone who finds it, is usually shared
that way by accident, and its contents are exactly what nobody meant to
publish; an admin who knows their domain's sharing hygiene can turn it on.
A link-only share never grants, opt-in or not. `allowFileDiscovery: false` is
Drive's own "reachable by link, findable by nobody", and Drive excludes those
files from its search for the same reason we exclude them from ours. Onyx's
file path misses this — it makes any `anyone` grant public without reading the
flag, though its folder path does read it — so a link pasted once would make a
document searchable by the whole workspace.
Group membership and shared-drive membership are deliberately not resolved
here. Both are directory state; expanding them per file would re-read the
directory once per document, so the group is recorded as a token and the
directory sync resolves it. A file whose every grant is unrepresentable
resolves to `link` rather than an empty ACL, keeping "hidden on purpose"
distinguishable from "hidden because we failed".
* feat(knowledge): write mirrored document ACLs without re-embedding
Admin mode mirrors a source's own permissions onto each document, and the whole
point is that permissions change far more often than content: somebody joins a
group, a folder is reshared. So the ACL write cannot be a field on the content
write. A document the sync classifies as unchanged never reaches the document
update path at all, and that path sets `processingStatus: 'pending'` — the sole
trigger of re-embedding — so routing an ACL change through it would re-embed a
corpus every time a group membership changed.
`persistDocumentAcls` assigns `acl` and nothing else, leaving `contentHash`,
`processingStatus`, `chunkCount` and the embedding rows untouched. Documents are
grouped by identical ACL before writing, since files under one folder
overwhelmingly share theirs, so a crawl of thousands resolves to a handful of
statements; `IS DISTINCT FROM` means a re-run that changes nothing writes
nothing, which is what lets permissions sync on a faster clock than content.
Both properties are covered by tests that fail if the assignment widens.
`SyncDocumentAccess` gains an `admin` arm alongside `members`. Both derive their
ACL from something the content sync does not know, so both are born hidden and
made visible by a separate pass — a document indexed before its permissions are
known is invisible, never workspace-wide.
`validateAcl` enforces a 5,000-token ceiling and the token shapes the database
constrains. A document whose ACL we cannot store is stored as readable by
nobody rather than skipped, because leaving the previous ACL in place would keep
serving it under permissions we just failed to verify. Onyx declares the same
ceiling and, by its own comment, never enforces it; catching it here also names
the offending document instead of failing whichever batch it shared a statement
with.
* feat(knowledge): crawl Google Drive as an administrator and mirror its permissions
Completes the admin-mode path end to end for Drive: one crawl under a service
account, each document stored with the ACL the source itself reports.
A service account owns nothing in a Workspace domain, so a crawl under one sees
an empty corpus until it impersonates somebody. The subject lives on the
connector rather than the credential — one `google-service-account` credential
matches every Google service, so a subject stored on it would silently apply to
a workflow reading that person's mail as well as to this crawl reading their
Drive. `serviceAccountSubjectFieldId` names the config field holding it, and the
service-account scope set is read-only and narrower than the interactive one: a
crawl that reads every file in a domain should never hold write access.
That administrator's email domain is also the tenant segment of every group
token the crawl writes. It has to be settled before the first token is stored,
because deriving it differently later would orphan every ACL already written —
which is why Drive could not be switched on until the subject existed.
The listing now requests `permissions` and `permissionIds` together, because
Drive sometimes reports more ids than it expands. A file whose two counts
disagree is left readable by nobody rather than mirrored under the subset that
arrived: that sounds like the safe direction but is not, since the grants that
went missing are exactly the ones nobody verified.
The ACL pass runs over the whole listing rather than the documents whose content
changed. A membership or sharing change moves no content, so restricting it to
changed documents would let a revoked grant stay readable until somebody
happened to edit the file. It runs after the content pass, so a document this
run inserted — born hidden — is present to be made readable, and before
reconciliation, so a revoked grant lands even on a run that removes nothing.
The access-mode vocabulary moves into one leaf module. Three separate queries —
the scheduler's due sweep, the queue's dispatch claim, and the engine's own lock
— each hard-coded `accessMode = 'workspace'`, so widening the engine without
widening all three would have left admin-mode connectors dispatchable but never
dispatched, or worse, dispatched and then refused. They now read one constant.
* fix(knowledge): make one email address mean one account
`user.email` is unique byte-for-byte only, so `Alice@corp.com` and
`alice@corp.com` can both exist as separate accounts. Every identity binding in
the product compares the case-folded address, so those two rows are one identity
to it: a credential-group enrollment for either matches both, and each account
receives the `s:` subject token of a managed credential belonging to the other.
That is live on the members-mode path today, and it is the same fold the `u:`
document token depends on.
`user_email_lower_unique` is the constraint that makes the state unreachable —
the standard Postgres form of a case-insensitive unique email, on
`lower(btrim(email))`. Access resolution probes that exact expression, so a
predicate written any other way would silently become a sequential scan of
`user` on every read; verified as an index scan against a real database.
The migration pre-checks for duplicates inside the runner's batch transaction,
so a database that already holds one fails the deploy with a sentence naming the
problem and rolls back having changed nothing. Without that check the concurrent
build would fail on the first duplicate and leave an INVALID index that
`IF NOT EXISTS` skips on every later run — the constraint would appear to exist
while enforcing nothing, which is the one outcome worth engineering against.
Both paths were verified against a real database in rolled-back transactions:
the index builds on current data, and a deliberately inserted case-variant
duplicate raises and rolls back.
Access resolution keeps its own ambiguity check rather than trusting the
constraint to still be there. An index can be dropped during an incident, and a
restore can bring back a database built before it existed; neither should
silently hand one person another's documents. An ambiguous address binds to
nothing, so both accounts keep the tokens every workspace member holds and lose
only what their identity would have granted.
The enrollment join also stops reading `normalized_email`. That column is
declared unique but written by nothing, so the `COALESCE` over it always fell
through to the folded address — and would have silently started matching a
broader set of people the day anything backfilled it.
* feat(knowledge): resolve directory groups so mirrored grants reach the right people
A mirrored ACL names a group; nothing until now said who was in one, so a
document Drive shared with a group was readable by nobody. This closes the loop:
the directory is enumerated into `knowledge_external_group` and its membership,
and access resolution turns a reader's address into the `g:` tokens they hold.
Groups are scoped by workspace, provider and tenant rather than by connector —
two connectors over one Google Workspace domain grant the same groups, and
resolving the directory once per connector would multiply Admin SDK traffic by
the number of knowledge bases. Membership is keyed by case-folded email rather
than Sim user id, because a directory reports addresses and most members of a
granted group have no Sim account; storing the address means someone who signs
up later inherits their grants on first read, with no backfill.
Nested groups are flattened. Onyx reads one level and stops, so a person who
belongs only through a subgroup silently gets nothing even though the source
grants them access. The walk carries a visited set and a depth bound, because
directories nest arbitrarily and will happily report a cycle.
The unit of work is a group, not the directory. A group that enumerates
completely is replaced in one transaction; one that does not is left exactly as
it was, with a failure recorded and its `lastSyncedAt` untouched. That is the
deliberate departure from Onyx, whose sync marks every row stale, upserts what
the source returned and sweeps the rest — clean until the directory half-fails,
at which point it revokes real members whose rows simply were not returned.
Here an outage costs freshness and nothing else.
That patience needs a bound, or a sync that stopped running would keep granting
forever from membership nobody has checked. A group unconfirmed for longer than
`EXTERNAL_GROUP_STALE_AFTER_MS` stops granting: an outage is survivable, an
abandoned sync is not. Verified against a real database — the read plans as
index scans on both sides, and a directory pushed past the window drops from 500
matching groups to none.
The refresh runs inside the admin-mode crawl, before ACLs are written, so a
crawl can never publish grants against membership this workspace has never read.
It is rate-limited on its own clock so a frequently-syncing connector does not
re-read the directory every run, and a failure is logged rather than thrown —
last-known-good membership is still serving reads, and failing the content sync
over it would strand the documents as well as the groups.
* feat(knowledge): let a connector be put into administrator mode
The mirroring path was complete but unreachable: `accessMode` accepted only
`workspace` and `members`, so nothing could ask for the mode the last four
commits built. This makes it selectable, end to end.
The mode-switch matrix stays linear rather than growing to three-by-three. Both
credential-backed modes change the same way — swap the credential, keep the
documents — so they share one fast path, and `members` remains the special case
it always was. What entering a mode does to existing ACLs moves into one table:
`workspace` publishes to the workspace, `members` and `admin` both hide, because
in both the ACL belongs to a pass that has not run yet. Hiding on entry is what
makes an interrupted switch safe — documents are hidden early, never shown
early — and the exit branch now reads that table instead of hard-coding
`WORKSPACE_ACL`.
Administrator mode takes the same role as members mode. Both decide whose data
the workspace indexes, and both refuse rather than warn when the connector
cannot deliver: a source that reports no per-document permissions has no
administrator mode, and one that has not been told which administrator to crawl
as would index every document readable by nobody — indistinguishable from a
broken sync. Failing when the mode is chosen says what is missing while the
person choosing it can still supply it.
`currentAccess` in the edit modal stops folding unknown modes into Workspace.
That catch-all would have told an admin their documents were visible to the
whole workspace when they were not, and silently rewritten the mode on the next
save.
The engine-ownership rule is now covered by a test that walks every mode: the
content and member engines hold mutually exclusive leases, so a mode claimed by
both — or by neither — is a connector that either never runs or runs twice.
* fix(knowledge): gate mirrored access on its own feature, not on Credential Groups
One availability check governed everything permission-aware, and half of it was
about Credential Groups. Administrator mode mirrors a source's own ACLs and
touches no Credential Group, so an operator turning that feature off would have
silently revoked every document an administrator crawl had mirrored — from a
feature it does not use.
The check now answers two questions from one billing read: whether source-
mirrored access is available, and whether member-scoped access is. Both are
enterprise features on Sim Cloud and both sit behind the same kill switch, so
turning permission-aware knowledge off still hides every permission-scoped
document on the next read; only the Credential Groups clause is now scoped to
the mode that needs it. They are returned as a pair so a caller cannot check one
and act on the other.
Access resolution mints each token family under its own answer, and choosing
administrator mode is refused when the workspace is not entitled to it — before
a crawl indexes a corpus whose ACLs nobody would be able to match.
* refactor(knowledge): drop directory-group columns nothing reads
`display_name`, `consecutive_failures` and `last_error` were written on every
sync and read by nothing. The staleness ratchet is `last_synced_at` alone — a
failed enumeration writes nothing at all, and the timestamp not advancing *is*
the record — so the failure counter measured something no decision consulted,
and the error string duplicated a log line. The display name was speculation
about a UI that does not exist.
Removing them takes `recordGroupFailure` with them: the failure path now writes
to the database not at all, which is both simpler and a stronger statement of
the invariant it was there to protect.
* feat(knowledge): mirror Confluence space permissions and page restrictions
Confluence joins Drive as a source an administrator crawl can mirror, and it is
the case that shaped the contract.
A page's restrictions come back only when that page is asked for, so they cannot
ride along with the listing the way Drive's permissions do. `getDocumentAcls`
resolves them for the whole listing at once, after it — round trips bounded by
the corpus rather than by the page size, with the space's principals, each
page's restriction, and every address resolved once per run and reused. Only an
unrestricted page pays for its ancestry, which is the expensive lookup.
A restriction replaces the space's permissions rather than narrowing them. Real
Confluence access is the intersection, so this over-grants in exactly one case:
somebody named on a page restriction who cannot view the space at all. That is a
misconfiguration in the source, it errs toward a page they were deliberately
named on, and it is what Onyx does. Representing the true intersection would
mean expanding both principal sets to member addresses, which our group tables
could do — but it emits one token per member, and a five-thousand-person space
would carry five-thousand-token ACLs on every restricted page.
`null` and `[]` are different answers throughout: no restriction means inherit
from the nearest restricted ancestor, then the space; a restriction naming
nobody means readable by nobody. Confluence itself only ever produces the first,
but collapsing them would publish every deliberately locked page.
One departure from the plan, which said to identify groups by name as Onyx does.
Onyx uses names because its membership sync is keyed by name; ours is keyed by
whatever the permissions API returns, and that is the id. Using it costs no
lookup per group and survives a rename, which a name-keyed ACL would not.
Directory enumeration moves behind one connector hook. The tenant is baked into
every stored group token, and only the connector knows what a tenant is for its
source — a Workspace domain for Drive, a site's cloud id for Confluence. The
engine previously derived it from the impersonation subject, which Confluence
does not have: its service account authenticates with an API token and
impersonates nobody, so directory refresh would have silently skipped.
The limit worth knowing: Confluence Cloud withholds an address whose owner's
profile hides it, and a person we cannot name cannot be granted access. Those
grants are dropped and counted rather than guessed at, and a group with a
withheld member is reported incomplete so it never replaces a stored membership
with a subset.
* feat(knowledge): refresh mirrored directories on their own clock
Group membership decides who can read an already-indexed document, so it has to
move independently of the corpus: someone leaving a group should lose access in
minutes, not on whatever schedule their documents happen to be re-crawled on.
Until now the only thing that refreshed a directory was the admin crawl itself,
which made the five-minute interval a ceiling rather than a cadence — on a
connector syncing daily, a revoked membership stood for a day, bounded only by
the staleness ratchet.
A scheduler now offers every admin-mode connector each tick and lets
`syncExternalDirectoryGroups` decide whether its directory is actually due. The
crawl keeps its own refresh, which is a floor rather than a duplicate: it is
what guarantees a crawl never publishes grants against membership nobody has
read.
Connectors sharing a directory cost one refresh between them — the first brings
it up to date and the rest skip on the interval gate. Two ticks overlapping on
one directory would both enumerate and write the same rows, which is wasteful
and never wrong, so it takes no lease to prevent; every write on this path is
idempotent, and a lease would be new state to keep correct for no behavioural
gain.
Failure is contained per connector. One workspace whose credential lapsed must
not stop the tick refreshing every other workspace's directory, and a test
covers exactly that.
The refresh moves out of the sync engine so both callers share it, and the
Confluence connector drops a concurrency helper it should never have had —
`mapWithConcurrency` already existed in `lib/core/utils`.
* fix(knowledge): close two admin-mode gaps found in an architecture audit
Two real bugs, then the duplication and drift a full re-read of the branch
turned up.
Administrator mode was unreachable for Confluence. Entering the mode required
an impersonation subject on every connector, but a Confluence service account
holds an API token that already speaks for the site and impersonates nobody, so
the check refused every attempt. A subject is now required only of a connector
whose auth declares a subject field, and a test pins the token-backed case.
An incremental listing could not carry a revoked grant. A permission change
moves no content — re-sharing a file does not touch its modified time in Drive,
restricting a page does not touch its version in Confluence — so an incremental
run listed only edited documents and the ACL pass refreshed only those. A grant
revoked on an unchanged document stood until the next full sync happened to
run, which is the over-grant direction. Administrator mode now always lists
the whole corpus; content is still hydrated by hash, so unchanged documents are
never re-fetched or re-embedded, and the cost is metadata pages only.
Configuration validation minted without impersonation. The shared token
resolver took the source config as optional, and the validate path did not pass
it, so a Drive service account checked its configuration against an empty
domain. The config is required now, and every caller — sync, validation, mode
switch, creation, and the directory scheduler — passes it.
The Workspace domain was derived in two places with a comment saying they must
agree. It is one function now, beside the Google directory adapter, which moves
from the knowledge library to the Drive connector where Confluence's equivalent
already lives; provider-specific API clients belong with their connector, and
the orchestration that calls them stays provider-agnostic. The Confluence
connector memoised its cloud id in four separate copies, which became one.
A group identifier is canonicalised once, in `canonicalGroupId`, by the crawl
that writes a token and the directory sync that stores the membership it
resolves against. Both already lower-cased by different routes; now they cannot
drift. Confluence identifies groups by id, so the docs claiming "never an opaque
id" were wrong and are corrected in the token vocabulary and the schema.
Removed what nothing read: the impersonation subject the token resolver
returned, a `force` flag no caller passed, two vestigial type aliases, and a
nesting-depth constant that had one consumer and now lives with it. The
directory recency check is one aggregate query rather than two probes; the
Confluence ACL resolver enriches principals once rather than once per page; the
mode picker validates its value instead of casting it; and the admin-mode hint
no longer describes a field only Drive has. The create path had two copies of
the admin-role gate, one per permission-scoped mode, and has one.
* fix(knowledge): read shared-drive permissions, and finish a pending switch before mirroring
Two mechanics found by tracing the admin-mode path end to end rather than
reading it.
Every file on a shared drive was invisible. Drive does not populate a file's
`permissions` when it lives on a shared drive — the docs say so outright, and
the field must come from `permissions.list` instead. The listing left those
files without an ACL, and the pass treated "no ACL" as "readable by nobody". A
whole shared drive indexed to nothing is the failure the plan's own note about
Onyx's `permissionIds` comment was warning about, and it was never built.
The contract now says what it should have: `getDocumentAcls` is called with
exactly the ids the listing could not answer for, and the engine merges the two
sources with the listing's answer winning where it exists. Drive implements it
by paging `permissions.list` per file under bounded concurrency; a file whose
inline entries were incomplete goes the same way instead of being hidden.
Confluence carries nothing inline, so it is unchanged. The merge is a pure
function with its own tests, and the shared-drive tests fail if the hook is
removed.
A switch into administrator mode whose hide outgrew its request budget was
never finished. The completion write cleared `accessRewritePending`, but the
only thing the content engine did with the flag was restore workspace ACLs,
which admin mode's SQL guard correctly ignored — so documents still carrying
`{ws}` from before the switch kept it until the pass overwrote them, and any the
listing missed kept it indefinitely. The pending hide now runs under the lease
before the pass writes real ACLs, mirroring how the member engine finishes its
own; the flag is then cleared on the strength of that.
The shared-drive group token is gone. Nothing resolved it — the directory sync
enumerates groups, not drives — and Drive already reports a drive's members as
ordinary inherited permissions on each file, so the token was both redundant
and a grant nobody could hold.
* docs(knowledge): state why the ACL ceiling exists rather than where the number came from
The comment justified 5,000 tokens by appeal to a reference implementation.
The real reason is that the ceiling is a bug detector: with group tokens a
legitimate document names tens of principals, so thousands means a connector
expanded a group to its members — the exact failure group tokens prevent, and
one that bloats the GIN index for every other document in the workspace.
* chore(knowledge): register the directory-sync cron, and collapse the two migrations into one
The directory-sync scheduler is now in the Helm cron map beside member-sync,
and in the self-hosting background-jobs table. The Helm templates iterate the
map, so the values entry is the whole change.
The unique address index and the directory-group tables ship as one migration.
The tables and the duplicate pre-check run inside the runner's batch
transaction; the embedded COMMIT then lets the index on the hot `user` table
build concurrently. A failure after that COMMIT replays the whole file against
tables that are already committed, so every earlier statement is idempotent.
Replaying it on a real database exposed one defect on the way: drizzle's
derived foreign-key name for the membership table is 71 characters, Postgres
silently truncates identifiers at 63, and the replay guard looked the full name
up in `pg_constraint` and never found it. Both foreign keys now carry explicit
short names in the schema. The pre-COMMIT section replays cleanly twice in a
rolled-back transaction with both guards resolving.
* refactor(auth): one email fold, in SQL and TypeScript, and no reads of the dead column
An address was folded five different ways across the codebase: three SQL
spellings — `lower(x)`, `lower(trim(x))`, `lower(btrim(x))` — plus reads of
`user.normalized_email`, plus inline `trim().toLowerCase()`. Only one of the
SQL forms matches the expression the new index is built on, so the others were
sequential scans of `user` wearing the costume of an indexed lookup.
`normalized_email` turned out to be populated after all — for a fifth of
accounts, by a signup plugin removed in June, using Gmail dot-and-tag stripping.
That is the right function for deduplicating signups and the wrong one for
identity: it merges addresses a mail provider may route to different people,
and it stops at the day the plugin left. Every read of the column is gone. The
column itself stays for one release, because Better Auth selects every schema
column and dropping it while the previous release still serves would break
sign-in; the drop is the follow-up, and the repo's drop audit will name the
argless reads that must be fixed first.
`foldedEmail` now lives in the schema beside the index that indexes it, so the
predicate and the index are one expression by construction. Its TypeScript
twin is `normalizeEmail` from `@sim/utils/string`, which the new access code
now uses instead of inlining the fold.
The index is no longer unique. Production holds thirteen addresses that
collide once folded — real, verified, active accounts — so a unique build would
fail the deploy by design. Access resolution refuses to bind an ambiguous
address, which keeps either account from reading the other's documents until
the pairs are merged and the index can be promoted.
The directory-sync cron joins docker/crontab; the parity audit caught that Helm
alone was not enough.
* fix(knowledge): Drive field mask named a permission field that does not exist
`permittedBy` is not a field of the Permission resource — inheritance lives in
`permissionDetails` — and Drive answers an invalid field selection with 400.
The mask was unconditional, so every `files.list` on this branch failed, in
every access mode, and the suite passed only because every Drive call is mocked
and the assertion checked a prefix of the mask. Caught by a reviewer checking
the mask against the API reference.
Also from that review: `groups.list?domain=` enumerates one domain, while a
Workspace customer routinely owns several — a grant to a group on a secondary
domain named a group the directory never listed, readable by nobody; it is
`customer=my_customer` now. A deleted account's grant no longer mints a token
for whoever is later provisioned with the recycled address. The Admin SDK
calls share the Drive retry wrapper, so one transient error no longer stales a
group. And the `permissionIds` count heuristic is gone: the field counts users
only, so any file with a domain or public grant tripped a needless fallback;
absence of `permissions` is the one signal that matters, and it is kept.
* fix(knowledge): close the audit findings on administrator access
Admin mode was unreachable end to end: the queue still refused every mode
but workspace, and neither modal ever sent the mode, so editing an admin
connector saved it as workspace and published the corpus. A switch into
admin mode also flipped before hiding, showing workspace documents under
a mode whose reader expects source ACLs.
- queue: dispatch every content-engine mode; the modals send the chosen
mode; the access field offers administrator mode on availability, not
on credential-group support
- switch: hide before flipping into a mode that hides on entry; the
engine finishes a pending hide before it lists, strips listing caps,
and hides owned documents the listing did not name
- Drive: domain shares resolve through a synthetic per-domain group whose
one member is a wildcard the reader matches by their own domain; CUSTOMER
members and every customer domain are covered
- Confluence: each page falls back to its own space's readers, not the
union across spaces; ancestors from the v2 collection the API still
serves; space-role assignments expanded; pagination follows next links;
per-page containment; retry on every call
- directory sync: one background job per directory instead of a full walk
inside the scheduler request; runnable statuses only; freshness keyed off
the latest confirmed group so one unreadable group cannot keep a directory
due forever
- config edits on an admin connector re-assert the administrator subject;
a connector whose ACL is derived cannot leave its documents behind
- the access-mode enum lives in the leaf module and the contract derives
from it; the mirror assertion moves to a leaf; the remaining ad-hoc
email folds use foldedEmail; the impersonated subject leaves the logs
* refactor(knowledge): tighten administrator access after the mechanics review
- directory refresh idempotency keys on the sync interval, so a cron wrapper
retry cannot start a second walk of the same directory; the scheduler
offers every admin connector and lets the tenant-level freshness check
dedupe, since a connector type does not imply one tenant
- config validation seeds the run context from the token, so a Confluence
service account validates without a discovery call it cannot make
- the domain-share vocabulary (group id, wildcard member, domain fold) lives
in one module; the Google directory drains pages through the shared
Google pagination helper; Confluence's getJson owns the not-found branch
- access-mode predicates take the union, and the engine narrows the locked
row once; runnable statuses are one constant everywhere
- a knowledge base with a mirrored connector reports hasPermissionScopedConnector
- dead export, unused import, and stale docblocks removed
* refactor(knowledge): collapse the access-mode vocabulary and the duplication around it
/simplify and /cleanup over the branch. The mode literal 'admin' appeared in
eight modules deciding four different things; it is now one leaf predicate
each site reads.
- access-modes owns MIRRORING_ACCESS_MODES/mirrorsSourceAcls and aclIsDerived;
the identity mapper, the second name for the same union, and the duplicate
credential-backed predicate are gone, and the contract derives its enum from
the leaf
- listing caps are stripped once per rule rather than at three depths: create,
the mode switch, and a config edit all key off aclIsDerived, so the stored
config agrees with what runs
- the mirroring assertion moved into the source-config validator the
application layer already owns, so orchestration stays mode-agnostic
- one resolver for a connector's token user, shared by the engine and the
directory refresh, replacing two copies that had already drifted
- Drive asks for permissions only on a run the engine says mirrors, drains its
permission pages through the shared Google helper, and spreads the ACL
context instead of relisting its fields
- Confluence memoises its three per-run lookups through one helper, keys pages
by one map, drops the unreachable space-principal branch, and shares the
cursor parser with the content listing
- nested Drive subgroups are read once per directory rather than once per
parent; the scheduler dispatches with bounded concurrency; batch loops use
chunkArray
Behaviour is unchanged except where the reviews found it wrong: a config edit
on a mirroring connector now strips caps and re-asserts the administrator, and
a crawl that is not mirroring no longer pulls a permission array per file.
* feat(search): unify source access and scale indexing
* fix(ci): align search cleanup checks and chart version
* fix(search): clean up source access and directory refresh
* improvement(search): remove Answer with Sim action
* improvement(search): simplify source setup and preserve account handoffs
* feat(assistant): use canonical search and personal integration accounts
* feat(assistant): connect personal accounts in chat and reuse desktop tools
* fix(slack): keep channel listing independent of credential storage
* improvement(assistant): streamline account checks and bound catalog caching
* improvement(search): unify source setup and member connection actions
* improvement(search): clarify setup requirements and focus account connections
* feat(search): add Gmail, Jira, GitHub and Calendar sources
* feat(search): document connector setup and harden source access
* feat(org): organization surface, app entry, and shared rail chrome
Adds the organization view at /o/[organizationId] — the viewer's own view of one
organization outside any workspace — and makes it the default landing for anyone
who belongs to an organization.
Organization surface
- Membership-gated layout with an explicit denial for non-members
- Sidebar mirroring the workspace rail: org header (empty menu for now), Search
and Collapse, Home / Integrations / Skills / Workspaces tabs, Chats section,
profile + help footer; the Workspaces tab grows a rail flyout when collapsed
- Empty pages for home, integrations, skills, workspaces, and chat/[chatId]
App entry
- New /home route resolves on the server to the organization home or the
workspace picker; /o resolves the same way
- Every default post-auth destination now points at APP_ENTRY_PATH via
lib/navigation/paths (proxy, login/signup/SSO/verify, OAuth callbacks,
billing return URLs, invites, impersonation, emails, desktop start route)
Shared chrome
- WorkspaceChrome takes its sidebar as a prop and exposes collapse/peek state
through a context, so both surfaces share one shell
- Flat hairline divider between rail and pane; no gutter, no radius, and no
layout transitions on collapse, fullscreen, or disclosure
- emcn useScrollEdges + scrollFadeClass: one scroll-edge fade that only fires
while content is hidden past an edge; dividers move to the neighboring block
- IdentityTile replaces every workspace/org avatar; no per-entity color in the
product (the DB column stays), Workspaces icon drawn in the house style
- Sidebar default width 256px, capped at 400px; SidebarTooltip, isNavItemActive,
getWorkspaceInitial, and the help URLs extracted to shared modules
* fix(search): support GitHub App grants and verify connector ACLs
* fix(search): deny revoked Atlassian grants and bound provider requests
* test(search): verify unchanged document indexing recovery
* fix(search): harden connector recovery and file lifecycle
* feat(org): home composer, page shell, and completion blue
Organization home gets the workspace chat's empty state, copied rather
than imported: the greeting, the composer with its resource, attachment,
skill, voice, and send controls, and a "Get started" list in the
suggested-actions chrome. Each step carries a ring that fills with a
check once done; "Create a workspace" reads the organization's real
workspace list, the rest wait on their signals.
Integrations, Skills, and Workspaces render through one OrganizationPage
shell: the workspace's top header bar, a fixed title/description/tabs/
search/action header in the home reading column, and a scroll region
with the sidebar's edge fade and padding. The active tab and the search
text live in the URL (`tab`, `q`) and are read back through
useOrganizationPageFilters, so a tab switch keeps the criteria and the
open field. Search expands leftward into a ChipInput with the canonical
quiet icon close button.
The emcn scroll-edge fade gains a horizontal axis (useScrollEdges
`axis: 'x'`, scrollFadeXClass) so the tab row scrolls under a fade when
it meets the controls beside it. `--brand-blue` joins the brand tokens
as the one blue for progress and completion states.
* feat(search): integrate organization sources and private assistant
* fix(org): clarify membership navigation and role-aware settings
* fix(ui): align organization and workspace sidebar interactions
* feat(org): organization surface UI, unified settings, and Sim Search setup
Home returns to the workspace chat's empty state, wired to the Assistant:
the greeting, the framed composer with the shared animated placeholder,
and a Get started list whose steps read completion from real state.
Search is its own tab, a home-style greeting over a pill field that docks
to the page head on submit with results scrolling beneath; a failed
search shows one quiet line and Try again rather than the server's text.
The Workspaces page is gone; the list lives in the sidebar above Chats
as rail chips with the sidebar's See more paging and the collapsed rail
flyout. The header dropdown is the organization card — mark, name,
member count — over a Settings row; the workspace header loses its
organizations list.
Organization settings become one surface: Account (General, the
organization's Subscription), Organization, Governance, and Sim Search,
with account sections rendered by the account plane's own renderer. The
settings Integrations section, admin-only, hosts every Sim Search source
with Set up, Manage, and a confirm-gated approval toggle; the Integrations
page shows every member the same list with only their own actions. The
setup's OAuth detours return to the settings section through one helper.
Sim Search in Slack is an outbound row in the shared settings sidebar.
The org layout seeds the viewer's profile through a shared prefetch, so
the footer paints hydrated and a page hydrating the same key no longer
mismatches. Loading pages are removed; content lands as it arrives.
* feat(search): enforce organization integration approvals
Persist admin approval independently of integration setup, wire the existing settings controls, and allow members to connect approved integrations. Enforce approval through shared search access checks and preserve indexed data for reapproval.
Fix organization scope handling in member sync and connector sync analytics. Include migration and coverage for approval authorization, member setup, and PostgreSQL access predicates.
* feat(connected-accounts): manage shared accounts at organization scope (#7586)
* feat(connected-accounts): manage shared accounts at organization scope
* fix(connected-accounts): handle state upgrades and align regression coverage
* fix(sidebar): align workspace switcher top spacing
* fix(connected-accounts): align organization settings and invitation flows
* chore(db): remove Drizzle Kit dependency patch
* fix(search): align operation registry coverage with integration approvals
---------
Co-authored-by: Waleed Latif <walif6@gmail.com>
* fix(search): harden organization setup and document access
---------
Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>
Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>
Co-authored-by: Theodore Li <theo@sim.ai>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Too many files changed for review (1993 files, 100 file limit). |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.