Skip to content

Extract query lib#823

Open
abnegate wants to merge 316 commits intomainfrom
feat-query-lib
Open

Extract query lib#823
abnegate wants to merge 316 commits intomainfrom
feat-query-lib

Conversation

@abnegate
Copy link
Copy Markdown
Member

@abnegate abnegate commented Mar 3, 2026

Summary by CodeRabbit

  • New Features

    • Rich relationship management, relationship hooks, tenant & permission filters, permission sync hooks, write hooks, transactional document/index/attribute operations, and new enums/value objects for permissions, types, operators and attributes.
  • Refactor

    • Public APIs migrated to object/value-object driven interfaces; adapters expose capability introspection and builder-driven SQL flows for clearer, consistent behavior.
  • Chores

    • Composer updated for local package support, prefer-stable/dev stability, and added a query dependency.
  • Tests

    • Test runner switched to parallelized Paratest with a dedicated destructive-test step.
  • Documentation

    • README examples updated to use new enum/value forms.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 3, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Large refactor: Query delegates to utopia-php/query BaseQuery; adapters, traits, hooks, and new domain types/enums (Attribute, Index, Relationship, Capability, etc.) migrate to value-object and capability models; CI, Docker, and composer updated to use a local query path and Paratest.

Changes

Cohort / File(s) Summary
Composer & CI
composer.json, .github/workflows/tests.yml, Dockerfile, docker-compose.yml
Add local utopia-php/query path repo, set minimum-stability: dev + prefer-stable, add brianium/paratest, switch CI to Paratest (4 processes), update Docker/build contexts and mounts for local query source.
Core Query
src/Database/Query.php
Make Query extend external BaseQuery, map old TYPE_* to enum-backed Method values, delegate parse/toArray and wrap exceptions.
Domain Value Objects
src/Database/Attribute.php, src/Database/Index.php, src/Database/Relationship.php
Add Attribute, Index, Relationship classes with toDocument/fromDocument and constructors; shift APIs to accept these objects.
Enums / Operators
src/Database/Capability.php, CursorDirection.php, OrderDirection.php, PermissionType.php, RelationType.php, RelationSide.php, SetType.php, OperatorType.php, src/Database/Operator.php
Introduce many enums (Capability, CursorDirection, OrderDirection, PermissionType, RelationType, RelationSide, SetType, OperatorType); refactor Operator to use OperatorType and remove string-constant groups.
Adapter Features & Surface
src/Database/Adapter.php, src/Database/Adapter/*, src/Database/Adapter/Feature/*
Introduce Feature interfaces (Attributes, Collections, Documents, Indexes, Relationships, Upserts, Timeouts, Transactions, Spatial, etc.), add supports()/capabilities(), add write hooks and row decoration, and change many adapter method signatures to use Attribute/Index/Relationship and enum types.
Adapter Implementations
src/Database/Adapter/MariaDB.php, MySQL.php, Mongo.php, SQLite.php, Pool.php
Large rewrites: builder-driven SQL, capability-driven logic, tenant/permission hooks, RetryClient for Mongo, upsert/index/attribute handling migrated to object/enums; many public signatures updated.
Traits: Collections/Documents/Indexes/Attributes/Relationships/Transactions
src/Database/Traits/*
Add comprehensive trait implementations for collection, document, index, attribute, relationship, and transaction lifecycles with validation, rollback, hooks, and events.
Hooks & Pipeline
src/Database/Hook/*
Add Read/Write hook interfaces and implementations (TenantWrite, PermissionWrite, MongoTenantFilter, MongoPermissionFilter, PermissionFilter, TenantFilter, RelationshipHandler, Relationship contract, WriteContext) to support tenanting, permissions, relationships and write decoration.
Permissions & Helpers
src/Database/Helpers/Permission.php, src/Database/Document.php
Switch permission constants to PermissionType enum values; Document::setAttribute now accepts SetType enum.
Utilities & Mirror/Pool
src/Database/Adapter/Mongo/RetryClient.php, src/Database/Pool.php, src/Database/Mirror.php
Add RetryClient wrapper for Mongo, update Pool and Mirror to accept domain objects and delegate capabilities/supports, propagate tenant settings.
Docs & Tasks
bin/tasks/relationships.php, README.md
Replace old onDelete/set-type constants with ForeignKeyAction and SetType enum value usages in tasks and README examples.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant Pool
    participant Adapter
    participant WriteHooks as "Write Hooks\n(PermissionWrite/TenantWrite)"
    participant Relationship as "RelationshipHandler"
    Client->>Pool: createDocument(collection, Document)
    Pool->>Adapter: createDocument(Document)
    Note right of Adapter: decorate row via write hooks
    Adapter->>WriteHooks: decorateRow(row, metadata)
    WriteHooks-->>Adapter: decorated row
    Adapter->>Adapter: persist row (DB)
    Adapter->>Relationship: afterDocumentCreate(collection, document)
    Relationship-->>Adapter: possibly update/populate related docs
    Adapter-->>Pool: Document (created)
    Pool-->>Client: Document (created)
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120 minutes

Possibly related PRs

  • Feat relationship updates #724 — relationship subsystem changes and relationship task updates (strong overlap with RelationshipHandler, Traits, and adapter relationship changes).
  • Upsert fix for postgres #613 — upsert-related adapter and API changes (overlaps with upsertDocuments and adapter upsert support).
  • Add operators base #713 — operator and OperatorType infrastructure (overlaps with Operator/OperatorType refactor and adapter operator handling).

Suggested reviewers

  • ArnabChatterjee20k

Poem

"🐰 I hopped through enums and hooks today,
Swapped old constants for types along the way.
Queries lean on a BaseQuery's steady light,
Adapters, hooks, and traits dance into the night —
A carrot-coded refactor, tidy and bright! 🥕"

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat-query-lib

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src/Database/Query.php (1)

62-76: Align cursor helper docblocks with the actual parameter type.

Lines 62 and 73 document Document, but the signature is mixed $value (Lines 65 and 76). Please update phpdoc to match actual behavior.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Query.php` around lines 62 - 76, The PHPDoc for
Query::cursorAfter and Query::cursorBefore incorrectly types the parameter as
Document while the method signatures accept mixed $value; update the `@param`
annotations in both docblocks to "@param mixed $value" (and keep "@return Query"
unchanged) so the docblocks match the actual signatures for the cursorAfter and
cursorBefore methods on the Query class.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@composer.json`:
- Around line 62-66: Replace the SSH Git URL in the composer.json "repositories"
entry with its HTTPS equivalent to avoid CI SSH auth issues; locate the
"repositories" array and update the url value currently set to
"git@github.com:utopia-php/query.git" to use
"https://github.com/utopia-php/query.git" so installs succeed in non-interactive
CI environments.

In `@src/Database/Query.php`:
- Around line 167-183: The code is using $limit as a fallback for both $offset
and $cursor which is incorrect; update the assignments in the TYPE_OFFSET and
TYPE_CURSOR_AFTER/TYPE_CURSOR_BEFORE branches so they use the actual first value
or null instead of $limit (i.e. change $values[0] ?? $limit to $values[0] ??
null) while keeping the existing checks that ignore subsequent offsets/cursors
and preserving $cursorDirection assignment (references: TYPE_OFFSET,
TYPE_CURSOR_AFTER, TYPE_CURSOR_BEFORE, $offset, $cursor, $values, $limit,
$cursorDirection, Database::CURSOR_AFTER, Database::CURSOR_BEFORE).

---

Nitpick comments:
In `@src/Database/Query.php`:
- Around line 62-76: The PHPDoc for Query::cursorAfter and Query::cursorBefore
incorrectly types the parameter as Document while the method signatures accept
mixed $value; update the `@param` annotations in both docblocks to "@param mixed
$value" (and keep "@return Query" unchanged) so the docblocks match the actual
signatures for the cursorAfter and cursorBefore methods on the Query class.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ba1ee9c and 44c19f9.

⛔ Files ignored due to path filters (1)
  • composer.lock is excluded by !**/*.lock
📒 Files selected for processing (2)
  • composer.json
  • src/Database/Query.php

Comment thread composer.json
Comment thread src/Database/Query.php Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
src/Database/Query.php (1)

161-177: ⚠️ Potential issue | 🟠 Major

Fix fallback leakage from limit into offset/cursor state.

Line 167 and Line 176 incorrectly use $limit as fallback, which can corrupt grouped query output when offset/cursor values are missing.

🔧 Proposed fix
                 case self::TYPE_OFFSET:
@@
-                    $offset = $values[0] ?? $limit;
+                    $offset = $values[0] ?? $offset;
                     break;
@@
                 case self::TYPE_CURSOR_AFTER:
                 case self::TYPE_CURSOR_BEFORE:
@@
-                    $cursor = $values[0] ?? $limit;
+                    $cursor = $values[0] ?? $cursor;
                     $cursorDirection = $method === self::TYPE_CURSOR_AFTER ? Database::CURSOR_AFTER : Database::CURSOR_BEFORE;
                     break;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Query.php` around lines 161 - 177, In Query:: (switch handling
TYPE_OFFSET / TYPE_CURSOR_AFTER / TYPE_CURSOR_BEFORE) remove the incorrect
fallback to $limit so $offset and $cursor aren't polluted: instead of assigning
$values[0] ?? $limit, only assign when an explicit value exists (e.g. check
isset($values[0]) or array_key_exists) and otherwise leave $offset/$cursor as
null; keep the first-occurrence guards intact and preserve setting
$cursorDirection when handling TYPE_CURSOR_AFTER vs TYPE_CURSOR_BEFORE
(Database::CURSOR_AFTER / Database::CURSOR_BEFORE).
🧹 Nitpick comments (1)
src/Database/Query.php (1)

53-73: Align cursor helper PHPDoc with the actual mixed parameter type.

Both docblocks still document @param Document $value, but the signatures now accept mixed.

📝 Suggested doc fix
-     * `@param` Document $value
+     * `@param` mixed $value
@@
-     * `@param` Document $value
+     * `@param` mixed $value
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Query.php` around lines 53 - 73, Update the PHPDoc for the Query
helper methods to match the actual parameter type: in the Query class update the
docblocks for cursorAfter and cursorBefore so the `@param` annotation is "@param
mixed $value" (instead of "@param Document $value") and keep the `@return`
annotation as "Query"; ensure these docblocks sit immediately above the
corresponding methods cursorAfter and cursorBefore to maintain accurate
IDE/typehinting.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In `@src/Database/Query.php`:
- Around line 161-177: In Query:: (switch handling TYPE_OFFSET /
TYPE_CURSOR_AFTER / TYPE_CURSOR_BEFORE) remove the incorrect fallback to $limit
so $offset and $cursor aren't polluted: instead of assigning $values[0] ??
$limit, only assign when an explicit value exists (e.g. check isset($values[0])
or array_key_exists) and otherwise leave $offset/$cursor as null; keep the
first-occurrence guards intact and preserve setting $cursorDirection when
handling TYPE_CURSOR_AFTER vs TYPE_CURSOR_BEFORE (Database::CURSOR_AFTER /
Database::CURSOR_BEFORE).

---

Nitpick comments:
In `@src/Database/Query.php`:
- Around line 53-73: Update the PHPDoc for the Query helper methods to match the
actual parameter type: in the Query class update the docblocks for cursorAfter
and cursorBefore so the `@param` annotation is "@param mixed $value" (instead of
"@param Document $value") and keep the `@return` annotation as "Query"; ensure
these docblocks sit immediately above the corresponding methods cursorAfter and
cursorBefore to maintain accurate IDE/typehinting.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 44c19f9 and d942f2b.

📒 Files selected for processing (3)
  • phpstan.neon
  • src/Database/Query.php
  • stubs/Query.stub

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
src/Database/Query.php (1)

159-169: ⚠️ Potential issue | 🟠 Major

Fix incorrect fallback variable usage for offset and cursor.

Lines 159 and 168 incorrectly fall back to $limit instead of the appropriate variable. This can silently corrupt the grouped query state.

  • Line 159: $offset = $values[0] ?? $limit should use $offset (or null)
  • Line 168: $cursor = $values[0] ?? $limit should use $cursor (or null)
🔧 Proposed fix
                 case self::TYPE_OFFSET:
                     // Keep the 1st offset encountered and ignore the rest
                     if ($offset !== null) {
                         break;
                     }

-                    $offset = $values[0] ?? $limit;
+                    $offset = $values[0] ?? null;
                     break;
                 case self::TYPE_CURSOR_AFTER:
                 case self::TYPE_CURSOR_BEFORE:
                     // Keep the 1st cursor encountered and ignore the rest
                     if ($cursor !== null) {
                         break;
                     }

-                    $cursor = $values[0] ?? $limit;
+                    $cursor = $values[0] ?? null;
                     $cursorDirection = $method === self::TYPE_CURSOR_AFTER ? Database::CURSOR_AFTER : Database::CURSOR_BEFORE;
                     break;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Query.php` around lines 159 - 169, The switch handling in
Query.php incorrectly falls back to $limit when assigning $offset and $cursor,
corrupting grouped query state; update the assignments in the cases that set
$offset and $cursor (the branches that assign $offset = $values[0] ?? $limit and
$cursor = $values[0] ?? $limit) to use a proper null fallback instead (e.g.
$offset = $values[0] ?? null and $cursor = $values[0] ?? null), keeping the
surrounding logic for cursorDirection (Database::CURSOR_AFTER /
Database::CURSOR_BEFORE) intact.
🧹 Nitpick comments (1)
src/Database/Query.php (1)

54-65: Consider tightening parameter type hint.

The PHPDoc indicates @param Document $value, but the signature uses mixed. Consider using Document as the type hint for better type safety, or update the PHPDoc to reflect the actual accepted types.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Query.php` around lines 54 - 65, The PHPDoc for cursorBefore
indicates the parameter is a Document but the method signature (and cursorAfter)
use mixed; update the signatures to use Document instead of mixed for
cursorBefore and cursorAfter (i.e., change the parameter type from mixed to
Document) to match the PHPDoc and improve type safety, or alternatively update
the PHPDoc to reflect mixed if these methods truly accept other types—adjust the
declarations in the Query class (cursorBefore, cursorAfter) so the docblock and
method signatures are consistent.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/Database/Query.php`:
- Around line 42-49: The method parseQuery currently declares a return type of
static but calls parent::parseQuery() which returns BaseQuery; change the method
signature to return BaseQuery instead of static and keep the try/catch that
wraps BaseQueryException into QueryException (i.e., update the return type on
parseQuery to BaseQuery to match parent::parseQuery and ensure the thrown
QueryException still wraps the original BaseQueryException).

---

Duplicate comments:
In `@src/Database/Query.php`:
- Around line 159-169: The switch handling in Query.php incorrectly falls back
to $limit when assigning $offset and $cursor, corrupting grouped query state;
update the assignments in the cases that set $offset and $cursor (the branches
that assign $offset = $values[0] ?? $limit and $cursor = $values[0] ?? $limit)
to use a proper null fallback instead (e.g. $offset = $values[0] ?? null and
$cursor = $values[0] ?? null), keeping the surrounding logic for cursorDirection
(Database::CURSOR_AFTER / Database::CURSOR_BEFORE) intact.

---

Nitpick comments:
In `@src/Database/Query.php`:
- Around line 54-65: The PHPDoc for cursorBefore indicates the parameter is a
Document but the method signature (and cursorAfter) use mixed; update the
signatures to use Document instead of mixed for cursorBefore and cursorAfter
(i.e., change the parameter type from mixed to Document) to match the PHPDoc and
improve type safety, or alternatively update the PHPDoc to reflect mixed if
these methods truly accept other types—adjust the declarations in the Query
class (cursorBefore, cursorAfter) so the docblock and method signatures are
consistent.

ℹ️ Review info

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d942f2b and b0a1faf.

📒 Files selected for processing (1)
  • src/Database/Query.php

Comment thread src/Database/Query.php Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Note

Due to the large number of review comments, Critical severity comments were prioritized as inline comments.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (6)
src/Database/Document.php (1)

232-243: ⚠️ Potential issue | 🟡 Minor

Update docblock to match the new parameter type.

The docblock still documents @param string $type but the actual parameter type is now SetType. This could confuse IDE autocompletion and static analysis tools.

📝 Proposed fix
     /**
      * Set Attribute.
      *
      * Method for setting a specific field attribute
      *
      * `@param` string $key
      * `@param` mixed $value
-     * `@param` string $type
+     * `@param` SetType $type
      *
      * `@return` static
      */
     public function setAttribute(string $key, mixed $value, SetType $type = SetType::Assign): static
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Document.php` around lines 232 - 243, Update the docblock for
the setAttribute method to match the new parameter type: replace the incorrect
"@param string $type" with "@param SetType $type" (or fully qualified
\Namespace\SetType if necessary) so IDEs and static analyzers reflect the actual
signature of setAttribute(string $key, mixed $value, SetType $type =
SetType::Assign) and keep the rest of the description intact.
src/Database/Mirror.php (1)

398-422: ⚠️ Potential issue | 🟠 Major

Use the filtered key when mirroring attribute renames.

beforeUpdateAttribute() can rewrite the attribute document, but the destination update still uses the caller’s raw $newKey. Any filter that renames destination attributes will drift on update.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Mirror.php` around lines 398 - 422, The destination update call
is still using the original caller $newKey even though
writeFilters->beforeUpdateAttribute may have rewritten the attribute document
(including renaming the key); update the call in Mirror:: where updateAttribute
is invoked to use the filtered key from the mutated $document (e.g.,
$document->getAttribute('key')) instead of $newKey, falling back to $newKey if
the document does not contain a key, so attribute renames applied by
beforeUpdateAttribute are honored; reference writeFilters,
beforeUpdateAttribute, $document, updateAttribute and $newKey when making the
change.
src/Database/Adapter/Pool.php (1)

101-104: ⚠️ Potential issue | 🟠 Major

Persist the timeout on the pool wrapper.

This override forwards the current call but never updates $this->timeout. Later delegate() calls therefore won’t reapply the timeout to newly borrowed adapters, so the setting is lost across pool checkouts.

⏱️ Proposed fix
 public function setTimeout(int $milliseconds, string $event = Database::EVENT_ALL): void
 {
+    parent::setTimeout($milliseconds, $event);
     $this->delegate(__FUNCTION__, \func_get_args());
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/Pool.php` around lines 101 - 104, The setTimeout
override in Pool::setTimeout(int $milliseconds, string $event =
Database::EVENT_ALL) forwards the call to delegate() but fails to persist the
value to the pool wrapper, so $this->timeout is never updated and newly borrowed
adapters don't get the timeout; update the Pool::setTimeout implementation to
assign the passed $milliseconds (and $event if you store per-event) to the
pool's internal property ($this->timeout or a per-event map) before or after
calling $this->delegate(__FUNCTION__, func_get_args()), ensuring future
delegate() calls reapply the stored timeout to adapters checked out from the
pool.
src/Database/Query.php (1)

240-257: ⚠️ Potential issue | 🟠 Major

Cursor queries stop working after toArray() / groupForDatabase() round-trips.

Documents::find() expects $cursor to be a Document so it can read order keys and $collection. toArray() collapses cursor documents to $id, and groupForDatabase() returns that scalar unchanged, so any serialized cursor query will fail when pagination code calls getAttribute() or getCollection() on it.

🔧 Proposed fix
-                if ($value instanceof Document && in_array($this->method, [Method::CursorAfter, Method::CursorBefore])) {
-                    $value = $value->getId();
+                if ($value instanceof Document && \in_array($this->method, [Method::CursorAfter, Method::CursorBefore], true)) {
+                    $value = $value->getArrayCopy();
                 }
                 $array['values'][] = $value;
             }
         }
@@
+        $cursor = $grouped->cursor;
+        if (\is_array($cursor)) {
+            $cursor = new Document($cursor);
+        }
+
         return [
             'filters' => $filters,
             'selections' => $selections,
             'limit' => $grouped->limit,
             'offset' => $grouped->offset,
             'orderAttributes' => $grouped->orderAttributes,
             'orderTypes' => $orderTypes,
-            'cursor' => $grouped->cursor,
+            'cursor' => $cursor,
             'cursorDirection' => $cursorDirection,
         ];

Also applies to: 280-317

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Query.php` around lines 240 - 257, The cursor-serialization code
in Query::toArray() currently replaces Document instances with their id for
cursor methods (Method::CursorAfter, Method::CursorBefore), which breaks
Documents::find() that expects full Document objects after groupForDatabase()
round-trips; change the branch so that when $value instanceof Document you
serialize it with $value->toArray() (not ->getId()) so the document structure is
preserved and can be rehydrated on read (also apply the same fix in the
analogous block around lines 280-317), making sure
groupForDatabase()/unserialize logic continues to round-trip these document
arrays back into Document objects so getAttribute() and getCollection() still
work.
src/Database/Adapter/Mongo.php (2)

1575-1592: ⚠️ Potential issue | 🟠 Major

Keep replacement-style updates for schemaless collections.

This now always wraps the payload in $set. When defined attributes are disabled, missing keys are supposed to disappear on update; with $set they persist forever because the old document is never replaced.

Suggested fix
             $options = $this->getTransactionOptions();
-            $updateQuery = [
-                '$set' => $record,
-            ];
+            $updateQuery = $this->supportForAttributes
+                ? ['$set' => $record]
+                : $record;
             $this->client->update($name, $filters, $updateQuery, $options);

Based on learnings: In src/Database/Adapter/Mongo.php, when getSupportForAttributes() returns false (schemaless mode), the updateDocument method intentionally uses a raw document without $set operator for replacement-style updates, as confirmed by the repository maintainer ArnabChatterjee20k.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/Mongo.php` around lines 1575 - 1592, The updateDocument
method currently always wraps the payload in a $set which prevents
replacement-style updates for schemaless collections; change updateDocument so
that after preparing $record (and unsetting '_id') it checks
$this->getSupportForAttributes(): if true keep $updateQuery = ['$set' =>
$record], but if false use $updateQuery = $record (a replacement document)
before calling $this->client->update($name, $filters, $updateQuery, $options) so
missing keys are removed for schemaless collections; refer to updateDocument and
getSupportForAttributes() to locate where to branch.

1542-1553: ⚠️ Potential issue | 🟠 Major

Scope the post-insert readback.

In shared-table mode _uid is only unique within _tenant. This follow-up find() reads back by _uid alone, so a custom ID collision can hydrate the newly created document from another tenant's row instead of the one that was just inserted.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/Mongo.php` around lines 1542 - 1553, The post-insert
readback in insertDocument uses a filter of ['_uid' => $document['_uid']], which
can collide across tenants; update the filter used in the inner client->find
call to scope by both '_uid' and '_tenant' (e.g. include '_tenant' =>
$document['_tenant'] when present) so the find() returns the document from the
same tenant that was just inserted, adjusting any variable names (filters) used
around client->find accordingly.
🟠 Major comments (19)
composer.json-64-72 (1)

64-72: ⚠️ Potential issue | 🟠 Major

Path repository will fail in CI environments.

The path repository pointing to ../query is only valid for local development where the sibling query directory exists. In CI environments (GitHub Actions, etc.) and for other contributors cloning this repo, this path won't exist and composer install will fail.

For CI compatibility, either:

  1. Publish utopia-php/query to Packagist before merging
  2. Use the official VCS repository URL as a fallback
🔧 Proposed fix using VCS repository
     "repositories": [
         {
             "type": "path",
             "url": "../query",
             "options": {
                 "symlink": true
             }
-        }
+        },
+        {
+            "type": "vcs",
+            "url": "https://github.com/utopia-php/query.git"
+        }
     ],

Note: Composer will prefer the path repository if available, falling back to VCS otherwise.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@composer.json` around lines 64 - 72, The current "repositories" entry uses a
path repo ("type": "path", "url": "../query") which will break CI because the
sibling ../query won't exist; update composer.json to remove or guard the
path-only repository and add a fallback VCS repository entry pointing to the
public repo URL for utopia-php/query (or publish to Packagist and use its
package name), so Composer can use the VCS source in CI while still preferring
the local path when present; modify the "repositories" array to include the
official VCS URL (or switch to Packagist) alongside or instead of the existing
path entry.
src/Database/Hook/MongoPermissionFilter.php-26-27 (1)

26-27: ⚠️ Potential issue | 🟠 Major

Escape regex metacharacters in role names to prevent regex injection.

Role names are directly interpolated into the regex pattern without escaping. If a role contains regex metacharacters (e.g., (, ), |, ., *), it could break the regex or enable ReDoS attacks.

🛡️ Proposed fix to escape role names
-        $roles = \implode('|', $this->authorization->getRoles());
+        $roles = \implode('|', \array_map(
+            fn($role) => \preg_quote($role, '/'),
+            $this->authorization->getRoles()
+        ));
         $filters['_permissions']['$in'] = [new Regex("{$forPermission}\\(\".*(?:{$roles}).*\"\\)", 'i')];
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Hook/MongoPermissionFilter.php` around lines 26 - 27, The code
builds a Regex from role names without escaping them, risking regex injection;
update the logic that creates $roles (from $this->authorization->getRoles()) to
escape each role with a regex-escaping function (e.g., preg_quote) before
joining them, then use the escaped string when constructing the Regex used in
the $filters['_permissions']['$in'] assignment (the
Regex("{$forPermission}\\(\".*(?:{$roles}).*\"\\)", 'i') instantiation) so
special characters in role names are neutralized.
src/Database/Traits/Attributes.php-998-1015 (1)

998-1015: ⚠️ Potential issue | 🟠 Major

Rollback the previous default when metadata persistence fails.

If the adapter update succeeds and updateMetadata() then fails, the rollback model recreates the old column without its previous default. The physical schema can remain mutated while metadata rolls back.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Attributes.php` around lines 998 - 1015, The rollback
Attribute is missing the original default value so if the adapter update
succeeds but updateMetadata() fails the physical column loses its default;
modify the rollback model creation (the Attribute instance assigned to
$rollbackAttrModel) to include the previous default (e.g., $originalDefault) and
ensure whatever variable holds the prior default is captured and passed into the
Attribute constructor, then keep the rollback closure that calls
$this->adapter->updateAttribute($collection, $rollbackAttrModel, $originalKey)
unchanged so the adapter will restore the column including its original default
when metadata persistence fails.
src/Database/Traits/Relationships.php-893-902 (1)

893-902: ⚠️ Potential issue | 🟠 Major

Preserve the original side during rollback.

This recreation hard-codes RelationSide::Parent. If a child-side delete fails after the physical schema change, rollback will restore the relationship on the wrong side.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Relationships.php` around lines 893 - 902, The rollback
recreates the Relationship with a hard-coded RelationSide::Parent which can
restore the relation on the wrong side; change the code that builds
$recreateRelModel in the rollback to use the original relationship side (e.g.,
the existing side value from the original Relationship instance or the local
$side variable) instead of RelationSide::Parent so the recreated Relationship
preserves the original side; update the constructor call that sets side to use
that original side source (RelationSide value obtained from the original model)
when constructing $recreateRelModel.
src/Database/Traits/Attributes.php-281-284 (1)

281-284: ⚠️ Potential issue | 🟠 Major

Rollback should only delete attributes created in this batch.

$attributeDocuments also contains schema-only orphans that already existed before this call. If one new attribute is created and updateMetadata() fails, cleanupAttributes() will delete those pre-existing columns too.

Also applies to: 317-323

src/Database/Traits/Attributes.php-546-559 (1)

546-559: ⚠️ Potential issue | 🟠 Major

Invalidate both collection and metadata caches after these mutations.

createAttribute(), updateAttribute(), and deleteAttribute() purge cached collection state and the metadata document. updateAttributeMeta() does neither, and renameAttribute() still leaves the metadata document cache warm, so callers can read stale attribute definitions after a successful write.

Also applies to: 1284-1292

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Attributes.php` around lines 546 - 559, After performing
attribute-mutating operations (specifically in updateAttributeMeta() and
renameAttribute()), ensure you clear both the collection cache and the metadata
document cache just like createAttribute()/updateAttribute()/deleteAttribute()
do: call the same cache invalidation logic used elsewhere after the
$this->updateMetadata(...) call (and before returning), and also ensure any
cache keys for the metadata document are purged so callers don't read stale
attribute definitions; reference updateAttributeMeta(), renameAttribute(), and
the existing $this->updateMetadata(...) / self::EVENT_ATTRIBUTE_UPDATE flow to
locate the correct spot to add the invalidation.
src/Database/Adapter.php-685-697 (1)

685-697: ⚠️ Potential issue | 🟠 Major

Don’t report unsupported relationship operations as success.

Traits\Relationships uses these return values to decide whether to update metadata. Returning true here lets an adapter without a real implementation claim success while leaving the physical schema unchanged.

🧱 Proposed fix
 public function createRelationship(Relationship $relationship): bool
 {
-    return true;
+    return false;
 }
 
 public function updateRelationship(Relationship $relationship, ?string $newKey = null, ?string $newTwoWayKey = null): bool
 {
-    return true;
+    return false;
 }
 
 public function deleteRelationship(Relationship $relationship): bool
 {
-    return true;
+    return false;
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter.php` around lines 685 - 697, The Adapter currently
reports success for unsupported relationship operations; update the methods
createRelationship(Relationship $relationship), updateRelationship(Relationship
$relationship, ?string $newKey = null, ?string $newTwoWayKey = null) and
deleteRelationship(Relationship $relationship) to not claim success — return
false (or throw a clear UnsupportedOperationException) instead of returning true
so Traits\Relationships won't assume metadata was updated; ensure the change is
applied to those method bodies and keep the signatures unchanged.
src/Database/Traits/Relationships.php-518-579 (1)

518-579: ⚠️ Potential issue | 🟠 Major

Update both relationship metadata documents atomically.

These updateAttributeMeta() calls commit independently. If the first collection update succeeds and the second one fails, the catch only reverts the adapter rename; it never restores the already-persisted metadata change on the first collection.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Relationships.php` around lines 518 - 579, The metadata
updates via updateAttributeMeta on the primary collection, the related
collection (oldTwoWayKey), and the junction (when ManyToMany) must be performed
atomically; currently if one update fails the others remain persisted. Fix by
either wrapping all metadata updates and the adapter rename in a single DB
transaction (begin/commit/rollback) so any failure rolls back all changes, or if
transactions are not available, capture the existing attribute state for
collection->getId()/$id, relatedCollection->getId()/oldTwoWayKey, and junction
entries (if RelationType::ManyToMany) before applying updates and, inside the
catch, call updateAttributeMeta to restore those saved states (and ensure
purgeCachedCollection is similarly reverted). Update logic references:
updateAttributeMeta, getJunctionCollection, withRetries->purgeCachedCollection,
and the adapter->updateRelationship/Relationship model so both metadata and
adapter rename are consistent.
src/Database/Traits/Relationships.php-143-157 (1)

143-157: ⚠️ Potential issue | 🟠 Major

Check the related collection for twoWayKey collisions before creating the relationship.

The duplicate scan only inspects the source collection. checkAttribute() only enforces limits/width, so an existing attribute on the related collection with the same $twoWayKey is not rejected before partial relationship creation starts.

Also applies to: 191-192

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Relationships.php` around lines 143 - 157, The
duplicate-check currently inspects only the source collection attributes (loop
over $attributes) and misses collisions on the related collection; update the
logic in the Relationships trait where you check for existing relationship
attributes (the foreach that compares ColumnType::Relationship->value, twoWayKey
and relatedCollection id) to also fetch and iterate the related collection's
attributes (use $relatedCollection->getAttribute('attributes', [])) and compare
their options['twoWayKey'] case-insensitively to $twoWayKey and their
relatedCollection id to the source collection id, throwing
DuplicateException('Related attribute already exists') on conflict; apply the
same symmetric check to the other similar block referenced by checkAttribute()
(the block around the other relationship-creation path) so both creation flows
validate twoWayKey collisions on the opposite collection before proceeding.
src/Database/Traits/Attributes.php-572-576 (1)

572-576: ⚠️ Potential issue | 🟠 Major

Keep the metadata-only helpers behind the same invariants as updateAttribute().

updateAttributeRequired() can mark an attribute required while leaving its old default in place, updateAttributeFilters() can remove mandatory filters like datetime, and updateAttributeDefault() never checks vector length against the configured dimension count. These helpers can persist attribute metadata that the full update path would reject.

Also applies to: 627-631, 644-653

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Attributes.php` around lines 572 - 576, The three
metadata-only helpers (updateAttributeRequired, updateAttributeFilters,
updateAttributeDefault) bypass the same validation/invariants enforced by
updateAttribute and can persist invalid metadata; change each helper to call the
full update path or reuse its validation logic by delegating to updateAttribute
(or to the common validation function used by updateAttribute) instead of
directly mutating via updateAttributeMeta: ensure updateAttributeRequired
validates compatibility between required flag and the existing default,
updateAttributeFilters preserves/enforces mandatory filters (e.g., datetime) and
validates any removed filters against attribute type, and updateAttributeDefault
enforces vector length vs configured dimension count and any type-specific
constraints before persisting. Ensure you reference and reuse updateAttribute or
its shared validators so all invariants are consistently applied.
src/Database/Hook/PermissionWrite.php-131-136 (1)

131-136: ⚠️ Potential issue | 🟠 Major

Route permission reads/deletes through WriteContext::execute.

The insert path uses the context executor, but these branches call execute() directly and some of the delete helpers ignore the boolean result entirely. A failed permission mutation can therefore leave <collection>_perms out of sync with the document write.

Also applies to: 187-192, 208-215, 227-231, 262-266

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Hook/PermissionWrite.php` around lines 131 - 136, The permission
delete branches in PermissionWrite (use of ($context->newBuilder),
$removeBuilder->delete(), and the subsequent ($context->executeResult) returning
$deleteStmt) call $deleteStmt->execute() directly and ignore the WriteContext
executor and result; change these to run deletes through the WriteContext
executor (use WriteContext::execute on the prepared statement returned by
($context->executeResult) with Database::EVENT_PERMISSIONS_DELETE), check the
boolean return value and propagate/handle failures (throw or abort the parent
write) so permission mutations cannot silently fail; apply the same fix pattern
to the other affected blocks that build/delete perms (the sections analogous to
lines 187-192, 208-215, 227-231, 262-266).
src/Database/Traits/Relationships.php-493-515 (1)

493-515: ⚠️ Potential issue | 🟠 Major

Don’t treat the unchanged source key as proof that the rename already happened.

When only $newTwoWayKey changes, $actualNewKey still points at the current source column. The orphan-recovery fallback will then mark $adapterUpdated = true as soon as it sees that existing column, even if the related/junction rename never happened.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Relationships.php` around lines 493 - 515, The recovery
logic in Relationships.php incorrectly treats finding $actualNewKey in schema as
proof the rename completed; instead, when catching the Throwable you must verify
the rename truly occurred by checking that the new column exists AND the
original source column was removed (or that the two-way/junction column change
also exists) — use getSchemaAttributes(), $this->adapter->filter($actualNewKey)
and also compare against the filtered original/source key (e.g., filteredOldKey)
and/or the filtered $newTwoWayKey to ensure the found column isn't simply the
unchanged source; only set $adapterUpdated = true if the new name exists AND the
old name does not (or the corresponding two-way column was updated), otherwise
rethrow the DatabaseException (preserving $e as previous).
src/Database/Traits/Documents.php-970-975 (1)

970-975: ⚠️ Potential issue | 🟠 Major

Use the hook's returned document in bulk updates.

updateDocument() reassigns afterDocumentUpdate(), but updateDocuments() ignores the returned value. Any relationship hook that rewrites the payload will be skipped on this path.

🔧 Proposed fix
                     $hook = $this->relationshipHook;
                     if ($hook?->isEnabled()) {
-                        $this->silent(fn () => $hook->afterDocumentUpdate($collection, $document, $new));
+                        $new = $this->silent(fn () => $hook->afterDocumentUpdate($collection, $document, $new));
                     }
 
                     $document = $new;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Documents.php` around lines 970 - 975, The bulk-update
path is ignoring the relationship hook's return value: in the block using
$this->relationshipHook and calling $hook->afterDocumentUpdate($collection,
$document, $new), capture and use the hook's returned document (replace $new
with the hook return when non-null) before assigning to $document so any payload
rewrites by afterDocumentUpdate are preserved; update the logic around
relationshipHook/isEnabled and the assignment to $document to use the hook's
return value (from afterDocumentUpdate) instead of discarding it.
src/Database/Traits/Documents.php-401-409 (1)

401-409: ⚠️ Potential issue | 🟠 Major

Run castingAfter() on single-document creates even when hooks are off.

This path only normalizes adapter output inside the relationship-hook branch. MariaDB::createDocument() returns the pre-cast document instance, so a single create can return DB-shaped values while createDocuments() returns normalized ones.

🔧 Proposed fix
         $hook = $this->relationshipHook;
         if ($hook !== null && !$hook->isInBatchPopulation() && $hook->isEnabled()) {
             $fetchDepth = $hook->getWriteStackCount();
             $documents = $this->silent(fn () => $hook->populateDocuments([$document], $collection, $fetchDepth));
-            $document = $this->adapter->castingAfter($collection, $documents[0]);
+            $document = $documents[0];
         }
 
+        $document = $this->adapter->castingAfter($collection, $document);
         $document = $this->casting($collection, $document);
         $document = $this->decode($collection, $document);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Documents.php` around lines 401 - 409, The current code
only runs $this->adapter->castingAfter(...) when $this->relationshipHook is
present and enabled, causing single-document creates to return DB-shaped values;
move or duplicate the adapter post-processing so castingAfter is applied for
single-document flows regardless of the relationship hook state: ensure that
after the potential hook population block you call
$this->adapter->castingAfter($collection, $document) (using the same normalized
$documents[0] result when hook ran) before running $this->casting(...) and
$this->decode(...), so both createDocument() and createDocuments() return
consistently normalized documents; use the existing symbols relationshipHook,
populateDocuments, castingAfter, casting, decode and silent to implement this
change.
src/Database/Traits/Documents.php-274-301 (1)

274-301: ⚠️ Potential issue | 🟠 Major

Preserve relationship roots when pruning selected fields.

$attributesToKeep only records the literal selector. A query like select(['author.name']) will therefore remove the populated author attribute itself, and aliased projections are dropped for the same reason. Please retain the root segment of dotted paths, plus any projection alias, before removing attributes.

Based on learnings: Repo utopia-php/database: Relationship selects are always evaluated in the main alias context (no per-collection aliasing). In Utopia\Database\Database::applySelectFiltersToDocuments, do not rely on Query::getAlias() for relationships; instead, preserve the root of dotted paths and any projection alias (Query::getAs()) when filtering attributes.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Documents.php` around lines 274 - 301, The pruning logic
in applySelectFiltersToDocuments builds $attributesToKeep from selectQueries by
using literal selectors only, which drops relationship root attributes and
aliased projections (e.g., select(['author.name']) or projections with
Query::getAs()). Update the loop that populates $attributesToKeep (the foreach
over $selectQueries and getValues()) to also: for each selector string preserve
its root segment (the part before the first dot) and, if Query::getAs() is
present for that query, preserve the alias name as well; do not rely on
Query::getAlias() for relationship selects. Ensure these root keys and
projection aliases are added to $attributesToKeep before the
wildcard/internal-key checks so removeAttribute only strips truly unselected
fields.
src/Database/Traits/Documents.php-1174-1215 (1)

1174-1215: ⚠️ Potential issue | 🟠 Major

Don't drop brand-new empty documents as a no-op.

When $old is empty and the incoming document has no user fields/operators, $hasChanges stays false and the item is removed from the batch. upsertDocument() then falls back to getDocument() and returns an empty document instead of creating a valid empty record.

🔧 Proposed fix
-            if (!$hasChanges) {
+            if (!$hasChanges && !$old->isEmpty()) {
                 // If not updating a single attribute and the document is the same as the old one, skip it
                 unset($documents[$key]);
                 continue;
             }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Documents.php` around lines 1174 - 1215, The current
change-detection logic in the Documents trait can wrongly treat brand-new empty
incoming docs as unchanged and drop them from $documents[$key], because when
$old is empty and regularUpdatesUserOnly and $operators are empty $hasChanges
remains false; update the logic in the block that computes $hasChanges (the
checks that reference $operators, $attribute, $skipPermissionsUpdate, $old,
regularUpdatesUserOnly, self::INTERNAL_ATTRIBUTES) to explicitly mark a document
as changed when $old is null/empty (new record) even if no user fields/operators
are present so upsertDocument() will create the empty record instead of falling
back to getDocument(); ensure this new case still respects internal attribute
filtering and the subsequent unset($documents[$key]) skip only applies to truly
identical existing documents.
src/Database/Adapter/MariaDB.php-603-609 (1)

603-609: ⚠️ Potential issue | 🟠 Major

Use side-aware junction naming when renaming many-to-many columns.

deleteRelationship() computes the junction table name differently for parent vs child sides, but updateRelationship() always uses <collection>_<related>. Child-side renames will target the wrong junction table.

🔧 Proposed fix
-                $junctionName = '_' . $collection->getSequence() . '_' . $relatedCollection->getSequence();
+                $junctionName = $side === RelationSide::Parent
+                    ? '_' . $collection->getSequence() . '_' . $relatedCollection->getSequence()
+                    : '_' . $relatedCollection->getSequence() . '_' . $collection->getSequence();
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/MariaDB.php` around lines 603 - 609,
updateRelationship() is using a fixed junction name ('_' .
$collection->getSequence() . '_' . $relatedCollection->getSequence()) which
ignores the relationship side and causes child-side renames to target the wrong
junction table; change the junction name computation in updateRelationship() to
mirror the logic used in deleteRelationship() (i.e., compute $junctionName based
on the $side or whichever variable deleteRelationship() uses to decide order of
$collection->getSequence() and $relatedCollection->getSequence()), then use that
side-aware $junctionName when calling $renameCol for $key/$newKey and
$twoWay/$newTwoWayKey so renames operate on the correct many-to-many table.
src/Database/Adapter/MariaDB.php-1211-1228 (1)

1211-1228: ⚠️ Potential issue | 🟠 Major

Add handlers for TYPE_COVERS, TYPE_NOT_COVERS, TYPE_SPATIAL_EQUALS, and TYPE_NOT_SPATIAL_EQUALS in the spatial query match statement.

The validator in src/Database/Validator/Queries.php allows these four new spatial query types, but the match statement in src/Database/Adapter/MariaDB.php (lines 1211-1228) does not handle them. These queries will throw Unknown spatial query method at runtime when executed. The same issue exists in src/Database/Adapter/Postgres.php.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/MariaDB.php` around lines 1211 - 1228, The match in
MariaDB.php handling spatial query methods misses Query::TYPE_COVERS,
Query::TYPE_NOT_COVERS, Query::TYPE_SPATIAL_EQUALS and
Query::TYPE_NOT_SPATIAL_EQUALS; add match arms that map TYPE_COVERS to
"ST_Covers({$alias}.{$attribute}, {$geom})", TYPE_NOT_COVERS to "NOT
ST_Covers(...)", TYPE_SPATIAL_EQUALS to "ST_SpatialEquals({$alias}.{$attribute},
{$geom})" and TYPE_NOT_SPATIAL_EQUALS to "NOT ST_SpatialEquals(...)" (use the
same pattern/templating as the existing arms such as Query::TYPE_CONTAINS and
Query::TYPE_EQUAL), and apply the identical changes to the corresponding match
in Postgres.php so both adapters support the four new Query types.
src/Database/Adapter/Mongo.php-1064-1069 (1)

1064-1069: ⚠️ Potential issue | 🟠 Major

Resolve partial-index types by attribute name, not loop offset.

Line 1165 builds $indexAttributeTypes as [$attrName => $type], but Line 1067 reads $indexAttributeTypes[$i]. That falls back to string for numeric/date fields and can exclude the real field type from the partial index entirely.

Suggested fix
         if (in_array($type, [IndexType::Unique, IndexType::Key])) {
             $partialFilter = [];
             foreach ($attributes as $i => $attr) {
-                $attrType = $indexAttributeTypes[$i] ?? ColumnType::String->value; // Default to string if type not provided
+                $sourceAttribute = $index->attributes[$i] ?? $attr;
+                $attrType = $indexAttributeTypes[$sourceAttribute] ?? ColumnType::String->value;
                 $attrType = $this->getMongoTypeCode($attrType);
                 $partialFilter[$attr] = ['$exists' => true, '$type' => $attrType];
             }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/Mongo.php` around lines 1064 - 1069, The partial index
builder in Mongo.php is using the loop index to look up attribute types but
$indexAttributeTypes is keyed by attribute name; update the lookup in the loop
that constructs $partialFilter (in the block handling IndexType::Unique and
IndexType::Key) to use the attribute name ($attr) as the key instead of $i, e.g.
fetch the type with $indexAttributeTypes[$attr] ?? ColumnType::String->value and
then pass that into $this->getMongoTypeCode so the correct field type is used
for the partial index filter.
🟡 Minor comments (12)
docker-compose.yml-6-7 (1)

6-7: ⚠️ Potential issue | 🟡 Minor

Document local development setup requirement for sibling query repository.

The volume mount ../query/src:/usr/src/code/vendor/utopia-php/query/src and build context .. assume a specific local directory structure where the query repository exists as a sibling. This dependency is not documented in CONTRIBUTING.md. While the volume mount failure is non-fatal in CI (the container starts regardless), local development with docker compose up -d --build will fail without the sibling query repository, making setup unclear for new contributors.

Also applies to: 21-21

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docker-compose.yml` around lines 6 - 7, The docker-compose configuration uses
a sibling repo via the volume mount
'../query/src:/usr/src/code/vendor/utopia-php/query/src' and a build context
'..' (with dockerfile database/Dockerfile), which must be documented; update
CONTRIBUTING.md to state that the `query` repository must exist as a sibling (or
provide steps to clone it) before running `docker compose up -d --build`, note
the mount is optional for CI and point to the alternative (remove/disable the
volume or use a docker-compose.override.yml) for contributors who cannot have
the sibling repo, and mirror this documentation for the other occurrence
referenced (line 21) so local dev setup is explicit.
README.md-636-638 (1)

636-638: ⚠️ Potential issue | 🟡 Minor

Add import statement for ForeignKeyAction in the example.

The example shows ForeignKeyAction::Cascade->value but doesn't include the necessary use statement. This could confuse developers trying to use the code.

📝 Suggested addition before line 636
use Utopia\Database\ForeignKeyAction;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` around lines 636 - 638, Add a use/import for the ForeignKeyAction
enum in the example so the symbols like ForeignKeyAction::Cascade->value,
ForeignKeyAction::SetNull->value, and ForeignKeyAction::Restrict->value resolve;
specifically, add the statement to import the class (e.g., use
Utopia\Database\ForeignKeyAction;) near the top of the snippet where the example
code begins so ForeignKeyAction is available in that example context.
README.md-758-776 (1)

758-776: ⚠️ Potential issue | 🟡 Minor

Add import statement for SetType in the example.

Similar to ForeignKeyAction, the SetType enum usage examples should include the import statement for clarity.

📝 Suggested addition before line 758
use Utopia\Database\SetType;
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@README.md` around lines 758 - 776, Add the missing import for the SetType
enum used in the README examples by inserting the statement importing SetType
(e.g., use Utopia\Database\SetType;) near the other example imports so the
usages of SetType::Assign / SetType::Append / SetType::Prepend in the
setAttribute examples resolve; ensure it appears alongside existing imports like
Permission and Role so the sample snippet is self-contained and clear.
src/Database/Relationship.php-21-31 (1)

21-31: ⚠️ Potential issue | 🟡 Minor

toDocument() omits collection and key properties.

The toDocument() method doesn't serialize $this->collection or $this->key, but fromDocument() accepts both as inputs (collection as parameter, key from attribute). This asymmetry may cause data loss during round-trip serialization if these fields need to be preserved.

If this is intentional (because these fields come from external context), consider documenting this behavior.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Relationship.php` around lines 21 - 31, The toDocument() method
on Relationship currently omits $this->collection and $this->key while
fromDocument() (and the static fromDocument method) accept collection and key,
causing asymmetrical serialization; modify Relationship::toDocument() to include
'collection' => $this->collection and 'key' => $this->key in the returned
Document (or, if omission is intentional, add a clear docblock on
Relationship::toDocument() and fromDocument() explaining that collection and key
are provided externally and therefore not persisted) so round-trip serialization
is consistent.
src/Database/Hook/RelationshipHandler.php-243-243 (1)

243-243: ⚠️ Potential issue | 🟡 Minor

Unused loop variable $index.

The $index variable from the foreach loop is declared but never used. This was correctly flagged by static analysis.

🔧 Proposed fix
-        foreach ($relationships as $index => $relationship) {
+        foreach ($relationships as $relationship) {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Hook/RelationshipHandler.php` at line 243, The foreach in
RelationshipHandler.php declares an unused loop variable $index; change the loop
signature from foreach ($relationships as $index => $relationship) to foreach
($relationships as $relationship) (or use foreach ($relationships as $_ =>
$relationship) if you prefer an explicit unused key) so the unused $index is
removed, keeping the loop body using $relationship unchanged.
src/Database/Hook/RelationshipHandler.php-1803-1814 (1)

1803-1814: ⚠️ Potential issue | 🟡 Minor

Same shadowing issue: $document in deleteCascade.

The loop variable $document shadows the method parameter, same issue as in deleteSetNull. The loop iterates over junction documents while the original parameter is the document being deleted.

🔧 Rename loop variable
-                foreach ($junctions as $document) {
+                foreach ($junctions as $junctionDoc) {
                     if ($side === RelationSide::Parent->value) {
                         $this->db->deleteDocument(
                             $relatedCollection->getId(),
-                            $document->getAttribute($key)
+                            $junctionDoc->getAttribute($key)
                         );
                     }
                     $this->db->deleteDocument(
                         $junction,
-                        $document->getId()
+                        $junctionDoc->getId()
                     );
                 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Hook/RelationshipHandler.php` around lines 1803 - 1814, In
deleteCascade, the foreach loop uses $document which shadows the method
parameter $document; rename the loop variable (e.g., to $junctionDocument or
$junctionDoc) wherever it's used within the loop so the parameter $document
remains the original document being deleted, and update the two deleteDocument
calls to reference the renamed loop variable
($junctionDocument->getAttribute($key) and $junctionDocument->getId()) while
leaving the method parameter untouched.
src/Database/Attribute.php-76-92 (1)

76-92: ⚠️ Potential issue | 🟡 Minor

fromArray() omits status and options fields.

Unlike fromDocument(), the fromArray() method doesn't extract status and options from the input array. This asymmetry could cause data loss when attributes with these fields are serialized via toDocument() and then reconstructed via fromArray().

🔧 Proposed fix to include status and options
     public static function fromArray(array $data): self
     {
         $type = $data['type'] ?? 'string';
 
         return new self(
             key: $data['$id'] ?? $data['key'] ?? '',
             type: $type instanceof ColumnType ? $type : ColumnType::from($type),
             size: $data['size'] ?? 0,
             required: $data['required'] ?? false,
             default: $data['default'] ?? null,
             signed: $data['signed'] ?? true,
             array: $data['array'] ?? false,
             format: $data['format'] ?? null,
             formatOptions: $data['formatOptions'] ?? [],
             filters: $data['filters'] ?? [],
+            status: $data['status'] ?? null,
+            options: $data['options'] ?? null,
         );
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Attribute.php` around lines 76 - 92, fromArray() currently
ignores the 'status' and 'options' keys so reconstructing an Attribute via
Attribute::fromArray loses data present in toDocument()/fromDocument(); update
Attribute::fromArray to read 'status' and 'options' from the input array (e.g.
$data['status'] ?? <sensible default> and $data['options'] ?? <default empty
array>) and pass them into the Attribute constructor (matching how
fromDocument() does), ensuring defaults mirror fromDocument/toDocument behavior.
src/Database/Hook/RelationshipHandler.php-272-283 (1)

272-283: ⚠️ Potential issue | 🟡 Minor

Loose equality comparison may cause unexpected behavior.

Using == for comparing $oldValue and $value (Line 272) can lead to unexpected type coercion, especially when comparing Documents, arrays, or mixed types. This could cause relationship updates to be incorrectly skipped.

🔧 Consider strict comparison or dedicated comparison logic
-            if ($oldValue == $value) {
+            if ($this->areRelationshipValuesEqual($oldValue, $value)) {

Alternatively, if loose comparison is intentional for this specific use case, add a comment explaining why.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Hook/RelationshipHandler.php` around lines 272 - 283, The loose
comparison using $oldValue == $value in RelationshipHandler (around the block
handling RelationType and RelationSide) can produce incorrect skips; change this
to strict comparison or explicit comparison logic: use === for scalar checks, if
both values are Document instances compare their IDs (e.g., $oldValue->getId()
=== $value->getId()), and for arrays use a deep equality check (or
serialize/sort then compare) before deciding to set or remove attributes via
$document->setAttribute/$document->removeAttribute; if loose comparison was
intentional, replace the == with a clear comment explaining why and what cases
rely on coercion.
src/Database/Hook/RelationshipHandler.php-1727-1732 (1)

1727-1732: ⚠️ Potential issue | 🟡 Minor

Parameter shadowing: $document reused in loop.

The foreach loop variable $document shadows the method parameter $document (from Line 1640), which could cause confusion or unintended behavior if the original document is needed after the loop.

🔧 Rename loop variable
-                foreach ($junctions as $document) {
-                    $this->db->skipRelationships(fn () => $this->db->deleteDocument(
-                        $junction,
-                        $document->getId()
-                    ));
+                foreach ($junctions as $junctionDoc) {
+                    $this->db->skipRelationships(fn () => $this->db->deleteDocument(
+                        $junction,
+                        $junctionDoc->getId()
+                    ));
                 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Hook/RelationshipHandler.php` around lines 1727 - 1732, The
foreach loop reuses the variable name $document which shadows the method
parameter $document; rename the loop variable (e.g., $junctionDocument or
$junctionItem) and update its usage inside the loop so the method parameter
remains available unchanged, keeping the existing calls to
$this->db->skipRelationships(fn () => $this->db->deleteDocument($junction,
<loop-var>->getId())); ensure only the loop variable name is changed (no other
logic altered).
src/Database/Traits/Indexes.php-359-366 (1)

359-366: ⚠️ Potential issue | 🟡 Minor

Fix rollback index TTL default: should be 0, not 1, to match TTL semantics elsewhere.

The rollback index uses getAttribute('ttl', 1) as default, but throughout the codebase TTL semantics treat 0 (or missing attribute) as "no TTL configured". In Documents.php:247, the code explicitly checks if ($ttlSeconds <= 0 || !$ttlAttr) to determine whether to skip TTL processing. Additionally, the Validator/Index.php:810 uses getAttribute('ttl', 0) as default. Using 1 as the default means if the original index had no TTL attribute, the rollback would incorrectly create an index with a 1-second TTL, causing documents to expire prematurely. Change the default to 0 to preserve the "no TTL" state when the attribute is missing.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Traits/Indexes.php` around lines 359 - 366, The rollback index
creation uses getAttribute('ttl', 1) which incorrectly sets a 1s TTL when the
attribute is missing; change the default to 0 so $rollbackIndex is created with
no TTL when absent by updating the call on $indexDeleted->getAttribute('ttl', 1)
to use 0 (the code building the Index instance with IndexType::from(...) and
attributes/lengths/orders should remain the same) to match Validator/Index.php
and Documents.php TTL semantics.
src/Database/Adapter/SQLite.php-43-70 (1)

43-70: ⚠️ Potential issue | 🟡 Minor

SQLite advertises it does not support Capability::Upserts but still implements upsert functionality.

Line 62 removes Capability::Upserts from the advertised capabilities, yet SQLite inherits upsertDocuments() from SQL and overrides executeUpsertBatch() to handle SQLite's ON CONFLICT syntax. This creates an inconsistency: the capability advertisement does not match the runtime behavior. While this doesn't currently cause dead code (no code gate upsert calls on supports(Capability::Upserts)), the mismatch violates the contract that removed capabilities should not be implemented. Either re-add Capability::Upserts to SQLite's capabilities, or remove the executeUpsertBatch() override if SQLite should not support upserts.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/SQLite.php` around lines 43 - 70, The SQLite adapter
currently removes Capability::Upserts in capabilities() while still implementing
upsertDocuments() and overriding executeUpsertBatch(), causing a contract
mismatch; fix by either (A) re-adding support: remove Capability::Upserts from
the $remove array in capabilities() so parent::capabilities() reports Upserts,
or (B) disable runtime support: delete or revert the
SQLite::executeUpsertBatch() override (and any SQLite-specific upsert helpers)
so the adapter no longer implements upsert behavior — choose one approach and
make the changes consistently (references: capabilities(), Capability::Upserts,
upsertDocuments(), executeUpsertBatch(), parent::capabilities()).
src/Database/Adapter/Mongo.php-1081-1081 (1)

1081-1081: ⚠️ Potential issue | 🟡 Minor

Remove the ->value accessor to fix the unreachable readiness loop.

At line 1081, $type is an IndexType enum case (assigned from $index->type at line 44), but the comparison $type === IndexType::Unique->value compares an enum case object to its backing scalar value. This will never match in PHP—enum cases and their backing values are distinct types. As a result, the readiness loop is unreachable and the code never waits for unique indexes to be fully built.

The fix is to compare the enum case directly:

Fix
-            if ($type === IndexType::Unique->value) {
+            if ($type === IndexType::Unique) {

This is consistent with all other enum comparisons in the same function (lines 1020, 1042, 1050, 1055, 1060), which use enum cases without the ->value accessor.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/Mongo.php` at line 1081, The readiness loop is
unreachable because it compares the enum case object $type to the backing scalar
of IndexType::Unique (using ->value); locate the comparison of $type and
IndexType::Unique in the readiness/wait loop and remove the ->value accessor so
the code compares the enum case directly (i.e., use IndexType::Unique) to match
how other enum checks in this function handle $type.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 3dfcd406-2dfe-4455-a2f0-baf7fa17d195

📥 Commits

Reviewing files that changed from the base of the PR and between b0a1faf and 4072192.

⛔ Files ignored due to path filters (1)
  • composer.lock is excluded by !**/*.lock
📒 Files selected for processing (128)
  • .github/workflows/tests.yml
  • Dockerfile
  • README.md
  • bin/tasks/relationships.php
  • composer.json
  • docker-compose.yml
  • src/Database/Adapter.php
  • src/Database/Adapter/Feature/Attributes.php
  • src/Database/Adapter/Feature/Collections.php
  • src/Database/Adapter/Feature/ConnectionId.php
  • src/Database/Adapter/Feature/Databases.php
  • src/Database/Adapter/Feature/Documents.php
  • src/Database/Adapter/Feature/Indexes.php
  • src/Database/Adapter/Feature/InternalCasting.php
  • src/Database/Adapter/Feature/Relationships.php
  • src/Database/Adapter/Feature/SchemaAttributes.php
  • src/Database/Adapter/Feature/Spatial.php
  • src/Database/Adapter/Feature/Timeouts.php
  • src/Database/Adapter/Feature/Transactions.php
  • src/Database/Adapter/Feature/UTCCasting.php
  • src/Database/Adapter/Feature/Upserts.php
  • src/Database/Adapter/MariaDB.php
  • src/Database/Adapter/Mongo.php
  • src/Database/Adapter/Mongo/RetryClient.php
  • src/Database/Adapter/MySQL.php
  • src/Database/Adapter/Pool.php
  • src/Database/Adapter/Postgres.php
  • src/Database/Adapter/SQL.php
  • src/Database/Adapter/SQLite.php
  • src/Database/Attribute.php
  • src/Database/Capability.php
  • src/Database/CursorDirection.php
  • src/Database/Database.php
  • src/Database/Document.php
  • src/Database/Helpers/Permission.php
  • src/Database/Hook/MongoPermissionFilter.php
  • src/Database/Hook/MongoTenantFilter.php
  • src/Database/Hook/PermissionFilter.php
  • src/Database/Hook/PermissionWrite.php
  • src/Database/Hook/Read.php
  • src/Database/Hook/Relationship.php
  • src/Database/Hook/RelationshipHandler.php
  • src/Database/Hook/TenantFilter.php
  • src/Database/Hook/TenantWrite.php
  • src/Database/Hook/Write.php
  • src/Database/Hook/WriteContext.php
  • src/Database/Index.php
  • src/Database/Mirror.php
  • src/Database/Operator.php
  • src/Database/OperatorType.php
  • src/Database/OrderDirection.php
  • src/Database/PermissionType.php
  • src/Database/Query.php
  • src/Database/RelationSide.php
  • src/Database/RelationType.php
  • src/Database/Relationship.php
  • src/Database/SetType.php
  • src/Database/Traits/Attributes.php
  • src/Database/Traits/Collections.php
  • src/Database/Traits/Databases.php
  • src/Database/Traits/Documents.php
  • src/Database/Traits/Indexes.php
  • src/Database/Traits/Relationships.php
  • src/Database/Traits/Transactions.php
  • src/Database/Validator/Attribute.php
  • src/Database/Validator/Datetime.php
  • src/Database/Validator/Index.php
  • src/Database/Validator/IndexedQueries.php
  • src/Database/Validator/Operator.php
  • src/Database/Validator/Permissions.php
  • src/Database/Validator/Queries.php
  • src/Database/Validator/Queries/Document.php
  • src/Database/Validator/Queries/Documents.php
  • src/Database/Validator/Query/Filter.php
  • src/Database/Validator/Query/Limit.php
  • src/Database/Validator/Query/Offset.php
  • src/Database/Validator/Sequence.php
  • src/Database/Validator/Spatial.php
  • src/Database/Validator/Structure.php
  • tests/e2e/Adapter/Base.php
  • tests/e2e/Adapter/MariaDBTest.php
  • tests/e2e/Adapter/MirrorTest.php
  • tests/e2e/Adapter/MongoDBTest.php
  • tests/e2e/Adapter/MySQLTest.php
  • tests/e2e/Adapter/PoolTest.php
  • tests/e2e/Adapter/PostgresTest.php
  • tests/e2e/Adapter/SQLiteTest.php
  • tests/e2e/Adapter/Schemaless/MongoDBTest.php
  • tests/e2e/Adapter/Scopes/AttributeTests.php
  • tests/e2e/Adapter/Scopes/CollectionTests.php
  • tests/e2e/Adapter/Scopes/CustomDocumentTypeTests.php
  • tests/e2e/Adapter/Scopes/DocumentTests.php
  • tests/e2e/Adapter/Scopes/GeneralTests.php
  • tests/e2e/Adapter/Scopes/IndexTests.php
  • tests/e2e/Adapter/Scopes/ObjectAttributeTests.php
  • tests/e2e/Adapter/Scopes/OperatorTests.php
  • tests/e2e/Adapter/Scopes/PermissionTests.php
  • tests/e2e/Adapter/Scopes/RelationshipTests.php
  • tests/e2e/Adapter/Scopes/Relationships/ManyToManyTests.php
  • tests/e2e/Adapter/Scopes/Relationships/ManyToOneTests.php
  • tests/e2e/Adapter/Scopes/Relationships/OneToManyTests.php
  • tests/e2e/Adapter/Scopes/Relationships/OneToOneTests.php
  • tests/e2e/Adapter/Scopes/SchemalessTests.php
  • tests/e2e/Adapter/Scopes/SpatialTests.php
  • tests/e2e/Adapter/Scopes/VectorTests.php
  • tests/e2e/Adapter/SharedTables/MariaDBTest.php
  • tests/e2e/Adapter/SharedTables/MongoDBTest.php
  • tests/e2e/Adapter/SharedTables/MySQLTest.php
  • tests/e2e/Adapter/SharedTables/PostgresTest.php
  • tests/e2e/Adapter/SharedTables/SQLiteTest.php
  • tests/unit/DocumentTest.php
  • tests/unit/OperatorTest.php
  • tests/unit/PermissionTest.php
  • tests/unit/QueryTest.php
  • tests/unit/Validator/AttributeTest.php
  • tests/unit/Validator/AuthorizationTest.php
  • tests/unit/Validator/DocumentQueriesTest.php
  • tests/unit/Validator/DocumentsQueriesTest.php
  • tests/unit/Validator/IndexTest.php
  • tests/unit/Validator/IndexedQueriesTest.php
  • tests/unit/Validator/OperatorTest.php
  • tests/unit/Validator/QueriesTest.php
  • tests/unit/Validator/Query/FilterTest.php
  • tests/unit/Validator/Query/OrderTest.php
  • tests/unit/Validator/Query/SelectTest.php
  • tests/unit/Validator/QueryTest.php
  • tests/unit/Validator/SpatialTest.php
  • tests/unit/Validator/StructureTest.php

Comment thread src/Database/Adapter/Mongo.php
Comment thread src/Database/Index.php
Comment thread src/Database/Traits/Documents.php
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
src/Database/Adapter/Pool.php (1)

80-88: Use the short class name for consistency.

Line 80 uses the fully qualified class name \Utopia\Database\Capability even though Capability is already imported at line 7. Use the short name for consistency with the rest of the file.

Suggested fix
-    public function supports(\Utopia\Database\Capability $feature): bool
+    public function supports(Capability $feature): bool
     {
         return $this->delegate(__FUNCTION__, \func_get_args());
     }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/Pool.php` around lines 80 - 88, Change the
fully-qualified parameter type in the supports method to the short imported
class name: replace "\Utopia\Database\Capability $feature" with "Capability
$feature" in the supports method signature so it matches the rest of the file
and uses the imported Capability symbol; ensure the method still calls
$this->delegate(__FUNCTION__, func_get_args()) unchanged.
src/Database/Adapter/SQLite.php (1)

247-248: Remove the duplicated _perms index creation.

These same two indexes are already created on Lines 247-248. Repeating them here just adds extra catalog probes and will become observable if index-create hooks/events are added later.

♻️ Proposed cleanup
-            $this->createIndex("{$id}_perms", new Index(key: '_index_1', type: IndexType::Unique, attributes: ['_document', '_type', '_permission']));
-            $this->createIndex("{$id}_perms", new Index(key: '_index_2', type: IndexType::Key, attributes: ['_permission', '_type']));

Also applies to: 265-266

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@src/Database/Adapter/SQLite.php` around lines 247 - 248, The code calls
$this->createIndex twice for the same "{$id}_perms" table with identical
Index(...) definitions (using Index(key: '_index_1', type: IndexType::Unique,
attributes: ['_document','_type','_permission']) and Index(key: '_index_2',
type: IndexType::Key, attributes: ['_permission','_type'])), causing duplicate
index creation; remove the duplicated createIndex calls (the repeated
"{$id}_perms" invocations) so each index is created only once, leaving a single
call to createIndex for each Index definition (references: createIndex, Index,
IndexType).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@src/Database/Adapter/SQLite.php`:
- Around line 45-65: The capabilities() method currently removes
Capability::Upserts from the $remove list which causes
supports(Capability::Upserts) to return false even though this class implements
SQLite-specific ON CONFLICT expressions and executeUpsertBatch(); fix by
removing Capability::Upserts from the $remove array (so Upserts is reported as
supported) and ensure any related logic in executeUpsertBatch() and
conflict-expression generation remains intact to be exercised when
supports(Capability::Upserts) is checked.

In `@src/Database/Mirror.php`:
- Around line 1074-1086: The setRelationshipHook method in Mirror currently
discards the caller's RelationshipHook by always installing a new
RelationshipHandler on the source/destination; instead, forward the provided
$hook (or null) directly to the underlying databases. In the
Mirror::setRelationshipHook(?RelationshipHook $hook) implementation replace the
calls that pass new RelationshipHandler($this->source)/new
RelationshipHandler($this->destination) with the original $hook (or null) so the
caller's custom RelationshipHook is preserved when calling
$this->source->setRelationshipHook(...) and
$this->destination->setRelationshipHook(...).
- Around line 321-333: The code calls Attribute::fromDocument($document) after
running filters in $this->writeFilters, but Filter::beforeCreateAttribute() can
return null to indicate the attribute should be skipped; update the blocks that
call beforeCreateAttribute (e.g., the loop using $this->writeFilters and the
code that assigns $filteredAttribute) to check if $document is null after the
loop and, if so, skip further processing (do not call Attribute::fromDocument) —
apply the same null-check/skipping logic to the other symmetric block that
processes attributes (the second occurrence mentioned around the attribute
handling).

---

Nitpick comments:
In `@src/Database/Adapter/Pool.php`:
- Around line 80-88: Change the fully-qualified parameter type in the supports
method to the short imported class name: replace "\Utopia\Database\Capability
$feature" with "Capability $feature" in the supports method signature so it
matches the rest of the file and uses the imported Capability symbol; ensure the
method still calls $this->delegate(__FUNCTION__, func_get_args()) unchanged.

In `@src/Database/Adapter/SQLite.php`:
- Around line 247-248: The code calls $this->createIndex twice for the same
"{$id}_perms" table with identical Index(...) definitions (using Index(key:
'_index_1', type: IndexType::Unique, attributes:
['_document','_type','_permission']) and Index(key: '_index_2', type:
IndexType::Key, attributes: ['_permission','_type'])), causing duplicate index
creation; remove the duplicated createIndex calls (the repeated "{$id}_perms"
invocations) so each index is created only once, leaving a single call to
createIndex for each Index definition (references: createIndex, Index,
IndexType).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 6220a957-b838-4386-98ab-e1a755c06239

📥 Commits

Reviewing files that changed from the base of the PR and between 4072192 and 2546230.

📒 Files selected for processing (7)
  • .gitignore
  • src/Database/Adapter/Pool.php
  • src/Database/Adapter/SQLite.php
  • src/Database/Database.php
  • src/Database/Mirror.php
  • tests/e2e/Adapter/Scopes/Relationships/ManyToManyTests.php
  • tests/e2e/Adapter/Scopes/SchemalessTests.php
✅ Files skipped from review due to trivial changes (1)
  • .gitignore

Comment thread src/Database/Adapter/SQLite.php
Comment thread src/Database/Mirror.php Outdated
Comment thread src/Database/Mirror.php Outdated
@abnegate abnegate closed this Mar 12, 2026
@abnegate abnegate reopened this Mar 12, 2026
abnegate added 22 commits March 14, 2026 22:50
abnegate and others added 4 commits May 1, 2026 10:03
…mo merged attributes

The Structure validator only depends on the collection and stable adapter
properties — allocating one per document was waste. Hoist the
construction outside the loop and reuse the same instance for every
document in the batch.

Memoize the merged attribute list (internal + collection) on the
validator. The merge inputs are immutable for the validator's lifetime
($attributes is a constant table, $collection is readonly), so caching
the result is safe and skips an `array_merge` per document.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ed order, fold search detection into find partition

The find pipeline previously walked queries twice: once for the partition
(vector / adapter-filter / other) and again via `extractSearchQueries` to
project a `_relevance` SELECT and ORDER BY. With FULLTEXT MATCH that
means the relevance MATCH is compiled and evaluated a second time.

Pick up search queries in the existing single-pass partition loop and
skip the relevance projection entirely when the caller already supplied
an explicit order — the Documents trait auto-appends `$sequence` as a
tiebreaker, so anything richer than a lone `$sequence` is caller-driven
and would be silently overridden by relevance order otherwise.

`extractSearchQueries` is kept as a protected method for adapter overrides
but is no longer called from `find`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Bump baseline count for the array_unique-on-mixed pattern in
  Document::fromRow (3 occurrences now).
- Tighten Structure::$mergedAttributes type to array<array<string, mixed>>
  to match what array_merge of internal+collection attributes actually
  produces.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Conflicts:
#	.github/workflows/tests.yml
#	composer.lock
#	tests/e2e/Adapter/Scopes/DocumentTests.php
@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

CI fix — composer.lock out of sync

What failed (run 25198182284)

Docker build step composer install failed during lock-file verification:

- Required package "utopia-php/query" is not present in the lock file.
- Required package "utopia-php/async" is not present in the lock file.
- Required (in require-dev) package "phpunit/phpunit" is in the lock file
  as "9.6.34" but that does not satisfy your constraint "^12.0".
- Required (in require-dev) package "brianium/paratest" is not present in the lock file.
- Required (in require-dev) package "phpstan/phpstan" is in the lock file
  as "1.12.33" but that does not satisfy your constraint "^2.0".

Root cause

The merge commit 04625671 (Merge origin/main into feat-query-lib) had a conflict in composer.lock and resolved it by taking main's version. That dropped the dependencies this branch had added/upgraded (utopia-php/query, utopia-php/async, paratest, phpunit ^12, phpstan ^2), so the lock no longer matched composer.json.

Fix

Regenerated composer.lock with composer update --ignore-platform-reqs --no-scripts --no-plugins. The lock now contains:

  • utopia-php/query 0.2.0
  • utopia-php/async dev-main
  • brianium/paratest v7.20.0
  • phpunit/phpunit 12.5.23
  • phpstan/phpstan 2.1.54

Verified composer install --dry-run passes the lock-file verification step locally.

The merge from main resolved composer.lock by taking main's version,
which dropped the new dependencies introduced on feat-query-lib
(utopia-php/query, utopia-php/async, brianium/paratest, phpunit ^12,
phpstan ^2). Docker build failed in `composer install` with
"Required package ... is not present in the lock file".

Regenerated via `composer update --ignore-platform-reqs` so the lock
matches composer.json again.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

Claude pushed fixes from: healing

0462567...92a30ed

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

Code Review

Multi-dimension review of the most recent push (since the last main merge), which adds the Redis adapter (~4242 lines) and a series of perf/fix follow-ups in Database, Document, Mirror, SQL, SQLite, Structure, and Traits/Documents.

Per-dimension summary

Security (Redis adapter): Several DoS-shaped patterns: loadCollectionDocuments/enforceUniqueIndexes MGET the entire collection on every read/write (memory blow-up & write amplification); applyPermissionFilter SUNIONs every role key (large role sets stall Redis); deleteByPattern uses blocking DEL + per-collection-drop SCAN (consider UNLINK and tighter scoping); TYPE_REGEX runs @preg_match on user PCRE with no length cap or backtrack-limit override (ReDoS); matchesDocument recurses unbounded over nested OR/AND trees (stack-overflow potential). All scale with deployment size, none block normal behaviour.

Performance (Redis adapter): Same hot paths are O(N) per call: nested multi(PIPELINE) inside tx() callbacks; find/count/sum always hydrate the whole collection before paginating; enforceUniqueIndexes is O(N) per write (could be O(1) via per-signature SET); getDocument makes 2 round trips because surfaceRelationshipAttributes re-reads meta.attrs; createDocuments is a naive loop, becoming O(N²) total. orderDocuments calls resolveDocumentAttribute twice per usort comparison (Schwartzian transform would help).

Correctness (highlights):

  • CRITICAL Redis::updateDocuments (~line 2231) builds docKey($col, $doc->getId()) without per-document tenant. upsertDocuments (line 2326) was correctly fixed to thread $document->getTenant() so a batch that mixes tenants under shared tables doesn't misroute to the adapter-bound bucket — the same fix is needed here.
  • WARNING Redis::deleteByPattern (~line 1862) starts cursor null and exits on false batch, silently leaving stale keys after a transient SCAN failure.
  • WARNING Redis::createDocument sequence assignment (~line 2096) does get → compare → set without WATCH/MULTI; concurrent calls race and the smaller value silently wins, breaking the $sequence-tiebreaker invariant.
  • WARNING Redis::enforceUniqueIndexes (~line 1684) compares $rowTenant raw against the (int|string) adapter tenant; a "42" vs 42 mismatch silently skips the colliding row and lets a duplicate through.
  • WARNING Document::fromRow is now used by SQL::rawQuery (line 1542) — rawQuery accepts arbitrary user SQL whose rows may legitimately contain nested arrays; the fast-path skip of nested-Document detection breaks that contract.
  • WARNING Structure::$mergedAttributes memo trusts the held collection isn't mutated between isValid() calls. (Comment tightened in this push to call out the invariant; behavioural fix deferred.)

Readability: Internal // === @architect:T20/T30/T40/T50/T56 owns: ... === markers leaked into the Redis adapter (10 occurrences). The $callerSuppliedOrder = ! (...) predicate in SQL.php requires double-negation to read. The Random short-circuits via shuffle to preserve usort transitivity comment in Redis::orderDocuments is stale — the random branch never calls usort. (All three fixed in this push.)

Maintainability: Redis::matchesDocument (~200 lines, 13-arm switch) duplicates the in-memory query engine that Memory.php already implements — extract into a shared InMemoryQueryEngine trait. Redis::createAttribute/updateAttribute/createIndex use the legacy positional signatures while every other adapter migrated to the typed Attribute/Index value objects. Redis does not override capabilities()Database::convertQueries consults Capability::DefinedAttributes/Objects and may take the wrong branch.

Testing: The new Document::fromRow factory has no direct unit tests (throw paths, dedupe semantics). The Mirror::getCollection cache-staleness fix has no regression test. RedisTest::testUpdateAttributeStructure/testUpdateAttributeSize skip rather than asserting the documented Redis no-op resize behaviour. SchemalessValidationTest asserts on a stubbed Adapter rather than a real adapter.

Fixed in this run

  • Stripped 10 @architect:T* task-ownership markers from src/Database/Adapter/Redis.php.
  • Collapsed redundant array_key_exists('$permissions', ...) branches in Document::fromRow.
  • Renamed SQL::generateOrderBy predicate to $shouldAutoOrderByRelevance (no double-negation).
  • Corrected stale usort transitivity rationale in Redis::orderDocuments docblock.
  • Tightened the misleading readonly/"never changes" claim on Structure::$mergedAttributes.

Deferred to follow-up PRs: every architectural / contract / testing finding above. They are too large to land safely as a drive-by, and several need behavioural-intent confirmation from the adapter owner.

- Strip leaked internal `@architect:T*` task-ownership markers from the
  Redis adapter; they were planning artifacts and add no value to readers.
- Document::fromRow: collapse two `array_key_exists('$permissions', ...)`
  branches into one — the second `is_array` check was redundant given the
  first arm throws on non-array.
- SQL::generateOrderBy: rename `$callerSuppliedOrder` to its positive
  counterpart `$shouldAutoOrderByRelevance` so the gate predicate reads
  without double-negation.
- Redis::orderDocuments: docblock said "preserve usort transitivity" but
  the random branch never calls usort — explain the actual reason
  (non-deterministic comparators are undefined for usort).
- Structure::$mergedAttributes: weaken the "Stable for the lifetime of
  this validator" comment — `readonly` blocks property reassignment but
  not inner Document mutation, so callers that mutate the held
  collection between `isValid()` calls would see a stale memo.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

Claude pushed fixes from: improvement

92a30ed...9946e3e

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

No critical/warning findings.

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Reviewed the recent perf/fix series (commits since ba8d911c). Six findings fixed in this push; three flagged-then-validated as false positives; the rest are policy choices that don't need code changes.

Fixed

[CRITICAL] Authorization parity in inlined skip togglesrc/Database/Traits/Documents.php (find post-processing). The inlined skip block in find only re-enabled auth when $previousStatus was true, drifting from Authorization::skip's unconditional $this->status = $initialStatus. A nested setStatus(true) inside the protected block could leak past the scope. Now restores $previousStatus directly via setStatus.

[CRITICAL] Relationship batching exceeds maxQueryValuessrc/Database/Hook/Relationships.php. Commit c2bda30d replaced N single-row updates with a single Query::equal('$id', $largeArray), which the DocumentsValidator rejects above 5000 values. A one-to-one or many-to-many diff with thousands of removed children would throw QueryException. All three batched paths (one-to-one removed, one-to-one existing, many-to-many junction removal) are now chunked by Database::RELATION_QUERY_CHUNK_SIZE, with accumulated $existingIds/$junctionIds across chunks.

[WARNING] MariaDB timeout staleness across pool checkoutssrc/Database/Adapter/MariaDB.php. Same shape as the MySQL 51ebe32e fix. Direct $stmt->execute() paths in SQL.php bypass $this->execute()'s lazy timeout push, so the cached $appliedMaxStatementTime could lie about the underlying PDO's session state if the pool replaces the connection. Now applies eagerly in setTimeout/clearTimeout while keeping the lazy execute() check cheap.

[WARNING] DocumentsValidator cache leak in setMaxQueryValuessrc/Database/Database.php. Cache key encodes maxQueryValues, but the setter didn't prune entries built under the old limit. Long-lived workers that retune accumulate stale validators with the old ceiling. Now clears the cache when the value actually changes.

[WARNING] Memory decodeObjectValue parse-misssrc/Database/Adapter/Memory.php. Returned the original $value when JSON decode failed, so non-array haystacks could reach jsonContains and surface a literal-string match via the final === arm. Now returns null, parity with decodeArrayValue.

[WARNING] MemoryTest::freshDatabase per-process isolation regressiontests/e2e/Adapter/MemoryTest.php:98. Used the literal 'utopiaTests' instead of $this->testDatabase (the per-process token from Base::setUp'utopiaTests_'.getTestToken()). Two paratest workers running Memory regression tests concurrently could collide on the same database name. Switched to $this->testDatabase.

[WARNING] Spatial cache key comment mismatchsrc/Database/Adapter/SQL.php. Doc comment claimed database/namespace/tenant scoping; the key is database/namespace/collectionId. Tenant exclusion is correct on shared tables (collection schema is shared), so clarified the comment to match.

Validated as false positives (no change)

  • Document::__clone is shallowDocument::__clone() (line 590) deep-clones nested Document and array entries, so the metadata cache's clone $cached is safe.
  • Memory::addPermissionEntry strips quotes — redundant but harmless. Document::getPermissionsByType() already strips " upstream before reaching the adapter, so the value passed to addPermissionEntry never has quotes.
  • getCollection cache key omits tenantPerDocument — the post-fetch tenant filter only references getSharedTables() and getTenant(). tenantPerDocument only affects writes; the cache is correctly scoped.

Not addressed (policy / lower priority)

  • Memory Equal/NotEqual NULL asymmetry comment.
  • Memory looseEquals widening string-vs-int via is_numeric (intentional MariaDB-shape parity).
  • Search relevance ordering when orderAttributes is set (changelog item, not a bug).
  • Adapter::filter() cache full-clear policy under hostile input (nuisance not a security issue).
  • Style items: extractSearchQueries dead code, OFFSET 0 trim, FTS5 SHA-1 truncation width.

Local commit 978716a8 produced by this run; the orchestrator will push it.

…ship chunking, validator cache

- MariaDB: apply session timeout eagerly in setTimeout/clearTimeout
  so direct $stmt->execute() paths inherit it, mirroring the MySQL
  fix; prevents stale timeouts leaking across pool checkouts.
- Relationships hook: chunk batched updateDocuments/deleteDocuments
  calls by RELATION_QUERY_CHUNK_SIZE so a one-to-many or many-to-many
  diff with thousands of removed peers stays within the validator's
  maxQueryValues ceiling.
- Database::setMaxQueryValues: clear the DocumentsValidator cache when
  the limit changes; the cache key encodes the old value.
- Traits/Documents (find): always restore previous Authorization status
  in the inlined skip toggle so a nested setStatus inside the try block
  cannot leak past the scope.
- Memory adapter: decodeObjectValue returns null on JSON parse miss
  (parity with decodeArrayValue); prevents scalar haystacks reaching
  jsonContains.
- MemoryTest::freshDatabase: use $this->testDatabase to honor the
  per-process isolation token.
- SQL adapter: clarify spatialCacheKey() comment to match code (tenant
  intentionally excluded; schema is shared across tenants).
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

Claude pushed fixes from: improvement

9946e3e...4f03607

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

CI fix: duplicate test methods from main merge

Failing run: https://github.com/utopia-php/database/actions/runs/25198581761

Root cause: the merge of origin/main into feat-query-lib (commit 04625671) resolved tests/e2e/Adapter/Scopes/DocumentTests.php by accepting both the old-API copy from main and the already-converted new-API copy on this branch for 11 tests. PHP refused to load the trait:

PHP Fatal error: Cannot redeclare Tests\E2E\Adapter\Scopes\DocumentTests::testPreserveSequenceUpsert()
in /usr/src/code/tests/e2e/Adapter/Scopes/DocumentTests.php on line 5225

This took out every adapter test job (Postgres, MariaDB, MySQL, Mongo, SQLite, Redis, all SharedTables variants) because paratest could not even start.

Fix: removed the two contiguous old-API blocks the merge left behind.

  • Lines 1160–1552 (9 duplicates): testPreserveSequenceUpsert, testRespectNulls, testCreateDocumentDefaults, testIncreaseDecrease, testIncreaseLimitMax, testDecreaseLimitMin, testIncreaseTextAttribute, testIncreaseArrayAttribute, testIncreaseDecreasePreserveDates.
  • Lines 3166–3529 (2 duplicates + 1 new test from main): testInvalidCreatedAndUpdatedAtThrowStructureException, testSingleDocumentDateOperations, testBulkDocumentDateOperations.

All 12 methods referenced Database::VAR_* constants and getSupportForAttributes() / getSupportForUpserts() — symbols that no longer exist on this branch (replaced by Attribute(...) + Capability / Feature\*), so they could not have run anyway. The new-API versions of the 11 duplicates are kept; testInvalidCreatedAndUpdatedAtThrowStructureException has no new-API counterpart on this branch and was deleted (re-port if you want the coverage).

Verification:

  • php -l clean
  • grep | uniq -c shows zero duplicate method names
  • grep -E "Database::VAR_|getSupportForAttributes|getSupportForUpserts" returns no hits in tests/e2e/Adapter/Scopes/DocumentTests.php
  • File function count back to 126 (matches pre-merge feat-query-lib)
  • Net change: pure 757-line deletion (git diff --minimal --stat)

The merge of origin/main into feat-query-lib (commit 0462567)
duplicated 11 test methods in tests/e2e/Adapter/Scopes/DocumentTests.php
by keeping both the old-API versions from main and the already-converted
new-API versions on this branch. PHP failed to load the trait with
"Cannot redeclare ...::testPreserveSequenceUpsert()", which broke every
adapter test job (paratest could not even start).

Two contiguous blocks of old-API content were left in the file:
- Lines 1160-1552: testPreserveSequenceUpsert, testRespectNulls,
  testCreateDocumentDefaults, testIncreaseDecrease, testIncreaseLimitMax,
  testDecreaseLimitMin, testIncreaseTextAttribute, testIncreaseArrayAttribute,
  testIncreaseDecreasePreserveDates.
- Lines 3166-3529: testInvalidCreatedAndUpdatedAtThrowStructureException,
  testSingleDocumentDateOperations, testBulkDocumentDateOperations.

Both blocks reference Database::VAR_* constants and getSupportForAttributes
/getSupportForUpserts methods that were removed on this branch in favour of
Attribute(...) constructor + Capability/Feature interfaces, so they could
not have run anyway.

The new-API versions of the duplicated tests remain (lines now 4468-4819
and 7318-7490 in the cleaned file). testInvalidCreatedAndUpdatedAt... has
no new-API counterpart on this branch and was a fresh test from main; it
is dropped here so the parser is unblocked — the maintainer can re-port it
in a follow-up if they want the coverage.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

Claude pushed fixes from: healing

4f03607...4f16422

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

CI healing — run 25198770515

Root cause: PHP fatal at parse time on every adapter job:

PHP Fatal error: Cannot redeclare Tests\E2E\Adapter\Scopes\DocumentTests::testPreserveSequenceUpsert()
in tests/e2e/Adapter/Scopes/DocumentTests.php on line 5225

The merge of origin/main into feat-query-lib (commit 04625671) had a conflict in tests/e2e/Adapter/Scopes/DocumentTests.php. The resolution kept both sides — main's pre-refactor copies and feat-query-lib's post-refactor copies — of 11 tests. PHP only reported the first redeclaration; the file actually contained 10 more.

The duplicated method names were:
testPreserveSequenceUpsert, testRespectNulls, testCreateDocumentDefaults, testIncreaseDecrease, testIncreaseLimitMax, testDecreaseLimitMin, testIncreaseTextAttribute, testIncreaseArrayAttribute, testIncreaseDecreasePreserveDates, testSingleDocumentDateOperations, testBulkDocumentDateOperations.

The two copies were not equivalent: the older copies still used the pre-refactor API (Database::VAR_STRING / Database::VAR_INTEGER / etc., getSupportForUpserts(), getSupportForAttributes(), Document-returning @depends chains) which no longer exists on this branch, while the newer copies use Capability::DefinedAttributes, Feature\Upserts, and new Attribute(key: …, type: ColumnType::…) consistent with the rest of feat-query-lib.

Fix:

  • Removed the 11 old-API duplicate methods.
  • Ported the lone remaining old-API test testInvalidCreatedAndUpdatedAtThrowStructureException (line 3028, not a duplicate, came from main only) to the new API — it was the only other caller of Database::VAR_STRING / getSupportForAttributes() in the file and would have failed at runtime once the parse error was cleared.

Verification:

  • php -l on the file: No syntax errors detected.
  • No remaining duplicate method declarations in the file.
  • No remaining references to Database::VAR_*, getSupportForUpserts, or getSupportForAttributes in this file.
  • Couldn't run the docker-based test step locally (this runner has PHP 8.3 and the project requires 8.4 + ext-mongodb 2.x), so the actual MySQL/SharedTables job needs to re-run on CI.

Commit: (fix): CI — drop duplicate old-API tests left by main merge in DocumentTests.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

Claude could not apply patches from: healing (merge conflicts with current branch tip). These tasks need manual attention.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented May 1, 2026

CI fix — duplicate test method declarations

What failed (run 25198270257)

Every adapter test job (MariaDB, MySQL, Postgres, SQLite, Mongo, Memory, Redis, Pool, Mirror, all SharedTables/Schemaless variants) failed at startup with:

PHP Fatal error: Cannot redeclare Tests\E2E\Adapter\Scopes\DocumentTests::testPreserveSequenceUpsert()
in /usr/src/code/tests/e2e/Adapter/Scopes/DocumentTests.php on line 5225

PHP couldn't load the trait, so 0 tests ran across all adapter jobs.

Root cause

The merge commit 04625671 (origin/mainfeat-query-lib) brought in copies of 11 test methods that already existed on feat-query-lib under the new query-lib API. Conflict resolution left both copies in the file:

  • testPreserveSequenceUpsert, testRespectNulls, testCreateDocumentDefaults, testIncreaseDecrease, testIncreaseLimitMax, testDecreaseLimitMin, testIncreaseTextAttribute, testIncreaseArrayAttribute, testIncreaseDecreasePreserveDates — old API copies clustered at lines 1160–1551
  • testSingleDocumentDateOperations, testBulkDocumentDateOperations — old API copies at lines 3224–3528

PHP's fatal hit on the first duplicate it encountered; the other 10 would have failed in turn.

The merge also brought in one new test (testInvalidCreatedAndUpdatedAtThrowStructureException) that called getSupportForAttributes() and Database::VAR_STRING — symbols that don't exist on the new-API adapters.

Fix

  1. Deleted the two clusters of old-API duplicate methods (the feat-query-lib versions using Feature\Upserts, Capability::DefinedAttributes, ColumnType::*, new Attribute(key: …) are kept).
  2. Translated the single new test to the new API: getSupportForAttributes()supports(Capability::DefinedAttributes), and createAttribute(...string, Database::VAR_STRING...)createAttribute(..., new Attribute(key: ..., type: ColumnType::String, ...)).

Verification

Re-ran the failed step locally:

docker compose exec tests vendor/bin/paratest --configuration phpunit.xml --functional --processes 4 \
  /usr/src/code/tests/e2e/Adapter/SharedTables/MariaDBTest.php

Result: Tests: 646, Assertions: 12657, Skipped: 9 — all green (the previous "1 error" on testInvalidCreatedAndUpdatedAtThrowStructureException is also gone).

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Review scoped to the four commits since the merge from main (04625671..HEAD): 12 source files, ~3.9K diff lines. Older commits in the 343-file branch were left to prior review cycles.

Per-dimension findings

Security & Correctness — One verified WARNING: Hook/Relationships chunks at the constant RELATION_QUERY_CHUNK_SIZE (5000) which matches the default maxQueryValues, but a caller that lowers the validator cap would still see chunked relationship updates throw QueryException. The earlier reported Memory::decodeObjectValue 'pass-through scalar' regression is a false positive — every matchesObject() branch already short-circuits on $haystack === null, so the tightened ?array return is semantically equivalent (and now cleaner).

Performance & Maintainability — The chunked find/update pattern in Relationships adds N round-trips where one would have sufficed pre-chunking, but that is the unavoidable cost of honoring maxQueryValues. At the default cap (5000) most relationship updates still fit in a single chunk. No other hot-path regressions found in the new code.

Readability & TestingMemory::decodeObjectValue lost its docblock when the body was tightened, leaving the new ?array contract less obvious than its sibling decodeArrayValue. Test coverage for the chunking boundary at >maxQueryValues is missing, but adding it requires fixtures of 5000+ child documents and exceeds this pass's scope.

Fixes applied

  • Hook/Relationships: extracted a relationQueryChunkSize() helper that returns max(1, min(RELATION_QUERY_CHUNK_SIZE, getMaxQueryValues())) and replaced all eleven array_chunk(..., Database::RELATION_QUERY_CHUNK_SIZE) sites with it. The constant still acts as a memory bound; the configured cap now acts as the validator ceiling.
  • Memory::decodeObjectValue: restored the docblock to mirror decodeArrayValue so the null-vs-array contract reads as intentional.

One local commit (fix(relationships): respect lowered maxQueryValues in chunked fan-out).

Comment thread src/Database/Hook/Relationships.php Outdated
// Chunk to honor the validator's maxQueryValues cap; without
// this a relationship update with thousands of removed
// children would throw QueryException.
foreach (\array_chunk($removedDocuments, Database::RELATION_QUERY_CHUNK_SIZE) as $chunk) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[WARNING — fixed] Chunking at Database::RELATION_QUERY_CHUNK_SIZE (5000) ignores a lowered setMaxQueryValues(). A caller that drops the validator cap to e.g. 1000 would still see this chunk hit Filter's maxValuesCount ceiling and throw QueryException. Replaced with $this->relationQueryChunkSize() which takes min(constant, getMaxQueryValues()) so the chunk respects whichever bound is tighter. Same fix applied to all eleven chunked-call sites in this file.

}

protected function decodeObjectValue(mixed $value): mixed
protected function decodeObjectValue(mixed $value): ?array
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[SUGGESTION — fixed] The signature tightening from mixed to ?array is correct (every matchesObject() branch already short-circuits on $haystack === null), but the docblock disappeared in the same commit, leaving the new contract implicit. Restored a docblock that mirrors decodeArrayValue and explains why callers can rely on a single null-guard.

Hardcoding the relationship fan-out chunk at RELATION_QUERY_CHUNK_SIZE
(5000) matched the default validator cap, but a caller that lowered
setMaxQueryValues() below 5000 would still see chunked find/update
calls throw QueryException — the validator rejects any IN-list whose
count exceeds the configured cap.

Introduce a private relationQueryChunkSize() helper that takes
min(RELATION_QUERY_CHUNK_SIZE, getMaxQueryValues()) and use it at every
chunked array_chunk() site in Hook/Relationships. The constant still
acts as a memory bound; the configured cap acts as the validator
ceiling. Floor at 1 so a misconfigured cap of 0 cannot crash array_chunk.

Also restore the docblock on Memory::decodeObjectValue so the
?array contract mirrors decodeArrayValue and matchesObject's
"null === no match" guard reads as intentional rather than incidental.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

Claude pushed fixes from: improvement

4f16422...1b783f7

Copy link
Copy Markdown
Contributor

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Ran one improvement cycle (/skills:improve) — six parallel dimension reviewers (security, performance, correctness, readability, maintainability, testing) over the most recent push. Two fixes committed locally; the orchestrator will push them.

Per-dimension summary

  • Security — 0 critical, 0 warnings, 3 suggestions. setTimeout float interpolation is not exploitable (typed int input, locale-independent float-to-string in PHP 8.0+); chunked relationship updates preserve getAuthorization()->skip() wrapping per chunk; the inlined auth-skip toggle in Traits/Documents.php is strictly more correct than the prior if ($previousStatus) enable() shape. Nothing actioned.
  • Performance — 0 critical, 2 warnings (both fixed). MariaDB::setTimeout/clearTimeout now route through a cache-guarded applyMaxStatementTime() helper so Pool::delegate() per-checkout calls no longer issue a redundant SET round-trip per request.
  • Correctness — 0 critical, 1 warning (fixed). Float interpolation 'SET max_statement_time = ' . $seconds could serialize as scientific notation for extreme values; now via sprintf('%.6F', ...). The "mirrors the MySQL adapter" comment was overclaiming parity (the bodies legitimately diverge); tightened.
  • Readability — 0 critical, 1 warning, 13 suggestions. Most are accurate-as-is or minor (docblock phrasing, missing one-line comment on clearTimeout). The Hook/Relationships.php chunking-pattern duplication is borderline; left in place since signatures genuinely differ.
  • Maintainability — 1 critical (fixed), 4 warnings (1 fixed). The hard-coded RELATION_QUERY_CHUNK_SIZE = 5000 was used at all 10 chunking sites in Hook/Relationships.php while the validator cap (maxQueryValues) is mutable via setMaxQueryValues() and may be lower — the chunking did not actually honor the limit it claimed to honor when a consumer lowered the cap. Routed through a new relationChunkSize() helper that returns min(RELATION_QUERY_CHUNK_SIZE, getMaxQueryValues()), floored at 1.
  • Testing — 1 critical, 7 warnings (test gaps; not added in this cycle). The newly-added eager setTimeout/clearTimeout paths and the relationship chunking-with-lowered-maxQueryValues regression scenario both lack tests. Memory decodeObjectValue behaviour change (return type narrowed to ?array) lacks a unit test. Document fromRow permission validation lacks a unit test. These are flagged for follow-up; adding them now would expand scope beyond this review push.

Final state

  • Total fixed: 1 critical + 3 warnings = 4 findings actioned
  • Local commits: 2
  • Tests: unit suite passes with the same pre-existing 2 errors / 1 failure (Swoole class not present, PHP-8.3-vs-8.4 datetime parser) on this and the prior commit; my changes do not introduce regressions.
  • Lint: Pint passes. PHPStan: 39 → 36 errors (eliminated 3 array_chunk argument.type warnings; remaining 36 are pre-existing Utopia\Query\Schema\Table class.notFound reports unrelated to this change).

$newIds = \array_map(function ($item) {
if (\is_string($item)) {
return $item;
} elseif ($item instanceof Document) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[CRITICAL] [maintainability] This site (and the four other newly-added chunking sites at 511, 525, 639, 652, plus the six pre-existing sites at 1445, 1543, 1647, 1735, 1798) hard-coded Database::RELATION_QUERY_CHUNK_SIZE = 5000, but the validator's maxValuesCount cap is set per-Database via setMaxQueryValues() and may be lower (e.g. a tenant configured for stricter validation). With the cap lowered, the chunked Query::equal('$id', $chunk) call still emits 5000 ids and the validator throws QueryException — the very failure mode this push was meant to prevent.

Fixed locally by routing all 10 chunking sites through a new relationChunkSize() helper that returns min(RELATION_QUERY_CHUNK_SIZE, getMaxQueryValues()), floored at 1 (setMaxQueryValues is unvalidated and array_chunk rejects 0).

// application leaked stale timeouts across pool checkouts under paratest;
// mirrors the MySQL adapter's eager-apply fix for the same shape of bug.
$seconds = $milliseconds / 1000.0;
$this->getPDO()->exec('SET max_statement_time = ' . $seconds);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[WARNING] [performance] This eager SET max_statement_time = ... runs unconditionally, even when the value is unchanged. The lazy execute() path on this same adapter already short-circuits redundant SETs by comparing against $appliedMaxStatementTime, but the new eager path bypasses that cache. Pool::delegate() calls setTimeout() on every pool checkout, so a hot read path now pays an unconditional SET round-trip per request even when the value didn't change.

[WARNING] [correctness] ' . $seconds for very large $milliseconds (above ~9e15) produces scientific notation via PHP's default float-to-string. MariaDB will likely accept it but it's brittle; sprintf('%.6F', $seconds) pins fixed-point format.

Fixed locally by extracting applyMaxStatementTime(float $seconds) that does the cache check + sprintf('%.6F', ...) and routing setTimeout, clearTimeout, and execute() through it. Also tightened the "mirrors the MySQL adapter" comment — the bodies legitimately diverge because each engine ships a different session variable, so the shared shape is the bug class, not the implementation.

{
return 'RAND()';
$this->getPDO()->exec('SET max_statement_time = 0');
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[WARNING] [performance] Same cache-guard issue as setTimeout above — every clearTimeout call writes a SET max_statement_time = 0 round-trip, even when the timeout was already cleared. Fixed by routing through the new applyMaxStatementTime(0.0) helper which short-circuits when $appliedMaxStatementTime === 0.0.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 1, 2026

Claude could not apply patches from: improvement (merge conflicts with current branch tip). These tasks need manual attention.

abnegate and others added 5 commits May 1, 2026 14:18
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces unsafe `(string) $mixed` and `(int) $mixed` casts with proper type
narrowing helpers (`payloadString`, `payloadStringOr`, `stringOrEmpty`,
`numericOr`, `intOr`, `recordIdentifier`) across the journal rollback,
attribute metadata access, unique-index pre-flight scan, query matching,
and operator math paths. Also relaxes helper signatures from `array<int, T>`
to `array<T>` to match the parent abstract's non-list contract.

Drops the inline `@var` overrides and `(string)` cast workarounds in
`rollbackJournal()` in favour of the typed payload accessors.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
In paratest's `--functional` mode `tearDownAfterClass` runs between every
test method, not just at suite end. The previous override scrubbed the
adapter keyspace and nulled `self::$database`, forcing the next test to
rebuild the database under a fresh `uniqid()` namespace while inherited
fixture statics (`$moviesFixtureInit`, `$documentsFixtureInit`, etc.)
stayed set — so cached fixture metadata pointed at collections that no
longer existed. Result: 84 errors and 8 failures across `find`, unique
index, and permission tests. Also align `setDatabase()` with the per-PID
`$this->testDatabase` so `testCreateExistsDelete` matches the actual
database name. CI Redis is ephemeral; leaking keys to process exit is
safe.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants