Skip to content

[Editorial P2.4]: AYONData marker codec - #70

Draft
jakubjezek001 wants to merge 3 commits into
feature/15-otio-pathsfrom
feature/16-otio-markers
Draft

jakubjezek001 wants to merge 3 commits into
feature/15-otio-pathsfrom
feature/16-otio-markers

Conversation

@jakubjezek001

@jakubjezek001 jakubjezek001 commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Changelog Description

New shared module ayon_editorial.otio.markers: reads and writes the AYONData marker and finds a clip by its clip_index, replacing the three get_marker_from_clip_index copies.

Additional info

  • Reads both shapes: json_metadata (Hiero) and flat metadata (Flame, Resolve). The marker name AYONData and json_metadata key are unchanged.
  • Resolve's marker unwrap stays in ayon_resolve and is passed in as unwrap.
  • Flame's name filter is the marker_name argument.
  • Deliberate difference: a marker with invalid json_metadata is skipped (and logged); the Hiero copy raised JSONDecodeError and aborted the search.
  • Also create_ayon_marker, parse_marker_text (Flame/Resolve JSON-or-note detection), decode_marker_data.

Testing notes:

  1. ./tools/manage.sh run-tests (tests/test_otio_markers.py)

Dependency

Close #16

Checklist

  • ruff check and pytest pass (Python 3.9 with OTIO 0.15, 0.16, 0.17 and 0.18)
  • python create_package.py --skip-zip builds
  • No frozen contract changed
  • Settings key renamed only with a conversion function (none renamed)
  • Layer rules kept (tests/test_dcc_agnostic.py)

@jakubjezek001
jakubjezek001 added this pull request to stack #72 October 29, 2026 15:32
@jakubjezek001 jakubjezek001 changed the title Add the AYONData marker codec [Editorial P2.4]: AYONData marker codec Oct 29, 2026
@jakubjezek001 jakubjezek001 added the type: feature Adding something new and exciting to the product label Oct 29, 2026
@jakubjezek001
jakubjezek001 marked this pull request as draft October 30, 2026 13:40
@jakubjezek001

Copy link
Copy Markdown
Member Author

Cross-review finding 70-1: parse_marker_text changes the Flame marker comment handling for brace text

client/ayon_editorial/otio/markers.py:96-103

The PR says parse_marker_text is the "Flame/Resolve JSON-or-note detection". The two hosts do not detect JSON the same way, and the shared function uses the Resolve rule only.

  • Flame (ayon_flame/otio/flame_export.py create_otio_markers): JSON only when re.findall(r"[{:}]", comment) gives 3 or more hits. Anything else, {}, {a} or see {shot}, becomes {"comment": text}.
  • Resolve (davinci_export.py): JSON when "{" in note and "}" in note.
  • Shared: the Resolve rule, then json.loads; a failure logs an error and returns {}.

So with fallback_key="comment" a Flame comment that has braces but fewer than 3 { : } characters changes from kept to dropped.

Failing input (run on Python 3.12 / OTIO 0.18, same code path on 0.15):

parse_marker_text("{}")          -> {}                       # Flame: {"comment": "{}"}
parse_marker_text("{a}")         -> {}  + log.error          # Flame: {"comment": "{a}"}
parse_marker_text("see {shot}")  -> {}  + log.error          # Flame: {"comment": "see {shot}"}

tests/test_otio_markers.py:251 (test_parse_marker_text_flame_variant_json_detection) states "Flame needs 3 of {, :, }; a JSON object always has them". That is false for {}. The test only feeds a valid JSON object, so it cannot catch the difference. The characterization rule in tests/AGENTS.md (assert each host's current output, even where variants disagree) is not met, and the PR body lists no deliberate difference for this.

Fix: either keep a Flame variant (a min_json_marks / detection argument), or state the difference in the PR and add {}, {a} cases for Flame.

Severity: should-fix.

@jakubjezek001

Copy link
Copy Markdown
Member Author

Cross-review finding 70-2: flat shape decodes to a shallow copy with OTIO AnyDictionary/AnyVector values, and the round-trip test only uses scalars

client/ayon_editorial/otio/markers.py:66-69 (decode_marker_data), test tests/test_otio_markers.py:183-190

decode_marker_data documents "the metadata itself as a plain dict", but dict(metadata) is shallow. Nested values of a flat marker stay OTIO types. The json_metadata shape returns plain Python objects, so the two shapes return different types for the same data. Encode then decode is not an identity for flat markers that hold nested data:

data = {"clip_index": "c", "tasks": {"comp": {"type": "Compositing"}}, "tags": ["a"]}
for flat in (False, True):
    d = decode_marker_data(create_ayon_marker(data, flat=flat).metadata)
    print(flat, d == data, type(d["tasks"]).__name__)
-> False True  dict
-> True  False AnyDictionary      # same on OTIO 0.15.0 and 0.18.1
json.dumps(decode_marker_data(flat marker))  -> TypeError: Object of type AnyVector is not JSON serializable

Hosts write nested creator data (task dicts, lists) into the Flame/Resolve flat shape, so a caller that compares or serialises the result breaks on flat markers only. The round-trip test uses scalars only ({"clip_index", "folderPath", "n"}), so it passes vacuously for the flat case.

Fix: convert recursively in the flat branch (a small recursive converter for AnyDictionary and AnyVector), and add nested data to test_create_ayon_marker_round_trip.

Severity: should-fix.

@jakubjezek001

Copy link
Copy Markdown
Member Author

Cross-review finding 70-3: module docstring claims to replace the Resolve api/lib.py lookup, which is not a clip_index lookup

client/ayon_editorial/otio/markers.py:3-6, and the issue scope ("clip-index lookup (today in each host's utils.py and client/ayon_resolve/api/lib.py)")

The only marker use in ayon_resolve/api/lib.py that works on OTIO (around line 1181 in the local checkout) is:

for marker in otio_clip.markers:
    if constants.AYON_MARKER_NAME in marker.name:
        otio_clip.metadata.update(marker.metadata)

It merges the marker data into the clip metadata by name. It is not a get_marker_from_clip_index copy. Nothing in markers.py covers it: find_clip_by_index needs a clip_index, and no helper "marker data of this clip". The other lib.py marker functions (set_ayon_marker, get_ayon_marker) call the Resolve API and json.dumps the note themselves; encode_marker_data is unused there.

So the docstring ("Replaces ... the clip lookup in ayon_resolve/api/lib.py") overstates what a Resolve adoption can drop, and the acceptance item "Markers written by every host today decode unchanged" has no characterization test for that path.

Fix: add a small helper (for example merge_marker_data(otio_clip, marker_name)) with a characterization test, or correct the docstring and the issue scope.

Severity: minor.

@jakubjezek001
jakubjezek001 force-pushed the feature/16-otio-markers branch from a784498 to 7030439 Compare October 30, 2026 13:49
@jakubjezek001

Copy link
Copy Markdown
Member Author

Author answers to 70-1 to 70-3: fixed in 7030439 (head of this PR).

  • 70-1: confirmed ({}, {a}, see {shot} differed from Flame). parse_marker_text now returns the JSON object only when it parses to a non-empty dict, otherwise the text stays under the fallback key. So {}, {shot} and see {shot} are comments as in Flame. One deliberate difference remains and is documented in the docstring and pinned by a test: text that looks like JSON but does not parse ({broken: }) stays a comment with a warning, where Flame dropped it (empty dict) and Resolve raised. The test docstring no longer claims a JSON object always has 3 of the characters.
  • 70-2: confirmed on 0.15 and 0.18. The flat shape now returns plain dict/list recursively; added a nested test, and a round trip through a written .otio file.
  • 70-3: the module docstring no longer claims to replace ayon_resolve/api/lib.py; it says that code merges marker metadata by name and stays in Resolve.

@jakubjezek001
jakubjezek001 force-pushed the feature/16-otio-markers branch 2 times, most recently from 9a7e0ea to cd397ac Compare October 30, 2026 14:15
Decode both marker shapes (json_metadata and flat), parse host marker
text, create markers and find a clip by clip_index. Host lookups are
reproduced in the tests; Resolve's marker unwrap stays in Resolve and is
passed in.
OTIO 0.15 has no Timeline.find_clips (added in 0.16), so the verbatim
host copies in the marker tests failed on the 0.15 CI job.
@jakubjezek001
jakubjezek001 force-pushed the feature/16-otio-markers branch from cd397ac to bd4f6fa Compare October 30, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type: feature Adding something new and exciting to the product

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Editorial P2.4]: AYON marker codec (otio/markers.py)

1 participant