Decoder fails to parse nested objects within arrays (returns empty objects) #33
Description
Decoder fails to parse nested objects within arrays (returns empty objects)
Description
The TOON decoder fails to properly decode nested objects within arrays. When encoding an array containing objects with nested structures, the encoder produces correct TOON format, but the decoder returns an array of empty objects [{}] instead of the original nested structure.
Steps to Reproduce
from toon_format import encode, decode
# Test data with nested object in array
test_data = [
{
"a": {
"b": "123",
},
}
]
# Encode - works correctly
encoded = encode(test_data, {"lengthMarker": "#"})
print("Encoded:")
print(encoded)
# Output:
# [#1]:
# -
# a:
# b: "123"
# Decode - fails
decoded = decode(encoded)
print("Decoded:", decoded)
# Output: [{}] ❌ Should be: [{"a": {"b": "123"}}]Expected Behavior
The decoder should return the original nested structure:
[{"a": {"b": "123"}}]Actual Behavior
The decoder returns an array with an empty object:
[{}]Environment
- Library version:
0.9.0-beta.1(commit:9c4f0c0c24f2a0b0b376315f4b8707f8c9006de6) - Python version: Python 3.12
- Installation:
toon-format @ git+https://github.com/toon-format/toon-python.git
Additional Information
What works:
- ✅ Flat objects in arrays decode correctly
- ✅ Nested objects at root level (not in arrays) decode correctly
- ✅ Arrays of primitives decode correctly
- ✅ Encoder produces correct TOON format for nested structures
What doesn't work:
- ❌ Nested objects within array items (using list format with
-markers)
Encoded TOON format (correct):
[#1]:
-
a:
b: "123"
The format appears correct, but the decoder fails to parse the nested structure within the array item.
Impact
This bug affects any use case where:
- Arrays contain objects with nested structures
- The agent output format includes nested objects within classification arrays
- Complex data structures need to be encoded/decoded with TOON
Workaround
Currently, the only workaround is to flatten nested structures before encoding:
# Instead of: [{"a": {"b": "123"}}]
# Use: [{"a_b": "123"}]However, this is not always feasible when the nested structure is required by the application logic.