Re-encoding live synthesized callables can save another function

[datvo06]

Follow-up to BasisResearch/RoboTL#576.

When we keep several synthesized loss factories alive and checkpoint them through Encodable, an earlier factory can be saved as a different function. I reproduced this on master with Python 3.14: the live functions return the right values, but some return another function's values after serialization and replay. This breaks evaluation caching and resume.

The synthesized filenames use Python object IDs, which can be recycled after decoding. A later decode can then overwrite the earlier function's cached source. One solution could be giving each synthesized source a unique filename, as we already do for REPL snippets.

[datvo06]

Reproduction:

from collections.abc import Callable
from pydantic import TypeAdapter
from effectful.handlers.llm import Encodable, Skill


@Skill.define
def propose(i: int) -> Callable[[int], int]:
      """Return a standalone function f(x: int) -> int computing x + {i}.

      Write {i} as an integer literal in the function body.
      """


functions = [propose(i) for i in range(16)]
codec = TypeAdapter(Encodable[Callable[[int], int]])
before = [f(0) for f in functions]
saved = [codec.dump_json(f) for f in functions]
after = [codec.validate_json(s)(0) for s in saved]

print("Live:", before)
print("Replayed:", after)
assert before == list(range(16))
assert after == before

Running:

  python -m effectful.handlers.llm.harness repro.py \
    --model gpt-4.1-mini --tool-choice none

Result:

  Live:     [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
  Replayed: [0, 1, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15, 15]
  AssertionError