Add regression-only Nori-Rel 30M - #17
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 3411564. Configure here.
adrian-prior
left a comment
adrian-prior
left a comment
There was a problem hiding this comment.
Thanks a lot for the submission! I left some minor comments, but in general this looks good to me. Let me know, if this is the final state I should run. For now, we have run submissions in the following way (e.g. for RT-PluRel):
- You tell me when the code is ready
- I run everything on Prior Labs infrastructure and commit the results
- You compare the results to what you got in your test runs -> if there are issues you can fix it and we do a rerun (of everything); Otherwise, we merge the PR.
Let me know, if you have any questions :)
| @@ -0,0 +1,303 @@ | |||
| { | |||
There was a problem hiding this comment.
Optional suggestion: You might want to add cell outputs, so users can easily read the notebook without having to run all cells themselves.
| The [Nori-Rel walkthrough](examples/nori_rel.ipynb) covers installation, | ||
| choosing a regression task, warming its DFS feature cache, running the frozen | ||
| 30M checkpoint, and saving the result. | ||
|
|
There was a problem hiding this comment.
I think this can be removed. I am unsure how this fits into the dependency section, and you already have a reference in the examples README.
| @@ -0,0 +1,5 @@ | |||
| """Nori-Rel: depth-2 DFS features with the frozen Nori 30M regressor.""" | |||
There was a problem hiding this comment.
I know that you describe how to add warm the cache in the notebook, but could you also add a warm_cache.py file like we have for most other methods (that don't share the cache with others)
| # prepared. The exact dependency pin above keeps this exemption reproducible. | ||
| exclude-newer-package = { relational-transformer = false } | ||
| # Exact model releases may be newer than the rolling global cutoff. | ||
| exclude-newer-package = { relational-transformer = false, synthefy = false, synthefy-nori = false } |
There was a problem hiding this comment.
Is this change needed? Otherwise, please revert.
| """Nori-Rel: frozen Nori 30M over depth-2 DFS features. | ||
|
|
||
| Nori performs regression by in-context learning; ``fit`` stores labeled rows and | ||
| does not update the public checkpoint. Large contexts use a seeded random window | ||
| with cache offload instead of silent quantization or subsampling. |
There was a problem hiding this comment.
As far as I can see, Nori-Rel makes pretty heavy use of the TabPFN-Rel harness, which in turn uses featuretools and many learnings from rdblearn. I think it would be nice to attribute this here and also document the differences/improvements that nori-rel (beyond swapping out the TFM).
2 participants
Summary
nori-relfor regression tasks only.Protocol
synthefy-nori==0.19.0,synthefy==7.0.463c9f7facf9fb32c37ce3fc2fba331d524696318818433f8af12c1137b96d9ff47e109b4eef5818d4e52a9656b2e573dbf13b74dThe model has no tuning axis and does not update the checkpoint. An internal seed-0 sweep covers all 9 regression tasks and wins 7/9 head-to-head against the selected
tabpfn-rel-clientbaseline at benchmark commitefac8f26435ff4cb682b2ca19329fb391a2bfa58. Nobaseline_resultsrows are included pending the regression-only reporting decision in #16.Checks
uv run pre-commit run --all-filesOMP_NUM_THREADS=1 uv run pytest— 396 passed, 20 skippeduv buildRelated to #16.