Skip to content

[feat][evaluation] third party evaluator #331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
HearyShen wants to merge 29 commits into
base: main
Choose a base branch
from
+3,910 −190
Open

[feat][evaluation] third party evaluator #331

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
bdd26ff
feat: init idl for third_party_evaluator
HearyShen Nov 27, 2025
dfc6f9f
feat: implement third party evaluator crud and run/debug
HearyShen Nov 28, 2025
d5f60da
feat: implement third party evaluator crud and run/debug
HearyShen Nov 28, 2025
84051a6
feat: implement third party evaluator crud and run/debug
HearyShen Nov 28, 2025
4e539cf
feat: add OldCustomEvaluatorCompatibilityInfo to CustomRPCEvaluatorVe…
HearyShen Dec 2, 2025
0f95d53
feat: replace OldCustomEvaluatorCompatibilityInfo with Ext for openso…
HearyShen Dec 4, 2025
afa1b7a
feat: add ext map to CustomRPCEvaluator DTO
HearyShen Dec 4, 2025
5302ebd
feat: move EvaluatorRunConfig into EvaluatorIDVersionItem considering…
HearyShen Dec 4, 2025
681ee22
feat: add EvaluatorRunConf into experiment EvaluationConfiguration DO…
HearyShen Dec 4, 2025
b035414
feat: custom rpc evaluator rely on separate dataset and target fields
HearyShen Dec 4, 2025
8a1f12f
Merge branch 'main' into feat/third_party_evaluator
HearyShen Dec 5, 2025
ea87987
Merge branch 'main' into feat/third_party_evaluator
HearyShen Dec 8, 2025
17ede8d
Merge branch 'main' into feat/third_party_evaluator
HearyShen Dec 12, 2025
7f355ba
Merge branch 'main' into feat/third_party_evaluator
HearyShen Dec 15, 2025
fbacfcf
Merge branch 'main' into feat/third_party_evaluator
HearyShen Dec 18, 2025
c53cf15
ut: fix unit tests
HearyShen Dec 18, 2025
4943a59
ut: add unit tests
HearyShen Dec 18, 2025
988fee8
feat: use evaluator_id_version_list in CreateExperimentRequest and Ex…
HearyShen Dec 18, 2025
e412f26
feat: use evaluator_id_version_list in CreateExperimentRequest and Ex…
HearyShen Dec 19, 2025
826ef1d
ut: add unit tests for CreateExperiment
HearyShen Dec 26, 2025
36fcc55
ut: add unit tests for EvalConfConvert
HearyShen Dec 26, 2025
fd03fc0
ut: add unit tests for ToExptDTO
HearyShen Dec 26, 2025
8960ecf
Merge branch 'main' into feat/third_party_evaluator
HearyShen Dec 26, 2025
c08d58e
secure: add authCustomRPCEvaluatorContentWritable to Debug/BatchDebug…
HearyShen Dec 29, 2025
251ec93
fix
dsf86 Dec 30, 2025
fcb8203
fix(evaluation): item eval lock timeout
lsy357 Dec 31, 2025
039e22e
Merge branch 'main' into feat/third_party_evaluator
HearyShen Dec 31, 2025
64fbd78
fix
dsf86 Jan 4, 2026
1ed13af
Merge branch 'feat/third_party_evaluator' of github.com:coze-dev/coze…
dsf86 Jan 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backend/go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,7 +260,7 @@ require (
gopkg.in/yaml.v2 v2.4.0
gopkg.in/yaml.v3 v3.0.1 // indirect
gorm.io/driver/sqlite v1.5.5 // indirect
gorm.io/hints v1.1.2 // indirect
gorm.io/hints v1.1.2
gorm.io/rawsql v1.0.3-0.20250401110442-7e49778bc820
stathat.com/c/consistent v1.0.0 // indirect
)
Expand Down
1 change: 0 additions & 1 deletion backend/go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -473,7 +473,6 @@ github.com/google/pprof v0.0.0-20240827171923-fa2c70bbbfe5/go.mod h1:vavhavw2zAx
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/s2a-go v0.1.8 h1:zZDs9gcbt9ZPLV0ndSyQk6Kacx2g/X+SKYovpnz3SMM=
github.com/google/s2a-go v0.1.8/go.mod h1:6iNWHTpQ+nfNRN5E00MSdfDwVesa8hhS32PhPO8deJA=
github.com/google/subcommands v1.2.0 h1:vWQspBTo2nEqTUFita5/KeEWlUL8kQObDFbub/EN9oE=
github.com/google/subcommands v1.2.0/go.mod h1:ZjhPrFU+Olkh9WazFPsl27BQ4UPiG37m3yTrtFlrHVk=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
Expand Down
Loading
Loading