Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
58ea40c
feat: MCP auto-exposure of resources-server tool routes (single modul…
codex Jul 16, 2026
10aee69
refactor: remove the replay fallback; refuse loudly instead
codex Jul 16, 2026
1f5f29b
test: add TestClient-based unit tests for mcp_auto_exposure
codex Jul 16, 2026
ffd0a6a
chore: remove prototypes/ demo harness from the PR (kept locally, git…
codex Jul 16, 2026
7175882
fix: normalize MCP-namespaced tool-call names before verify
adil-a Jul 16, 2026
ed3cbc2
fix(verify): scope tool-name normalization to MCP-exposed servers, pr…
adil-a Jul 17, 2026
2288614
fix: harden mcp_auto_exposure per team review — 55 findings addressed
adil-a Jul 17, 2026
0b3606e
style: prune reviewer-narration comments; keep constraints only
adil-a Jul 17, 2026
ce98a45
feat: install-time verify wrap, mcp_excluded_paths opt-out, session-m…
adil-a Jul 17, 2026
bfaff54
feat: enable MCP exposure on in-tree tool servers; per-session allowe…
adil-a Jul 17, 2026
a117445
chore: gitignore local e2e_mcp verification harness
adil-a Jul 17, 2026
f926cdb
refactor: make expose_tools_over_mcp a config field, not a ClassVar
adil-a Jul 17, 2026
4e38dcb
scope: narrow PR to engine + tests; revert in-tree server enablement
adil-a Jul 17, 2026
72b7d66
scope: drop MCP session-token expiry (no consumer)
adil-a Jul 17, 2026
2e1e1a3
refactor: collapse the three MCP contract members into one mcp_tools(…
adil-a Jul 17, 2026
416b257
docs: make mcp_auto_exposure.py readable — reading guide, timeline ba…
adil-a Jul 17, 2026
cb118bf
scope: cut verified-dead code — cookie synthesis, install-time allow-…
adil-a Jul 20, 2026
6d4c231
scope: second verified simplification pass (912 -> 831 lines)
adil-a Jul 20, 2026
9e3af88
fix: defaulted query params refuse at startup (re-apply); docs for au…
adil-a Jul 20, 2026
8e95e93
docs: correct auto-exposure claims found by verification pass
adil-a Jul 20, 2026
4d6d569
feat!: remove gym_tool/MCPResourcesServer; auto-exposure is the one M…
adil-a Jul 20, 2026
3891dd6
docs: fix four verified inaccuracies from the removal review pass
adil-a Jul 20, 2026
0218c1b
docs: make workplace_assistant the featured Claude Code example
adil-a Jul 20, 2026
534fd1b
docs: fix seven cold-reader findings from the fresh-eyes review
adil-a Jul 21, 2026
72c0523
ci: allowlist the response_model-filtering test fixture for detect-se…
adil-a Jul 21, 2026
61fd7bb
updated workplace assistant to work with mcp server, updated preproce…
arti4nvj Jul 21, 2026
7080f27
fix: ruff lint error
arti4nvj Jul 21, 2026
4d1c2f2
fix: revert unrelated ruff changes from external files
arti4nvj Jul 21, 2026
b90e63e
Merge branch 'main' of https://github.com/NVIDIA-NeMo/Gym into arti/w…
arti4nvj Jul 25, 2026
d6244f6
Merge branch 'main' of https://github.com/NVIDIA-NeMo/Gym into arti/w…
arti4nvj Jul 25, 2026
9d2ce8b
Merge branch 'main' into arti/workplace-claude-demo
arti4nvj Jul 25, 2026
681af4d
Merge branch 'main' of https://github.com/NVIDIA-NeMo/Gym into arti/w…
arti4nvj Jul 27, 2026
033d151
nootebook updates to address MR comments
arti4nvj Jul 27, 2026
59aade6
Merge branch 'arti/workplace-claude-demo' of https://github.com/NVIDI…
arti4nvj Jul 27, 2026
5b76f57
added notebook to fern/versions/latest/pages/evaluation-tutorials/ind…
arti4nvj Jul 27, 2026
3fd59e8
nit with tab fix
arti4nvj Jul 28, 2026
c23fe79
Merge branch 'main' into arti/workplace-claude-demo
arti4nvj Jul 28, 2026
b6fc72d
Merge branch 'main' into arti/workplace-claude-demo
arti4nvj Jul 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions fern/versions/latest/pages/evaluation-tutorials/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ Here are the hands-on walkthroughs for running benchmarks, collecting rollouts,
Run the EvalPlus coding benchmark and inspect rollout and aggregate metric outputs.
</Card>

<Card title="Workplace Assistant with Claude Code" href="https://github.com/NVIDIA-NeMo/Gym/blob/main/resources_servers/workplace_assistant/notebooks/workplace-claude-demo.ipynb">
Run an agentic tool-use benchmark end to end with the Claude Code agent harness — config, rollouts, and BLADE analysis.

<Badge minimal outlined>notebook</Badge>
</Card>

<Card title="Environment List" href="/evaluation/environment-list">
Browse the built-in benchmark and training environments.
</Card>
Expand Down
7 changes: 7 additions & 0 deletions resources_servers/workplace_assistant/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,13 @@ def setup_webserver(self) -> FastAPI:
app.post("/{path}")(self.route_to_python_function)
return app

# Register all 27 workplace tools as MCP tools via the catch-all route when expose_tools_over_mcp is enabled.
def mcp_tools(self, harvested, catchall):
specs = get_tools(["email", "calendar", "analytics", "project_management", "customer_relationship_manager"])[
"schemas"
]
return harvested + [catchall.tool(s["name"], s["parameters"], s.get("description")) for s in specs]

async def seed_session(self, request: Request, body: BaseSeedSessionRequest) -> BaseSeedSessionResponse:
# init session once for each sample.
session_id = request.session[SESSION_ID_KEY]
Expand Down
18 changes: 3 additions & 15 deletions resources_servers/workplace_assistant/dataset_preprocess.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
# Run as `python resources_servers/workplace_assistant/dataset_preprocess.py --split test``
import argparse
import json
from copy import deepcopy
from dataclasses import dataclass
from typing import Any

Expand Down Expand Up @@ -84,26 +83,15 @@ def __init__(self, prompt_to_use: str = "SYS_PROMPT"):
)

def get_samples(self, split):
dataset = load_dataset("Nexusflow/250319-workplace_assistant-fulleval", split=split)
dataset = load_dataset("nvidia/Nemotron-RL-agent-workplace_assistant", split=split)

processed_samples = []

for d in dataset:
# convert into create params
create_params = deepcopy(self.base_create_params)
create_params["input"].append(
{
"role": "user",
"content": d["problem"],
}
)

ground_truth = json.loads(d["solution"]) # json loads ground truths/solutions

processed_samples.append(
self.WorkbenchSample(
create_params=create_params,
reference=ground_truth,
create_params=d["responses_create_params"],
reference=d["ground_truth"],
category=d["category"],
environment_name=d["environment_name"],
)
Expand Down
Loading
Loading