Skip to content

Store reviewer logs as markdown files - #110

Open
skaltman wants to merge 17 commits into
mainfrom
reviewer-log-82
Open

Store reviewer logs as markdown files#110
skaltman wants to merge 17 commits into
mainfrom
reviewer-log-82

Conversation

@skaltman

@skaltman skaltman commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Change trajectory reviewer log format from .jsonl to Markdown.

Each conversation gets its own Markdown file. Each Markdown file has a YAML header and a Markdown body:

  • The YAML front matter contains the flags, notes, and metadata needed to restore review state in the reviewer app.
  • The Markdown body contains the conversation and repeats its flags and notes for the purposes of human readability. The YAML remains the source of truth for the flags and notes.

Locally, the review files are written to a directory set by the COMMONS_REVIEW_DIR environment variable or the review_dir arg to trajectory_review(). On Connect, they are stored in a pin so they persist across sessions.

Markdown format
---
generated_by: commons::trajectory_review
schema_version: 1
conversation: 20260728t222656.055-r0ai9xw7ic
last_active: 2026-07-28T15:33:48-0700
updated_at: '2026-08-14T22:14:01Z'
flags:
  conversation: no
  exchanges:
    - 1
notes:
  - user: unknown
    time: '2026-08-14T21:58:01Z'
    text: note about question 1
    exchange: 1
  - user: unknown
    time: '2026-08-14T22:13:45Z'
    text: note on question with searched context
    exchange: 3
  - user: unknown
    time: '2026-08-14T22:14:01Z'
    text: note on question with search pool
    exchange: 1
---

# 20260728t222656.055-r0ai9xw7ic

<!-- Generated by commons::trajectory_review(). Manual edits are not supported. -->

## Conversation-level review

**Flagged:** No

**Review notes:** None

## Exchange 1

**Trust:** Verified (A)

**Flagged:** Yes

**Review notes:**

> note about question 1
>
> _unknown, 2026-08-14T21:58:01Z_

> note on question with search pool
>
> _unknown, 2026-08-14T22:14:01Z_

### User

> How many patients had at least one serious adverse event, by arm?

### Tool call: `search_pool`

#### Arguments

 ```json
  {
    "query": "patients with at least one serious adverse event by arm"
  }
```
### Tool result: `search_pool`

_Discovery result omitted from the review log._

### Tool call: `call_measure`

#### Arguments

```json
{
  "name": "ae_overview",
  "arguments": "{\"population\": \"SAF\"}"
}
```

### Tool result: `call_measure`

|statistic                                                  |A: Drug X   |B: Placebo |C: Combination |
|:----------------------------------------------------------|:-----------|:----------|:--------------|
|Total number of patients with at least one AE              |100 (74.6%) |98 (73.1%) |103 (78.0%)    |
|Total number of AEs                                        |502         |480        |604            |
|Total number of deaths                                     |25 (18.7%)  |23 (17.2%) |22 (16.7%)     |
|Total number of patients withdrawn from study due to an AE |3 (2.2%)    |6 (4.5%)   |5 (3.8%)       |
|Total number of patients with at least one                 |NA          |NA         |NA             |
|Serious AE                                                 |85 (63.4%)  |80 (59.7%) |87 (65.9%)     |
|Related Serious AE                                         |64 (47.8%)  |52 (38.8%) |64 (48.5%)     |
|Related AE                                                 |86 (64.2%)  |85 (63.4%) |92 (69.7%)     |
|Severe AE (at greatest intensity)                          |77 (57.5%)  |70 (52.2%) |79 (59.8%)     |
|Grade 3-5 AE                                               |90 (67.2%)  |83 (61.9%) |93 (70.5%)     |
|Grade 4-5 AE                                               |77 (57.5%)  |70 (52.2%) |79 (59.8%)     |

Result stored as `r1`, available in your R session.
A data frame with 11 rows and 4 columns:
* statistic: character with 0 NAs, and 11 unique values
* A: Drug X: character with 1 NAs, and 9 unique values ("100 (74.6%)", "502", "25 (18.7%)", "3 (2.2%)", "85 (63.4%)", "64 (47.8%)", "86 (64.2%)", "77 (57.5%)", "90 (67.2%)")
* B: Placebo: character with 1 NAs, and 9 unique values ("98 (73.1%)", "480", "23 (17.2%)", "6 (4.5%)", "80 (59.7%)", "52 (38.8%)", "85 (63.4%)", "70 (52.2%)", "83 (61.9%)")
* C: Combination: character with 1 NAs, and 9 unique values ("103 (78.0%)", "604", "22 (16.7%)", "5 (3.8%)", "87 (65.9%)", "64 (48.5%)", "92 (69.7%)", "79 (59.8%)", "93 (70.5%)")

### Assistant

> Using the ae_overview measure (safety population), patients with at least one serious adverse event by arm:
>
> - Arm A (Drug X): 85 (63.4%)
> - Arm B (Placebo): 80 (59.7%)
> - Arm C (Combination): 87 (65.9%)
Rendered --- generated_by: commons::trajectory_review schema_version: 1 conversation: 20260728t222656.055-r0ai9xw7ic last_active: 2026-07-28T15:33:48-0700 updated_at: '2026-08-14T22:14:01Z' flags: conversation: no exchanges: - 1 notes: - user: unknown time: '2026-08-14T21:58:01Z' text: note about question 1 exchange: 1 - user: unknown time: '2026-08-14T22:13:45Z' text: note on question with searched context exchange: 3 - user: unknown time: '2026-08-14T22:14:01Z' text: note on question with search pool exchange: 1 ---

20260728t222656.055-r0ai9xw7ic

Conversation-level review

Flagged: No

Review notes: None

Exchange 1

Trust: Verified (A)

Flagged: Yes

Review notes:

note about question 1

unknown, 2026-08-14T21:58:01Z

note on question with search pool

unknown, 2026-08-14T22:14:01Z

User

How many patients had at least one serious adverse event, by arm?

Tool call: search_pool

Arguments

 {
   "query": "patients with at least one serious adverse event by arm"
 }

Tool result: search_pool

Discovery result omitted from the review log.

Tool call: call_measure

Arguments

{
  "name": "ae_overview",
  "arguments": "{\"population\": \"SAF\"}"
}

Tool result: call_measure

statistic A: Drug X B: Placebo C: Combination
Total number of patients with at least one AE 100 (74.6%) 98 (73.1%) 103 (78.0%)
Total number of AEs 502 480 604
Total number of deaths 25 (18.7%) 23 (17.2%) 22 (16.7%)
Total number of patients withdrawn from study due to an AE 3 (2.2%) 6 (4.5%) 5 (3.8%)
Total number of patients with at least one NA NA NA
Serious AE 85 (63.4%) 80 (59.7%) 87 (65.9%)
Related Serious AE 64 (47.8%) 52 (38.8%) 64 (48.5%)
Related AE 86 (64.2%) 85 (63.4%) 92 (69.7%)
Severe AE (at greatest intensity) 77 (57.5%) 70 (52.2%) 79 (59.8%)
Grade 3-5 AE 90 (67.2%) 83 (61.9%) 93 (70.5%)
Grade 4-5 AE 77 (57.5%) 70 (52.2%) 79 (59.8%)

Result stored as r1, available in your R session.
A data frame with 11 rows and 4 columns:

  • statistic: character with 0 NAs, and 11 unique values
  • A: Drug X: character with 1 NAs, and 9 unique values ("100 (74.6%)", "502", "25 (18.7%)", "3 (2.2%)", "85 (63.4%)", "64 (47.8%)", "86 (64.2%)", "77 (57.5%)", "90 (67.2%)")
  • B: Placebo: character with 1 NAs, and 9 unique values ("98 (73.1%)", "480", "23 (17.2%)", "6 (4.5%)", "80 (59.7%)", "52 (38.8%)", "85 (63.4%)", "70 (52.2%)", "83 (61.9%)")
  • C: Combination: character with 1 NAs, and 9 unique values ("103 (78.0%)", "604", "22 (16.7%)", "5 (3.8%)", "87 (65.9%)", "64 (48.5%)", "92 (69.7%)", "79 (59.8%)", "93 (70.5%)")

Assistant

Using the ae_overview measure (safety population), patients with at least one serious adverse event by arm:

  • Arm A (Drug X): 85 (63.4%)
  • Arm B (Placebo): 80 (59.7%)
  • Arm C (Combination): 87 (65.9%)

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown

Preview deployed to Connect (dogfood.team.pct.posit.it): https://dogfood.team.pct.posit.it/connect/#/apps/d7a36cae-8f27-448b-a478-61b81fbe3942/draft/361570

Deployed from commit a6b457d.

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown

Preview deployed to Connect (connect.staging.pct.posit.it): https://connect.staging.pct.posit.it/connect/#/apps/ad662e1b-5048-4acc-9ad7-f9478c92274e/draft/1804

Deployed from commit a6b457d.

@skaltman
skaltman marked this pull request as ready for review August 14, 2026 23:38
@skaltman
skaltman requested a review from simonpcouch August 14, 2026 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant