[deps] Pandoc 3.8.3, typst 0.14.2

[cscheid]

Brings:

• Pandoc to 3.8.3
• typst to 0.14.2

[posit-snyk-bot]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scanner	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.

[cscheid]

@tarleb It seems that there's a -f markdown change from Pandoc 3.6.3 to 3.7.0.2. Let's talk about it when we meet next? Thanks!

[tarleb]

Sorry, I hadn't seen this. I'll try to be there today.

[cscheid]

@tarleb suggests " -> \" as a path forward. Let's try it.

[cscheid]

@tarleb suggests " -> \" as a path forward. Let's try it.

On further thought, this only works if we know that the replacement for the shortcode is in a position where this escaping is required. In order to do that, we would have to enrich the LPEG parser to learn about parsing links and images, and then to process shortcodes differently inside links and images. That's a pretty large undertaking.

[cscheid]

@gordonwoodhull We should take a look at this PR together next week; there's some brand+typst failures on the update to Pandoc 3.8.3 that I think you'll be better equipped to sort out :)

Typst (sometimes +brand, often #NormalTok) failures:

• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985308?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985266?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985297?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985262?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985274?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985286?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985304?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985249?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985267?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985276?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985285?pr=13249 (typst + brand + escaping issue)
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985281?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228906/job/57982977618?pr=13249

JATS failures (looks like the Pandoc output doesn't validate?)

• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985295?pr=13249

Docusaurus failures (apparently from display math markdown writer changing)

• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985300?pr=13249

LaTeX compilation failures:

• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985291?pr=13249
• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985305?pr=13249

DocX snapshot failures:

• https://github.com/quarto-dev/quarto-cli/actions/runs/20197228910/job/57982985303?pr=13249

[gordonwoodhull]

I've pushed a fix: page inset needs to be typst raw inline.

Most of the other failures seem to be because we need to update the Typst templates from Pandoc to get Skylighting support. Not trivial as there's a bunch of new stuff, but not that bad.

[gordonwoodhull]

Importing highlighting-definitions in the typst template fixes all of the typst failures except for this listings one:

docs/smoke-all/crossrefs/float/typst/typst-listings-1.qmd

So all the stuff I need to update (merging Chris Kenny's improvements with brand yaml, basically) is not reflected in tests.

TL;DR We've been applying highlight-style for all formats, it just wasn't implemented for Typst before.

Now syntax highlighting will match by default!

Claude's explanation:

Quarto always sets highlight-style for ALL formats, including typst:

1. Default value: kDefaultHighlightStyle = "arrow" defined in src/command/render/constants.ts:8 since March 2021
2. Applied unconditionally: resolveTextHighlightStyle() in src/command/render/pandoc.ts:1741 is called for all formats with no format-specific checks
3. For typst: Since extras.html?.[kTextHighlightingMode] is undefined for typst, it falls through to the default case (line 1789-1796) which sets highlight-style to the light theme path
4. Result: Quarto passes --highlight-style=<path-to-arrow-light.theme> to pandoc even for typst output

Before pandoc 3.8/3.9: This didn't matter because pandoc ignored the highlight-style for typst and used typst's native highlighting.

After pandoc 3.8/3.9: Pandoc now respects --syntax-highlighting for typst output, so quarto's default "arrow" style triggers skylighting for ALL code blocks, including unlanguaged ones (wrapped in NormalTok).

What is NormalTok and why are unlanguaged blocks also invoking skylighting?

NormalTok is a TokenType in skylighting for "regular, unhighlighted text within a highlighted code block."

The typst writer explicitly adds a "default" class to code blocks before calling highlight, even when no language is specified. This allows skylighting to find a syntax definition and process all code blocks.

So the flow is:

1. Quarto passes --syntax-highlighting=arrow to pandoc (the default)
2. Pandoc's typst writer adds "default" class to unlanguaged code blocks
3. Skylighting finds a syntax for "default" and tokenizes everything as NormalTok
4. Output: #Skylighting(([#NormalTok("some code.");],))
5. This triggers highlighting-definitions to be included in the template

[cderv]

Thanks @gordonwoodhull! What I mentioned yesterday is also that Pandoc now uses --syntax-highlighting flag. This is new from pandoc 3.8 https://pandoc.org/releases.html#pandoc-3.8-2025-09-06

Add a new command line option --syntax-highlighting; this takes the values none, default, idiomatic, a style name, or a path to a theme file. It replaces the --no-highlighting, --highlighting-style, and --listings options, which will still work but with a deprecation warning. (Albert Krewinkel)

So I think we need to check if we use that flag, and probably anticipate the end of deprecation warning in our default file use.

We've been applying highlight-style for all formats,

The new option is now syntax-highlighting to use in the default file
https://pandoc.org/MANUAL.html#general-writer-options-1

Currently we use

highlight-style: C:\Users\chris\Documents\DEV_R\quarto-cli\src\resources\pandoc\highlight-styles\arrow-light.theme

And maybe in other places.

Should we do this in another PR maybe ? It is kind of part of the pandoc update though.

[cderv]

I am just updating to benefit from new condensed logging

[cscheid]

I think the biggest cause of failures right now if from a LaTeX change in how table counters work in the Pandoc writer and templates. I still don't understand how to avoid that problem.

[cderv]

Oh, I have dealt with that recently for the rticles R package. I needed to add the missing template part to the rticles' built-in template (rstudio/rticles#596)
Initially, this is the Pandoc change jgm/pandoc@e13aa5c following report jgm/pandoc#11141 and then the latex template change: jgm/pandoc@d835461 (jgm/pandoc#11201).

They do something in the Writer that expects something in the template. It could be either this template addition, or some adaptation in the way we create the table.

Also related, this change in Pandoc is for ensuring that unlabelled tables don't increment counter. I remember this was a problem in cross ref with counter workaround to decrement or something, so it could be now conflicting with what Pandoc is doing.

Hope it helps understand. (Disclaimer: I did not look at failing examples, just what I looked for my rticles maintainance about this)

[gordonwoodhull]

Pushed the typst template updates.

[cderv]

I have done remaining update for template file, mostly beamer and LaTeX.

I skipped revealjs as it will require a bit more work so this will be done later in #13852

I'll handle test failures next

[cderv]

• ./smoke/render/render-format-extension.test.ts and ./smoke/extensions/extension-render-journals.test.ts are failing because it requires all the quarto-journals to be updated. We don't have a good system to have some extensions for Quarto release, and some version for Quarto PreRelease. So I wonder if we should just opt-out the test for now... I'll think of something

• ./smoke/manuscript/render-manuscript.test.ts fails on JATS validation. This is because there is two times the same id. One of figure and one on the table

       <fig id="tbl-history">
         <caption><p>Table 1: Recent historic eruptions on La
         Palma</p></caption>
         <table-wrap>
   -        <table>
   +        <table id="tbl-history">

  Possibly related to pandoc change for table attributes ? https://pandoc.org/MANUAL.html#extension-table_attributes

  Attributes may be attached to tables by including them at the end of the caption. (For the syntax, see header_attributes.)

  : Here's the caption. {#ident .class key="value"}

  And if so, it won't impact only JATS I guess.

  UPDATE: this is confirmed as setting from: markdown-table_attributes will fix the issue !
  So we should probably for now opt-out this extensions until we can change our support. 🤔

  I'll push a commit to try out this so we can handle later the support.

It will solve also

• ./smoke/jats/render-manuscript.test.ts

[cderv]

I confirm some of the problem may come from what I mentioned earlier at #13249 (comment)

Tests failing because of this

• docs/smoke-all/2023/10/24/7334.qmd
• docs/smoke-all/article-layout/tables/compute-table-margin.qmd
• docs/smoke-all/2023/11/21/7655.qmd
• docs/smoke-all/2023/10/18/7267.qmd
• docs/smoke-all/2023/11/02/7262.qmd

Pandoc is now doing the following

  let makeUnnumbered x = "{\\def\\LTcaptype{none} % do not increment counter" $$ x $$ "}"
  return
    $ (if null capt then makeUnnumbered else id)
    $ "\\begin{longtable}[]" <>
          braces ("@{}" <> colDescriptors isSimpleTable tbl <> "@{}")
          -- the @{} removes extra space at beginning and end

When no caption, pandoc will insert \def\LTcaptype{none} but when a caption it won't.
However, Quarto is handling the caption itself, and pandoc does not know about it. So it will still insert \def\LTcaptype{none} even when there is a \caption which leads to the error.

     Package caption Error: No float type 'none' defined.
      
      See the caption package documentation for explanation.
      Type  H <return>  for immediate help.
       ...                                              
                                                        
      l.196 \caption{\label{tbl-table1}A table}

So we need to find a way to communicate to pandoc, or post process the tex file in some situations.

As a reminder, this change in Pandoc was done so that uncaptionned longtable created by Pandoc don't increment the counter, which I think we are doing differently.
Initially like this #7271 but files still has comment, even if we don't use addtocounter anymore.

quarto-cli/src/resources/filters/quarto-post/latex.lua

Lines 302 to 311 in f33f44f

	-- Pandoc emits longtable environments by default;
	-- longtable environments increment the _table_ counter (!!)
	-- http://mirrors.ctan.org/macros/latex/required/tools/longtable.pdf
	-- (page 13, definition of \LT@array)
	--
	-- This causes double counting in our table environments. Our solution
	-- is to decrement the counter manually after each longtable environment.
	--
	-- This hack causes some warning during the compilation of the latex document,
	-- but the alternative is worse.

I am not sure what we do now... 🤔 maybe using \longtable*. Anyhow,

• Either we adapt
• Or we consider that we don't need Pandoc fix at all because we already handle it and we remove \def\LTcaptype{none} in post processing if we see it.

However, it seems longtable and latex has evolved since the counter problem so it may require some redesign. 🤷‍♂️

Update 08/01

I finally found why we get bitten by that while we are handling caption usually.

This happens because we do call pandoc.write() within our LUA filter

quarto-cli/src/resources/filters/customnodes/floatreftarget.lua

Line 415 in ee3f232

local raw_output = pandoc.RawBlock("latex", pandoc.write(pandoc.Pandoc({tbl}), "latex"))

I have verified that all failing test goes through that part.

This means we get the LTcaptype{none} added as the table does not have caption for Pandoc.
The caption being handled by floatreftarget processing this now RawBlock content.

We could maybe remove the inserted latex when we process the raw table 🤔

quarto-cli/src/resources/filters/customnodes/floatreftarget.lua

Lines 440 to 458 in ee3f232

	local longtable_match, longtable_pattern = _quarto.modules.patterns.match_in_list_of_patterns(el.text, _quarto.patterns.latexLongtableEnvPatterns)
	if longtable_match and longtable_pattern then
	made_fix = true
	local raw = el
	-- special case for longtable floats in LaTeX
	local extended_pattern = {".-"}
	for _, pattern in ipairs(longtable_pattern) do
	table.insert(extended_pattern, pattern)
	end
	table.insert(extended_pattern, ".*")
	local longtable_preamble, longtable_begin, longtable_content, longtable_end, longtable_postamble = _quarto.modules.patterns.match_all_in_table(extended_pattern)(raw.text)
	if longtable_preamble == nil or longtable_begin == nil or longtable_content == nil or longtable_end == nil or longtable_postamble == nil then
	warn("Could not parse longtable parameters. This could happen because the longtable parameters\n" ..
	"are not well-formed or because of a bug in quarto. Please consider filing a bug report at\n" ..
	"https://github.com/quarto-dev/quarto-cli/issues/, and make sure to include the document that\n" ..
	"triggered this error.")
	return {}
	end
	-- split the content into params and actual content

[cderv]

Among failing test, we have indeed the docusaurus change

Previously

---
title: Equations
format: docusaurus-md
---




$$
price = \hat{\beta}_0 + \hat{\beta}_1 \times area + \epsilon
$$

Inline equations as well: $e = mc^2$.

and now

---
title: Equations
format: docusaurus-md
---





$$price = \hat{\beta}_0 + \hat{\beta}_1 \times area + \epsilon$$

Inline equations as well: $e = mc^2$.

I am trying to find what change about the newlines. It seems this is the pandoc.write() call here

quarto-cli/src/resources/extensions/quarto/docusaurus/docusaurus_citeproc.lua

Line 15 in 2d33d1c

return pandoc.Pandoc({pandoc.RawBlock('markdown', result_str)})

I can't find for sure if the syntax change is problematic for docusaurus math yet. If not, we could just fix the snapshot.

UPDATE 07/01

I see in pandoc 3.7, this line (Source: https://pandoc.org/releases.html#pandoc-3.7-2025-05-14)

Omit initial newlines in gfm math blocks to avoid an ugly blank line.

So maybe it could be linked... I notice this change in Panodc

# with quarto 1.8.26 and old pandoc
❯ echo -e '$$\ne = mc^2\n$$' | /opt/quarto/bin/quarto pandoc -f markdown -t markdown_strict+tex_math_dollars
$$
e = mc^2
$$

# with this branch pandoc
❯ echo -e '$$\ne = mc^2\n$$' | ./package/dist/bin/quarto pandoc -f markdown -t markdown_strict+tex_math_dollars

$$e = mc^2$$

So this is why new line changed... but this is not with gfm. So side effect ?

---

Looking more into this there is two problems:

Pandoc change removing new lines

I tested how Docusaurus (with remark-math + rehype-katex) renders display math with different formats.

The breaking change:

Pandoc	Output format	Docusaurus rendering
3.6.3	$$\nequation\n$$ (newlines inside)	Display math (centered, block)
3.8.3	$$equation$$ (single line)	Inline math (left-aligned)

Root cause: remark-math only recognizes $$\n...\n$$ (with newlines inside the delimiters) as display math. The single-line $$...$$ format is parsed as inline math, even when it's on its own paragraph.

HTML output difference:

...

...

I used a test repo where I created minimal Docusaurus site with KaTeX to verify. Both $...$ and $$...$$ single-line formats render identically as inline/left-aligned math. Only $$\n...\n$$ with inner newlines renders as centered display math.

Our Docusaurus format should use WebTeX by default!

I opened an issue with the detail

• [Docusaurus] html-math-method: webtex config ignored - math rendered as tex_math_dollars #13853

But in short

• html-math-method: webtex is supposed to be the default as documented https://quarto.org/docs/output-formats/docusaurus.html#latex-math
• There was probably no test so it was broken with custom writer change

So the issue with new pandoc is mainly for html-math-method: katex where the display math won't be recognize anymore. Still unsure why pandoc changed this. 🤔

UPDATE

I opened an issue to pandoc as I am not sure this is really expected change for blocks like Display Math

• display math newlines removed in Markdown output (3.7 vs 3.6.4) jgm/pandoc#11384

Right now, we'll need to post process to add new lines after $$ or preprocess to avoid converting Display Math somehow.

UPDATE 08/01

Pandoc has fixed the issue with the newline in math at jgm/pandoc#11384

So we could opt-out the test for now and reactivate for next Pandoc that will have the fix ?

[cderv]

Docx change in snapshot is docs/smoke-all/2024/01/31/8507.qmd.

Update on 07/01

In order to help find such issues in the future and be quicker, I have (with claude help) improve our snapshot error callback to include a diff

We can see the Docx change is only about ids, probably due to new internal reference doc in pandoc. so I'll update the snapshot as it is safe

[cderv]

Remaining Test Failures

1. LaTeX longtable caption error (Package caption Error: No float type 'none' defined)

   • Root cause: Pandoc 3.8+ wraps uncaptioned longtables with {\def\LTcaptype{none}...} but Quarto adds captions separately
   • Tests:

     ./run-tests.sh docs/smoke-all/2023/10/24/7334.qmd
     ./run-tests.sh docs/smoke-all/2023/10/18/7267.qmd
     ./run-tests.sh docs/smoke-all/2023/11/02/7262.qmd
     ./run-tests.sh docs/smoke-all/2023/11/21/7655.qmd
     ./run-tests.sh docs/smoke-all/article-layout/tables/compute-table-margin.qmd

2. commonmark_x format failures (needs investigation)

Fixed by ee3f232

Tests:

./run-tests.sh docs/smoke-all/2023/04/11/format_underscores.qmd
./run-tests.sh docs/smoke-all/2023/05/01/5377.qmd

This is a new one because of 72452ee I did to try avoid the duplicate id problem but new table_attributes is not supported by commonmark_x so I can't do table_attribute = false without checks 🫤

4. quarto-journals extension failures (extensions need Pandoc 3.8 updates)

   • Tests:

     ./run-tests.sh ./smoke/render/render-format-extension.test.ts
     ./run-tests.sh ./smoke/extensions/extension-render-journals.test.ts

   For this one I don't get a good solution yet, because

   • If we update the Journals template repo, then some users may install them from github with Quarto 1.8 and it won't work.
   • If we merge this PR before, the formats won't be working until we dod the update.

   So we have a version support problem with Quarto extension as quarto install extension or quarto update extension will check main by default (and not a release). 🤔

[gordonwoodhull]

Pushed fixes and tests to address #13857 (comment)

We were still calling the Typst columns() function, which is obsolete. We missed this change in the Pandoc template in Dec 2024 and now we have #set page(columns) in page.typ, equivalent to upstream setting page in the article function.

Thanks @mcanouil for reporting this!