Feat/temporal vbr

[thomas-vilte]

Description

This is the last missing term from compute_vbr, and it is fully portable.

libopus compares the frame's spectral level against a running average and spends more bits on frames that stand out (celt_encoder.c):

celt_glog follow=-QCONST32(10.0f, DB_SHIFT-5);
celt_glog offset = shortBlocks?HALF32(SHL32(LM, DB_SHIFT-5)):0;
for(i=start;i<end;i++)
{
   follow = MAXG(follow-QCONST32(1.0f, DB_SHIFT-5), SHR32(bandLogE[i],5)-offset);
   if (C==2) follow = MAXG(follow, SHR32(bandLogE[i+nbEBands],5)-offset);
   frame_avg += follow;
}
frame_avg /= (end-start);
temporal_vbr = SUB32(SHL32(frame_avg, 5),st->spec_avg);
temporal_vbr = MING(GCONST(3.f), MAXG(-GCONST(1.5f), temporal_vbr));
st->spec_avg += MULT16_32_Q15(QCONST16(.02f, 15), temporal_vbr);

It then applies the result by scaling the target:

if (!has_surround_mask && tf_estimate < QCONST16(.2f, 14))
{
   amount = MULT16_16_Q15(QCONST16(.0000031f, 30), IMAX(0, IMIN(32000, 96000-bitrate)));
   tvbr_factor = SHR32(MULT16_16(SHR32(temporal_vbr, DB_SHIFT-10), amount), 10);
   target += (opus_int32)MULT16_32_Q15(tvbr_factor, target);
}

The Q-domain shifts in both blocks are identities in the float build, so the calculation is effectively in direct log2 units and the factor becomes:

temporal_vbr * amount

amount naturally goes to zero as the bitrate approaches 96 kb/s. Once there are enough bits available, shifting bits between frames based on their spectral level is no longer worth paying for.

I verified the port by instrumenting both implementations on the same clip: frame_avg, spec_avg, and temporal_vbr match frame by frame to the fifth or sixth decimal place. Over 3000 frames, the average temporal_vbr is 0.0634 in pion vs 0.0629 in libopus.

Measurement

Packet size and round-trip SNR in VBR over 3000 frames of music:

bitrate	without this term	with this term	libopus
24000	52.14 B / 6.1694 dB	52.21 B / 6.2959 dB	53.50 B / 6.3764 dB
48000	111.33 B / 9.7837 dB	111.55 B / 9.8859 dB	113.71 B / 10.0016 dB
96000	241.81 B / 14.9433 dB	241.81 B / 14.9469 dB	244.03 B / 15.0810 dB

The bit usage barely changes (+0.07 and +0.22 bytes per frame), while SNR improves by +0.127 dB at 24 kb/s and +0.102 dB at 48 kb/s.

That's exactly what this term is supposed to do: it doesn't really ask for more bits, it moves them toward the frames that can make better use of them.

At 96 kb/s it has essentially no effect, as expected since the factor is nearly zero there.

CBR is unchanged: 15.0721 dB at 96 kb/s, identical to before.

VBR status after this change

bitrate	pion	libopus	pion / libopus bytes	SNR gap
24000	52.21 B / 6.2959	53.50 B / 6.3764	97.6%	−0.081
48000	111.55 B / 9.8859	113.71 B / 10.0016	98.1%	−0.116
96000	241.81 B / 14.9469	244.03 B / 15.0810	99.1%	−0.134

pion is now spending between 1% and 2.4% fewer bytes than libopus and scores between 0.08 and 0.13 dB lower, roughly in proportion to the difference in bitrate.

At this point, the only remaining compute_vbr term is the tonality boost, which requires analysis.c. In this particular reference configuration it does not actually apply either: analysis->valid is 0, confirmed through instrumentation.

Reference issue

Part of #34.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.09%. Comparing base (e670259) to head (356b9aa).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #205      +/-   ##
==========================================
+ Coverage   93.08%   93.09%   +0.01%     
==========================================
  Files          58       58              
  Lines       10366    10389      +23     
==========================================
+ Hits         9649     9672      +23     
  Misses        505      505              
  Partials      212      212              

Flag	Coverage Δ
go	93.09% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.