Comparers control how a received result is compared to its verified file. They apply to both text and binary targets.
A comparer is helpful when a result has changed, but not enough to fail verification. For example when rendering images/forms on different operating systems, or when part of a text snapshot legitimately varies between environments.
There are two kinds, selected by the type of the target being compared:
- String comparers (
StringCompare) for text targets. - Stream comparers (
StreamCompare) for binary targets.
Note that a comparer only runs when the received and verified content are not already identical. A matching result never reaches the comparer, so the passing path keeps the fast comparison.
In both cases the returned CompareResult.NotEqual takes an optional message that will be rendered in the resulting text displayed to the user on test failure.
Comparers change only the equality check. The received and verified content is left as-is, so relaxing a comparison does not churn existing verified files, unlike scrubbing or ignoring members.
Comparers are matched on the extension of the target being compared, and the first match wins:
- An instance comparer registered for that extension:
settings.UseStringComparer(compare, "json"). - An instance comparer registered with no extension, which applies to all extensions:
settings.UseStringComparer(compare). - A static comparer registered for that extension:
VerifierSettings.RegisterStringComparer("json", compare). - For text targets only, the static default:
VerifierSettings.SetDefaultStringComparer(compare).
So a global comparer can be registered once and overridden for specific tests.
The extension match is exact, and an extension with no match is a silent no-op rather than an error. Care is needed where a setting changes the extension of a snapshot: UseStrictJson renames text snapshots from txt to json, so a comparer registered against txt stops running once it is enabled. SetDefaultStringComparer avoids pinning an extension.
Static comparers must be registered before the first verification runs, so a ModuleInitializer is the usual location.
For sample purposes only case differences will be ignored:
static Task<CompareResult> CompareIgnoringCase(
string received,
string verified,
IReadOnlyDictionary<string, object> context)
{
if (string.Equals(received, verified, StringComparison.OrdinalIgnoreCase))
{
return Task.FromResult(CompareResult.Equal);
}
var result = CompareResult.NotEqual("Differed by more than case");
return Task.FromResult(result);
}[Fact]
public Task StringComparerInstance()
{
var settings = new VerifySettings();
settings.UseStringComparer(CompareIgnoringCase, "txt");
return Verify("TheText", settings);
}
[Fact]
public Task StringComparerInstanceFluent() =>
Verify("TheText")
.UseStringComparer(CompareIgnoringCase, "txt");VerifierSettings.RegisterStringComparer(
extension: "txt",
compare: CompareIgnoringCase);
await Verify("TheText");Or as the default for all text extensions:
VerifierSettings.SetDefaultStringComparer(CompareIgnoringCase);
await Verify("TheText");Information can be passed from a test to its comparer via Context.
For samples purposes only the image sizes will be compared:
static Task<CompareResult> CompareImages(
Stream received,
Stream verified,
IReadOnlyDictionary<string, object> context)
{
// Fake comparison
if (received.Length == verified.Length)
{
return Task.FromResult(CompareResult.Equal);
}
var result = CompareResult.NotEqual();
return Task.FromResult(result);
}If an input is split into multiple files, and a text file fails, then all subsequent binary comparisons will revert to the default comparison.
[Fact]
public Task InstanceComparer()
{
var settings = new VerifySettings();
settings.UseStreamComparer(CompareImages, "png");
return VerifyFile("sample.png", settings);
}
[Fact]
public Task InstanceComparerFluent() =>
VerifyFile("sample.png")
.UseStreamComparer(CompareImages, "png");VerifierSettings.RegisterStreamComparer(
extension: "png",
compare: CompareImages);
await VerifyFile("TheImage.png");When a converter splits an input into multiple targets, for example a source document plus derived outputs such as rendered images or extracted text, a lenient comparer on a derived target can mask a real change in the source. Setting BypassComparersForSubsequentOnDifference on the source target ensures that, when the source differs from its verified file, all subsequent targets skip their registered comparers and fall back to exact comparison:
// A converter that emits a canonical source document alongside derived targets (eg rendered pages).
// The source is flagged so that, when it differs, the derived targets skip their (potentially lenient)
// comparers and fall back to exact comparison, ensuring a real change in the source is never masked.
public static ConversionResult ConvertDocument(Stream document, IReadOnlyDictionary<string, object> context)
{
Target[] targets =
[
new("docx", document)
{
BypassComparersForSubsequentOnDifference = true
},
new("png", RenderPage(document))
];
return new(info: null, targets);
}The flag must be set on the source target, and that target must precede the derived targets in the conversion result.
A converter that says which of its targets is the source, and which were derived from it, needs no flag. Verify compares the source first, and when it differs compares its derived targets exactly. That applies only to the targets derived from that source, where the flag applies to every target after it. See Source and derived targets.
// Large enough that a snapshot of any usual size is a couple of reads, and small enough
// to stay under the large object heap threshold and inside the array pool's buckets.
const int bufferSize = 64 * 1024;
public static async Task<CompareResult> AreEqual(Stream stream1, Stream stream2)
{
EnsureAtStart(stream1);
EnsureAtStart(stream2);
var buffer1 = ArrayPool<byte>.Shared.Rent(bufferSize);
var buffer2 = ArrayPool<byte>.Shared.Rent(bufferSize);
try
{
while (true)
{
// The two streams are independent, so the reads overlap instead of running
// one after the other. Both files are read in full whenever they match,
// which is the usual case on a passing run. WhenAll rather than awaiting in
// sequence, so a failure on one side cannot leave the other unobserved.
var read1 = ReadBufferAsync(stream1, buffer1);
var read2 = ReadBufferAsync(stream2, buffer2);
await Task.WhenAll(read1, read2);
var count1 = read1.Result;
var count2 = read2.Result;
// Callers do not always guarantee the streams are the same length
// (e.g. a non-seekable received stream), so a length difference must
// be treated as not-equal instead of a short-circuit to equal.
if (count1 != count2)
{
return CompareResult.NotEqual();
}
if (count1 == 0)
{
return CompareResult.Equal;
}
// Compare exactly the bytes read (a rented buffer's trailing bytes
// are arbitrary, so they must not be included).
if (!buffer1.AsSpan(0, count1).SequenceEqual(buffer2.AsSpan(0, count1)))
{
return CompareResult.NotEqual();
}
}
}
finally
{
ArrayPool<byte>.Shared.Return(buffer1);
ArrayPool<byte>.Shared.Return(buffer2);
}
}
static void EnsureAtStart(Stream stream)
{
if (stream.CanSeek &&
stream.Position != 0)
{
throw new("Expected stream to be at position 0.");
}
}
static async Task<int> ReadBufferAsync(Stream stream, byte[] buffer)
{
// Read up to bufferSize (not buffer.Length): a rented buffer may be
// larger, and both streams must be read in equal-sized chunks so the
// length comparison above stays aligned.
var bytesRead = 0;
while (bytesRead < bufferSize)
{
#if NET6_0_OR_GREATER
// Memory overload: the byte[] overload wraps every call in a Task on a
// FileStream opened for async IO.
var read = await stream.ReadAsync(buffer.AsMemory(bytesRead, bufferSize - bytesRead));
#else
var read = await stream.ReadAsync(buffer, bytesRead, bufferSize - bytesRead);
#endif
if (read == 0)
{
// Reached end of stream.
return bytesRead;
}
bytesRead += read;
}
return bytesRead;
}Verify includes a built-in Structural Similarity Index (SSIM) comparer for PNG files. It is opt-in and, when enabled, replaces the default byte-for-byte comparison for the .png extension.
This is useful when rendered images differ slightly between runs (e.g. anti-aliasing, font hinting, platform-specific rasterization) but are perceptually identical.
public static class ModuleInitializer
{
[ModuleInitializer]
public static void Init() =>
VerifierSettings.UseSsimForPng();
}The default threshold is 0.98. SSIM scores range from 0 (completely different) to 1 (identical). A custom threshold can be supplied:
public static class ModuleInitializer
{
[ModuleInitializer]
public static void Init() =>
VerifierSettings.UseSsimForPng(threshold: 0.995);
}SSIM can also be enabled per-verification, via an instance of VerifySettings or fluently on a SettingsTask:
[Fact]
public Task InstanceSsimForPng()
{
var settings = new VerifySettings();
settings.UseSsimForPng();
return VerifyFile("sample.png", settings);
}
[Fact]
public Task InstanceSsimForPngFluent() =>
VerifyFile("sample.png")
.UseSsimForPng(threshold: 0.995);Dimension mismatches between the received and verified images are always reported as not equal, regardless of threshold.
The SSIM score can also be computed directly, outside of a verification. This is useful, for example, to report a similarity metric per rendered page in a custom report:
public static double Score(Stream received, Stream verified) =>
Ssim.Compare(received, verified);Ssim.Compare accepts two PNG streams, two PNG byte arrays, or two PngImage instances decoded via PngDecoder.Decode. Scores range from 0 (completely different) to 1 (identical). Comparing images of different dimensions throws an ArgumentException.
To treat a dimension mismatch as a distinct state instead of an exception, decode with PngDecoder.Decode and check dimensions before comparing:
public static double? ScoreIfSameSize(Stream received, Stream verified)
{
var receivedImage = PngDecoder.Decode(received);
var verifiedImage = PngDecoder.Decode(verified);
if (receivedImage.Width != verifiedImage.Width ||
receivedImage.Height != verifiedImage.Height)
{
return null;
}
return Ssim.Compare(receivedImage, verifiedImage);
}PngDecoder supports the same PNG subset as the comparer (see below).
The bundled decoder targets the common subset of PNGs produced by test scenarios:
- 8-bit bit depth
- Color types: grayscale, RGB, RGBA, grayscale+alpha, and paletted (with optional
tRNStransparency) - Non-interlaced images
Unsupported variants (16-bit, Adam7 interlacing) cause the decoder to throw. For scenarios that require full PNG support, use one of the below comparers.
- Verify.AngleSharp.Diffing: Comparison of html files via AngleSharp.Diffing.
- Verify.ICSharpCode.Decompiler: Comparison of assemblies and types via ICSharpCode.Decompiler.
- Verify.ImageHash: Comparison of images via ImageHash.
- Verify.ImageMagick: Verification and comparison of images via Magick.NET.
- Verify.ImageSharp.Compare: Verification and comparison of images via Codeuctivity.ImageSharp.Compare.
- Verify.Phash: Comparison of images via Phash.
- Verify.Quibble: Comparison of objects via Quibble.
- YellowDogMan.Verify.ssimulacra2: Verification and comparison of images via ssimulacra2.