Skip to content

Conversation

Copy link

Copilot AI commented Nov 1, 2025

Summary

Successfully improved Behat output to include diffs on failed comparisons, making it much easier to debug test failures.

Completed Tasks

  • Add sebastian/diff v3 as a dependency (compatible with PHP 7.2+)
  • Create a helper method to generate diffs with proper type hints
  • Update check_string method to show diffs when comparisons fail
  • Update compare_tables method to show diffs
  • Update JSON comparison methods to show diffs with proper error checking
  • Update CSV comparison method to show diffs
  • Update YAML comparison method to show diffs
  • Add parameter and return type hints
  • Add error checking for JSON encoding/decoding
  • Fix pre-existing bug in CSV comparison variable usage
  • Fix typo in YAML docstring
  • Verify PHP syntax is correct
  • Run code review and address all feedback
  • Run security checks
  • Update to use sebastian/diff v3 API for PHP 7.2+ compatibility

Changes Summary

3 files changed, 78 insertions(+), 13 deletions(-)

  1. composer.json: Added sebastian/diff v3 as an explicit dependency
  2. src/Context/Support.php: Added generate_diff() helper method using v3 API
  3. src/Context/ThenStepDefinitions.php: Enhanced all comparison methods to show diffs

Example Output

When a comparison fails, users will now see:

--- Expected
+++ Actual
@@ -1,3 +1,3 @@
 Hello World
-This is a test
+This is different
 Third line

Quality Assurance

✅ No PHP syntax errors
✅ No security vulnerabilities
✅ All code review feedback addressed
✅ Proper error handling for JSON operations
✅ Full type hints added
✅ Compatible with PHP 7.2+

Ready for merge!

Original prompt

This section details on the original issue you should resolve

<issue_title>Improve Behat output to include diffs on failed comparisons</issue_title>
<issue_description>When Behat compares two elements and finds them to not match, it would be helpful for debugging purposes to have a diff available that shows the exact difference between the two elements, like in this example output:

image 2018-08-16 at 9 21 38 am

The SebastianBergmann\Diff\Differ class is available and can be used for that.</issue_description>

<agent_instructions>Behat/Behat#1532 might be useful as inspiration.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@swissspidy Looks like an upcoming Behat release will have better diffs out of the box, see https://github.com/Behat/Behat/pull/1532

Not in v3.16 yet, so have to wait for the next one.

Requires PHP 7.2+ though, see #228

Alternatively we can copy that logic of course.</comment_new>


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Improve Behat output to include diffs on failed comparisons Add unified diff output to failed Behat comparisons Nov 1, 2025
Copilot AI requested a review from swissspidy November 1, 2025 20:14
Copilot finished work on behalf of swissspidy November 1, 2025 20:14
@codecov
Copy link

codecov bot commented Nov 1, 2025

Codecov Report

❌ Patch coverage is 0% with 51 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/Context/ThenStepDefinitions.php 0.00% 34 Missing ⚠️
src/Context/Support.php 0.00% 17 Missing ⚠️

📢 Thoughts on this report? Let us know!

@swissspidy

This comment was marked as resolved.

This comment was marked as resolved.

Copilot finished work on behalf of swissspidy November 1, 2025 20:33
@swissspidy swissspidy marked this pull request as ready for review November 1, 2025 21:18
@swissspidy swissspidy requested a review from a team as a code owner November 1, 2025 21:18
@swissspidy

This comment was marked as resolved.

@swissspidy swissspidy marked this pull request as draft November 14, 2025 09:07
@swissspidy swissspidy marked this pull request as ready for review November 14, 2025 09:34
@swissspidy
Copy link
Member

btw this is what a failed test looks like with this change:

--- Failed steps:

001 Scenario: WP-CLI loads for your tests # features/testing.feature:3
      Then STDOUT should contain:         # features/testing.feature:7
        $ wp eval 'echo "Hello world.";'
        Hello world.
        cwd: /var/folders/44/hnwfklt963g865t5yf7yn5jw00lhhc/T/wp-cli-test-run--6916f4c4ba91a2.23538094/
        run time: 0.25022101402283
        exit status: 0

        --- Expected
        +++ Actual
        @@ @@
        -Hello world.fff
        +Hello world.
         (Exception)

@swissspidy swissspidy added this to the 5.0.4 milestone Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Behat output to include diffs on failed comparisons

2 participants