Add data loading test helpers for Swift Testing and draft contributor information about writing tests #1362
Conversation
…ion for new tests
heckj
left a comment
heckj
left a comment
There was a problem hiding this comment.
(ignore if it's not useful - just saw the PR and stopped in to look)
| ### Testing in Xcode | ||
| ### Adding new tests | ||
| We recommend that you use [Swift Testing](https://developer.apple.com/documentation/testing) when you add new tests. |
There was a problem hiding this comment.
| We recommend that you use [Swift Testing](https://developer.apple.com/documentation/testing) when you add new tests. | |
| Please use [Swift Testing](https://developer.apple.com/documentation/testing) when you add new tests. |
Maybe make this stronger? I wasn't sure how heavily you wanted to lean into swift-testing.
There was a problem hiding this comment.
I'm not sure what amount of assertiveness (if that's the correct word) is right for this messaging. That's precisely what I hoped to get feedback on in this PR.
The team has previously talked about using Swift Testing for new tests and to slowly and incrementally transition existing tests.
| - Prefer small test inputs that ideally use a virtual file system for both reading and writing. | ||
| - Consider using parameterized tests if you're making the same verifications in multiple configurations or on multiple elements. | ||
| - Think about what information would be helpful to someone else who might debug that test case if it fails in the future. |
There was a problem hiding this comment.
I wasn't quite sure what it is that I should do with the thinking about this. I get the reason you're asking for the consideration, but am unsure of how to apply that consideration into a test. Any suggestions?
| We recommend that you use [Swift Testing](https://developer.apple.com/documentation/testing) when you add new tests. | ||
| Currently there are few existing tests to draw inspiration from, so here are a few recommendations: | ||
| - Prefer small test inputs that ideally use a virtual file system for both reading and writing. |
There was a problem hiding this comment.
Do we have any notes or internal docs on what functions to use, or examples to reference, that use the virtual file system setup? That would be great to point to, if we can.
There was a problem hiding this comment.
I can probably update each bullet to include an example that's both using Swift Testing and one that uses XCTest.
| - Think about what information would be helpful to someone else who might debug that test case if it fails in the future. | ||
| - Use `#require` rather that force unwrapping for behaviors that would change due to unexpected bugs in the code you're testing. | ||
| If you're updating an existing test case with additional logic, we appreciate it if you also modernize that test, but we don't expect it. |
There was a problem hiding this comment.
| If you're updating an existing test case with additional logic, we appreciate it if you also modernize that test, but we don't expect it. | |
| If you're updating an existing test case with additional logic, we appreciate if you also modernize that test while updating it, but we don't expect it. |
2 participants
Bug/issue #, if applicable:
Summary
This PR does 3 things to enable the team to start adopting Swift Testing:
let context = try await load(catalog: ...)etc.Dependencies
None
Testing
Nothing in particular. This is a test-only / documentation-only change.
Checklist
Make sure you check off the following items. If they cannot be completed, provide a reason.
AddedUpdated tests./bin/testscript and it succeeded