[MNT] Intermediate test plan

[satvshr]

Metadata

• Reference Issue: fixes [MNT] Intermediate test plan #1614
• New Tests Added: No
• Documentation Updated: No

Details

• What does this PR implement/fix? Explain your changes.
  This PR implements the setting up of the v1 and v2 test servers using docker via localhost whenever pytest is run, eventually we plan to migrate fully over to these tests instead of using test.openml.org.
  • Add docker-compose.yml.
  • Add docker/update.sh from the server-api repository to setup the test server.
  • Changes to test.yml and conftest.py to run the containers on CI and locally.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.75%. Comparing base (d421b9e) to head (f169950).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1629      +/-   ##
==========================================
+ Coverage   52.04%   52.75%   +0.71%     
==========================================
  Files          36       36              
  Lines        4333     4333              
==========================================
+ Hits         2255     2286      +31     
+ Misses       2078     2047      -31     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[satvshr]

Docker fails to build for Windows, given that the image of database are built using Linux binaries. It works locally because Docker Desktop ends up using WSL in the background (as far as I can understand) to run these containers without errors. Currently I am skipping running docker for the windows env during CI runs only, and using this for tests that will end up using the docker servers:

is_windows_ci = (sys.platform == "win32" or os.name == "nt") and os.environ.get("CI") == "true"
skip_on_windows_ci = pytest.mark.skipif(
    is_windows_ci,
    reason="Skipping Docker tests on Windows CI (No WSL2 available)"
)

# Used in the form:
@skip_on_windows_ci
def test_database_connection():
    ...

Since we eventually plan on migrating all tests to these local servers, and we would like windows to run them too, we should think of another method to run these tests on windows CI.