sklearn ships with several utility functions (e.g. check_estimator) for checking that externally developed estimators fit sklearn's expected interface. It would be good to try to integrate these into our test suite, fixing issues as they arise (and they definitely will arise). See https://scikit-learn.org/stable/developers/develop.html for more.

Also related: #6105 (one of the main compatility issues we'll have to resolve, among others).