Cycle 5: Reliable performance benchmarking for the Astropy project

[astrofrog]

This proposal was solicited by the Strategic Planning Committee as it was identified that no existing proposal addressed the roadmap item about performance benchmark reporting (hence why this was opened after the deadline).

Note that this is separate from the NumFOCUS approach which @pllim mentioned in https://groups.google.com/g/astropy-dev/c/Ns2jj7qtW-s - the approach in the current proposal has a lower monthly cost and includes developer time to make it happen.

[pllim]

I would prefer this approach over NumFOCUS AWS if we can pull it off.

But I do have concern on the bus factor here. Will we be back to square one when Aperio decide to stop maintaining this server, like when the funding runs out, or you all win lotteries and retire?

[astrofrog]

I think the best way to guard against this is to have the astropy project pay directly for the server once we have identified which one to get and for us to openly document the server set up or at least if not completely open, have it in a repo that at the very least the CoCo have access to. Then anyone else can take over maintenance of the server. We plan to have the server side set up be as simple as possible, with all the important config being in eg the core package repo.