Explicitly specify minimum versions of dependencies

[astrofrog]

As mentioned by @bsipocz in #27 we should specify minimum versions of our dependencies wherever possible. At some point I can try and build the astropy docs with progressively older versions of the deps and see when it fails.