fastnumbers 5.0: fast conversion of iterables

[SethMMorton]

fastnumbers has made many changes since the 3.X branch. The 4.X branch introduced quite a few of these changes but was short-lived so we will present all of the changes from 3.X to 5.0 as if they all happened in 5.0.

Major public changes

• fastnumbers is now faster overall compared to the 3.X branch (see the "Major internal changes" sections for details now) - here are some specifics:
  • Integer conversions are 2X faster for most real-world integers.
  • Float conversions are 1X-5X (or more) faster (the more characters in the input string, the greater the speed-up).
  • Through the new map option or the try_array function, conversions of entire iterables of input are 2X faster than user-side iteration (on top of the above speed-ups).
• All the fast_* functions are deprecated, and now have equivalent functions beginning with try_*
  • The try_ prefix is more indicative of the intent of the function.
  • These functions introduce a super-set of functionality over the older fast_* functions, the two most prominent being:
    • The ability to convert an entire iterable at once with the map option.
    • The ability to provide error handling for type errors with the on_type_error option.
  • The deprecated functions will never be removed from the public API
• All the is* functions are deprecated, and now have equivalent functions beginning with check_*
  • The check_ prefix is more indicative of the intent of the function.
  • These functions introduce a super-set of functionality over the older is* functions, most prominently the new strict option for check_float that allows the ability to control whether input strings require a "." character or not.
  • The deprecated functions will never be removed from the public API
• The map option to the try_* functions support returning an iterator (like the built-in map), or returning a list directly.
  • When converting a collection of data, you will always see better performance using the map option over iterating in Python-land.
• The try_array function is used to convert an iterable directly into a 1-D NumPy array.
  • It can also accept an existing array into which to place the output.
• The default for allow_underscores is now False instead of True for all of the non-deprecated functions as a means to reduce user surprise and also slightly improve performance.

Major internal changes

• Complete re-write from C99 to C++17 and a significantly more maintainable code base.
  • Use of exceptions, std::optional, std::variant/std::visit and RAII have made error handling and reference counting much less error-prone - the likelihood of segfaults, SystemErrors, and memory leaks is far lower with the new code.
  • The new object-oriented paradigm makes adding new features far easier than the highly-coupled C code the came before.
• All public functions use the new vectorcall protocol (via METH_FASTCALL) which means a large reduction of "function call overhead" compared to the 3.X branch.
• Float parsing now uses fast_float::from_chars to do all floating point parsing. The ways in which this introduces speed-ups is numerous:
  • The actual string-to-float parsing is very fast, even compared to the (apparently very) "naïve" algorithm that existed in the 3.X branch.
  • Because from_chars is correct-and-accurate for all input, fastnumbers never has to fall back on Python's conversion function (which is not usually fast).
  • Because there no longer is a cut-off from using a "naïve" algorithm vs. more robust, the input no longer needs to be pre-scanned to detect which algortithm to use.
    • This also means that a source of platform-to-platform inconsistency/error has been eliminated.
• Lower-level parsing code has been micro-optimized by moving to strategies involving pre-computed tables and bitshifts.