[LoadGen] Simplify installation with a pip one-liner

[wangshangsam]

This PR essentially:

1. Move the CFLAGS="-std=c++14 -O3" into setup.py as extra_compile_args.
2. Move the absl-py and numpy dependencies into requirements.txt, then enable setup.py to read from requirements.txt.

Therefore, this PR enables one-liner installation of LoadGen from the top of the master branch without the need to cloning the repo first:

pip install git+https://github.com/mlcommons/inference.git#subdirectory=loadgen

The motivation I had is that, in the pyproject.toml of the specific benchmark (e.g., multimodal/vl2l), now I can just add mlcommons_loadgen @ git+https://github.com/mlcommons/inference.git#subdirectory=loadgen as a dependency, then I don't need a separate step to build/install the LoadGen first before installing the benchmark, thereby achieving an one-liner installation of the benchmark too:

pip install git+https://github.com/mlcommons/inference.git#subdirectory=multimodal/vl2l

cc @johncalesp @nvzhihanj

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[wangshangsam]

@pgmpablo157321 @arjunsuresh @mrmhodak @hanyunfan I wondering if you could review this PR and see if you are open to such a change in the LoadGen's installation process?

[arjunsuresh]

@wangshangsam The changes look fine to me but pip install mlcommons_loadgen is easier right?

[wangshangsam]

@wangshangsam The changes look fine to me but pip install mlcommons_loadgen is easier right?

The

NOTE: Take into account that we only update the published wheels after an official release, they may not include the latest changes.

in https://github.com/mlcommons/inference/blob/master/loadgen/README_BUILD.md made me think that the package in pypi is not updated as frequently, but then I just took a look at the commit histroy again, and it does seem that VERSION.txt gets a bump up immediately and a new wheel gets published whenever there's some change to the loadgen code. Is this being done manually or automated in the CI?
If the package in pypi will always automatically contain the latest LoadGen (from the master branch), then yes, I agree that pip install mlcommons_loadgen would be easier.

[arjunsuresh]

Yes. We need to update the readme. The pypi release is done automatically via GitHub action. @anandhu-eng can you please update the loadgen readme?

https://github.com/mlcommons/inference/blob/master/.github/workflows/build_wheels.yml

[anandhu-eng]

Hey everyone, I have raised a PR here: #2398