Add support for detecting the number of available Neuron cores

[aws-taylor]

Before or while filing an issue please feel free to join our slack channel to get in touch with development team, ask questions, find out what's cooking and more!

Issue #, if available:

Description of changes:

Similar to GPUs, when running on AWS Inf1 instance types, automatically detect the number of available Neuron cores and set the number of workers accordingly.

Testing done:

Ran tests on inf1, logs correctly show 16 neuron cores on an inf1.6xlarge.

...
   Temp directory: /tmp
    Number of GPUs: 0
    Number of Neuron Cores: 16
    Number of CPUs: 24
    Max heap size: 10457 M
    Python executable: python
...

To run CI tests on your changes refer README.md

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[aws-taylor]

I believe the following block in `getDefaultWorkers()` will cause it to have the desired effect: ``` if (workers == 0) { workers = getNumberOfNeuronCores(); } ``` From: lxning ***@***.***> Reply-To: awslabs/multi-model-server ***@***.***> Date: Wednesday, August 24, 2022 at 12:15 PM To: awslabs/multi-model-server ***@***.***> Cc: "Goodhart, Taylor" ***@***.***>, Mention ***@***.***> Subject: Re: [awslabs/multi-model-server] Add support for detecting the number of available Neuron cores (PR #1002) @aws-taylor<https://github.com/aws-taylor> This PR make MMS get the number of Neuron Cores, but don't see it is used in MMS internally. Will you have another PR to address it? — Reply to this email directly, view it on GitHub<#1002 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ANYNIBQKSB272JDEDFIIR7DV2ZYBRANCNFSM57MQNFLA>. You are receiving this because you were mentioned.Message ID: ***@***.***>