add vmargs=-XX:-UseContainerSupport in config

[lxning]

Issue #, if available:
#99

Description of changes:
Apply the fixing in pytorch inference toolkit

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[davidthomas426]

I'll just note that this issue along with workarounds and fixes have shown up across different inference toolkits.

Here is a list of links showing that it's been a recurring problem.

Also, this fix may create other problems, as turning off container support means that the JVM does not respect Docker container memory limits.

We should make sure to address this uniformly across the inference toolkits and deep-learning-containers, while allowing users to easily customize without needing onerous workarounds such as using derived deep-learning-container images or even forking toolkit.

Links:

• SageMaker docs troubleshooting:
  • https://docs.aws.amazon.com/sagemaker/latest/dg/deploy-model-troubleshoot.html
• Other issues, across inference toolkits:
  • Base Inference Toolkit: JVM detect the CPU count as 1 when more CPUs are available for the container.  sagemaker-inference-toolkit#82
  • HuggingFace toolkit: Num CPU incorrect sagemaker-huggingface-inference-toolkit#3
• Other pull request from this repo:
  • Turn off UseContainerSupport: Add UseContainerSupport flag for model-server to see all available CPUs  #98
  • Same?: Expose all CPUs to TorchServe #100
• Other pull requests, other toolkits:
  • fix: Fixing issue #82 sagemaker-inference-toolkit#83
  • Allow user to change vmargs: Add environment variable VMARGS sagemaker-inference-toolkit#118
    • Note previous hack, requiring derived container (but not forked toolkit), using DockerFile change:
      • JVM detect the CPU count as 1 when more CPUs are available for the container.  sagemaker-inference-toolkit#82 (comment)
      • JVM detect the CPU count as 1 when more CPUs are available for the container. #99 (comment)
  • [huggingface_pytorch, huggingface_tensorflow][build] Huggingface inference DLC deep-learning-containers#1077
    • @vdantu mentioned this in PR code review, but it was ignored twice
• DLC repo still has vmargs with +UseContainerSupport in three different places:
  • https://github.com/aws/deep-learning-containers/blob/master/autogluon/build_artifacts/inference/config.properties
  • https://github.com/aws/deep-learning-containers/blob/master/mxnet/inference/docker/artifacts/config.properties
  • https://github.com/aws/deep-learning-containers/blob/master/pytorch/inference/docker/build_artifacts/config.properties

I still don't think this is an exhaustive list.

[chen3933]

The PR will be updated to allow customization of vmargs
Example :
aws/sagemaker-inference-toolkit@c01dde7

[rohithkrn]

python 3.7 tests failing at coverage report step. Failing to invoke coverage command. Works fine for python3.6
ERROR: InvocationError for command /codebuild/output/src309395522/src/github.com/aws/sagemaker-pytorch-inference-toolkit/.tox/py37/bin/coverage report --fail-under=90 --include '*sagemaker_pytorch_serving_container*' (exited with code 1)