cm4mlperf November 2024

[arjunsuresh]

What's Changed

• fixed several URLs (all tests passed) by @gfursin in fixed several URLs (all tests passed) #342
• Merge from go by @arjunsuresh in Merge from go #345
• Changes for supporting model and dataset download to host - Mixtral by @anandhu-eng in Changes for supporting model and dataset download to host - Mixtral #346
• Support custom git clone branch in docker by @anandhu-eng in Support custom git clone branch in docker #343
• Merge from go by @arjunsuresh in Merge from go #347
• Merge from go, fixes Make push MLPerf results to github repository script Windows compatible #337 by @arjunsuresh in Merge from go, fixes #337 #348
• Merge from go by @arjunsuresh in Merge from go #349
• Includes cuda version to run suffix by @anandhu-eng in Includes cuda version to run suffix #354
• Fixes for const in script by @arjunsuresh in Fixes for const in script  #355
• Fix docker image naming SCC24, extended CM script tests by @arjunsuresh in Fix docker image naming SCC24, extended CM script tests #356
• Fixes for MLPerf Inference Github Actions by @arjunsuresh in Fixes for MLPerf Inference Github Actions #362
• Fix typo in gh action by @arjunsuresh in Fix typo in gh action #363
• Fix CUDA num_devices by @arjunsuresh in Fix CUDA num_devices #365
• Support cleaning of Nvidia SDXL model by @arjunsuresh in Support cleaning of Nvidia SDXL model #366
• Improvements to Nvidia MLPerf interface by @arjunsuresh in Improvements to Nvidia MLPerf interface #367
• Fix typo by @arjunsuresh in Fix typo #368
• Fixes to pull changes for Nvidia implementation by @arjunsuresh in Fixes to pull changes for Nvidia implementation #369
• Improvements to MLPerf inference final report generation by @arjunsuresh in Improvements to MLPerf inference final report generation #371
• Fix typo by @arjunsuresh in Fix typo #372
• Support get-platform-details for mlperf-inference by @arjunsuresh in Support get-platform-details for mlperf-inference #373
• Support system_info.txt in MLPerf inference submission generation by @arjunsuresh in Support system_info.txt in MLPerf inference submission generation #374
• Cleanups for mlperf inference get-platform-details by @arjunsuresh in Cleanups for mlperf inference get-platform-details #375
• Improvements to get-platform-details by @arjunsuresh in Improvements to get-platform-details #376
• Improvements for reproducing AMD implementation by @anandhu-eng in Improvements for reproducing AMD implementation #379
• Improvements to amd LLAMA2 70B command generation - Added server scenario by @anandhu-eng in Improvements to amd LLAMA2 70B command generation - Added server scenario #383
• Build wheels and release them into PYPI by @anandhu-eng in Build wheels and release them into PYPI #385
• Merge from main by @arjunsuresh in Merge from main #386
• Update build_wheels.yml by @arjunsuresh in Update build_wheels.yml #387
• Merge from main by @arjunsuresh in Merge from main #388
• Update build_wheels.yml by @arjunsuresh in Update build_wheels.yml #389
• Merge from main by @arjunsuresh in Merge from main #390
• Utilise command generated from preprocess function in batch file by @anandhu-eng in Utilise command generated from preprocess function in batch file #393
• Do not pass mlperf_conf for inference-src >= 4.1.1 by @arjunsuresh in Do not pass mlperf_conf for inference-src >= 4.1.1 #404
• Fix version check for mlperf-inference-src by @arjunsuresh in Fix version check for mlperf-inference-src #405
• Added no-compilation-warning variation for loadgen by @arjunsuresh in Added no-compilation-warning variation for loadgen #406
• Fix typo by @arjunsuresh in Fix typo #407
• VERSION -> VERSION.txt in mlperf-inference-loadgen by @arjunsuresh in VERSION -> VERSION.txt in mlperf-inference-loadgen #408
• Fix typo by @arjunsuresh in Fix typo #409
• Support 8G Nvidia GPUs for MLPerf Inference by @arjunsuresh in Support 8G Nvidia GPUs for MLPerf Inference #411
• Fix bug on benchmark-program exit check by @arjunsuresh in Fix bug on benchmark-program exit check #412
• Improve the benchmark-program-mlperf run command by @arjunsuresh in Improve the benchmark-program-mlperf run command #413
• CM4MLOps snapshot with MLPerf inference: 20241024 by @gfursin in CM4MLOps snapshot with MLPerf inference: 20241024 #415
• fix batch size duplication issue by @anandhu-eng in fix batch size duplication issue #416
• Update cm repo branch - docker by @anandhu-eng in Update cm repo branch - docker #422
• Fixes for Nvidia MLPerf inference SS and MS by @arjunsuresh in Fixes for Nvidia MLPerf inference SS and MS #423
• Improvements for Nvidia MLPerf inference by @arjunsuresh in Improvements for Nvidia MLPerf inference #428
• added compressed_tools module by @anandhu-eng in added compressed_tools module #430
• Updated logic for mounting non cache folder by @anandhu-eng in Updated logic for mounting non cache folder #427
• Fixes for latest MLPerf inference submission checker changes by @arjunsuresh in Fixes for latest MLPerf inference submission checker changes #431
• Fixes for Latest MLPerf inference changes by @arjunsuresh in Fixes for Latest MLPerf inference changes #432
• Fixes for latest MLPerf inference changes by @arjunsuresh in Fixes for latest MLPerf inference changes #433
• Submission generation fixes by @anandhu-eng in Submission generation fixes #424
• Support custom path for saving platform details by @anandhu-eng in Support custom path for saving platform details #418
• Anandhu eng patch 1 by @arjunsuresh in Anandhu eng patch 1 #434
• Add getting started to cm4mlops docs by @anandhu-eng in Add getting started to cm4mlops docs #435
• Merge from Mlperf inference by @anandhu-eng in Merge from Mlperf inference #436
• Fixes for docker detached mode by @arjunsuresh in Fixes for docker detached mode #438
• Fixes for get-platform-details by @arjunsuresh in Fixes for get-platform-details #441
• Testing CM Test automation by @arjunsuresh in Testing CM Test automation #442
• Added github action for individual CM tests by @arjunsuresh in Added github action for individual CM tests #443
• Fix Individual CM script test by @arjunsuresh in Fix Individual CM script test #445
• Fix individual CM script test by @arjunsuresh in Fix individual CM script test #446
• Fix gh action for individual CM sript tests by @arjunsuresh in Fix gh action for individual CM sript tests #447
• Initial PR - gh actions for submission generation for non CM based benchmarks by @anandhu-eng in Initial PR - gh actions for submission generation for non CM based benchmarks #440
• Fix GH action for individual CM script testing by @arjunsuresh in Fix GH action for individual CM script testing #449
• Enable docker run for individual CM script tests by @arjunsuresh in Enable docker run for individual CM script tests #450
• Fixes huggingface downloader by @arjunsuresh in Fixes huggingface downloader #452
• capture framework version from cm_sut_info.json by @anandhu-eng in capture framework version from cm_sut_info.json #451
• Sync: Mlperf inference by @arjunsuresh in Sync: Mlperf inference #444
• Support docker_base_image and docker_cm_repo for CM tests by @arjunsuresh in Support docker_base_image and docker_cm_repo for CM tests #453
• Improved docker meta for cm test script by @arjunsuresh in Improved docker meta for cm test script #454
• Support test_input_index and test_input_id to customise CM test script by @arjunsuresh in Support test_input_index and test_input_id to customise CM test script #458
• Sync Mlperf inference by @arjunsuresh in Sync Mlperf inference #457
• Added test case 7 by @anandhu-eng in Added test case 7 #448
• Improvements to version-detect in get-generic-sys-util by @arjunsuresh in Improvements to version-detect in get-generic-sys-util #460
• Sync <- Mlperf inference by @arjunsuresh in Sync <- Mlperf inference #461
• Update VERSION | Test pypi release by @arjunsuresh in Update VERSION | Test pypi release #462
• Update setup.py by @arjunsuresh in Update setup.py #463
• Use pkg-config deps for get-generic-sys-util by @arjunsuresh in Use pkg-config deps for get-generic-sys-util #464
• Fix pstree version detect on macos by @arjunsuresh in Fix pstree version detect on macos #466
• Fix Nvidia mlperf inference retinanet | onnx version by @arjunsuresh in Fix Nvidia mlperf inference retinanet | onnx version #468
• Support detached mode for nvidia-mlperf-inference-gptj by @arjunsuresh in Support detached mode for nvidia-mlperf-inference-gptj #469
• Cleanups for get-generic-sys-util by @arjunsuresh in Cleanups for get-generic-sys-util #470
• Fix tmp-run-env.out name by @arjunsuresh in Fix tmp-run-env.out name #471
• Add Version RE for g++-11 by @arjunsuresh in Add Version RE for g++-11 #472
• Use berkeley link for imagenet-aux by default by @arjunsuresh in Use berkeley link for imagenet-aux by default #473
• Code cleanup by @anandhu-eng in Code cleanup #475
• Fixes for Sdxl MLPerf inference by @arjunsuresh in Fixes for Sdxl MLPerf inference  #481
• Added google dns to mlperf-inference docker by @arjunsuresh in Added google dns to mlperf-inference docker #484
• enable submission generation gh action test globally by @anandhu-eng in enable submission generation gh action test globally #487
• Update test-scc24-sdxl.yaml by @arjunsuresh in Update test-scc24-sdxl.yaml #488
• Enables docker run in inference submission generation by @anandhu-eng in Enables docker run in inference submission generation #486
• Added docker detatched option by @anandhu-eng in Added docker detatched option #477
• Fixes Add version detect commands and version check regular expressions for all packages in get-generic-sys-util #261 partially by @Oseltamivir in Fixes #261 partially #426
• Sync Mlperf inference by @arjunsuresh in Sync Mlperf inference #490
• Enabled docker run - gh action submission generation by @anandhu-eng in Enabled docker run - gh action submission generation #489
• Fixes for MLPerf inference, intel conda URL by @arjunsuresh in Fixes for MLPerf inference, intel conda URL #491
• Dont use '-dt' for Nvidia ml-model-gptj by @arjunsuresh in Dont use '-dt' for Nvidia ml-model-gptj #492
• Update starting weights filename for SDXL MLPerf inference by @arjunsuresh in Update starting weights filename for SDXL MLPerf inference #494
• Implements Add an option to use host CM_REPO as docker_cm_repo #455: Copy local repo to docker instead of git clone by @Oseltamivir in Implements #455: Copy local repo to docker instead of git clone #467
• Fixes for Nvidia MLPerf inference gptj,sdxl by @arjunsuresh in Fixes for Nvidia MLPerf inference gptj,sdxl #495
• Cleanups to MLPerf inference preprocess script by @arjunsuresh in Cleanups to MLPerf inference preprocess script #496
• Support sample-ids for coco2014 accuracy script by @arjunsuresh in Support sample-ids for coco2014 accuracy script #497
• Support download to host - amd llama2 by @anandhu-eng in Support download to host - amd llama2 #480
• Sync from GO by @arjunsuresh in Sync from GO #498
• Added a retry for git clone failure by @arjunsuresh in Added a retry for git clone failure #499
• Use custom version for dev branch of inference-src by @arjunsuresh in Use custom version for dev branch of inference-src #500
• Fix path error shown to user by @anandhu-eng in Fix path error shown to user #502
• Fixes for the MLPerf inference nightly test failures by @arjunsuresh in Fixes for the MLPerf inference nightly test failures #506
• pip install cm4mlops - handle systems where sudo is absent by @anandhu-eng in pip install cm4mlops - handle systems where sudo is absent #504
• Sync <- Mlperf inference by @arjunsuresh in Sync <- Mlperf inference #507
• logic update for detect sudo by @anandhu-eng in logic update for detect sudo #508
• Fix Nvidia MLPerf inference gptj model name suffix by @arjunsuresh in Fix Nvidia MLPerf inference gptj model name suffix #509
• Skip sys-utils-install when no sudo is available by @arjunsuresh in Skip sys-utils-install when no sudo is available #512
• Fixes against issue: No SUDO access but INFO:root: * cm run script "detect sudo" #510 by @anandhu-eng in Fixes against issue: #510 #514
• Support for privileged mode in docker by @anandhu-eng in Support for privileged mode in docker #511
• Allow default_version update inside variations by @arjunsuresh in Allow default_version update inside variations #515
• MLPerf inference changes by @arjunsuresh in MLPerf inference changes #518
• Removed dev branch for SDXL by @arjunsuresh in Removed dev branch for SDXL #519
• update_env_if_env -> update_meta_if_env by @arjunsuresh in update_env_if_env -> update_meta_if_env #520
• Automatically generate system_meta.json if not found by @anandhu-eng in Automatically generate system_meta.json if not found #523
• Added test case 8 by @anandhu-eng in Added test case 8 #524
• Add cuda information to system meta by @anandhu-eng in Add cuda information to system meta #526
• Improvements to github actions by @arjunsuresh in Improvements to github actions #527
• Fix README generation in MLPerf inference submission by @arjunsuresh in Fix README generation in MLPerf inference submission #529
• Avoid creation of empty model_mapping.json by @arjunsuresh in Avoid creation of empty model_mapping.json #530
• Enhancements to Docker for Multi-User Setups by @arjunsuresh in Enhancements to Docker for Multi-User Setups #539
• Enable update mlperf inference measurements.json file from existing file by @arjunsuresh in Enable update mlperf inference measurements.json file from existing file #540
• Github action updates by @arjunsuresh in Github action updates #548
• Enable creating test dataset from main mixtral dataset by @anandhu-eng in Enable creating test dataset from main mixtral dataset #550
• Added RTX4090x1 configs by @arjunsuresh in Added RTX4090x1 configs #551
• change in variation name by @anandhu-eng in change in variation name #552
• Fix skip model for MLPerf inference llama2 and mixtral by @arjunsuresh in Fix skip model for MLPerf inference llama2 and mixtral  #553
• Support min_query_count and max_query_count for mlperf inference by @arjunsuresh in Support min_query_count and max_query_count for mlperf inference #554
• Remove TEST05 for MLPerf inference by @arjunsuresh in Remove TEST05 for MLPerf inference #555
• Fix CM_ML_MODEL_PATH export for mixtral by @arjunsuresh in Fix CM_ML_MODEL_PATH export for mixtral #556
• Fix the saving of console logs in benchmark-program - fixes MLPerf console logs are coming empty #533 by @arjunsuresh in Fix the saving of console logs in benchmark-program - fixes #533 #558
• Fix 3dunet SS latency in configs by @arjunsuresh in Fix 3dunet SS latency in configs #559
• Added sympy dependency for nvidia mlperf inference gptj by @arjunsuresh in Added sympy dependency for nvidia mlperf inference gptj #560
• Added new docker image name for Nvidia MLPerf inference LLM models by @arjunsuresh in Added new docker image name for Nvidia MLPerf inference LLM models #562
• Build TRTLLM for Nvidia MLPerf LLM models by @arjunsuresh in Build TRTLLM for Nvidia MLPerf LLM models #563
• Update default-config.yaml by @arjunsuresh in Update default-config.yaml #564
• fix get_cudnn and get_tensorrt not detecting multi-digit version numbers correctly by @Submandarine in fix get_cudnn and get_tensorrt not detecting multi-digit version numbers correctly #565
• Fixes MLPerf sdxl "not supported between instances of 'str' and 'int'" #566, type error for query counts by @arjunsuresh in Fixes #566, type error for query counts #567
• add gnn dataset download script by @anandhu-eng in add gnn dataset download script #570
• rename mixtral dataset download script by @anandhu-eng in rename mixtral dataset download script #569
• enable user to submit a result for both closed and open by @anandhu-eng in enable user to submit a result for both closed and open #535
• use the default sut folder name supplied if cm-sut-json is not there by @anandhu-eng in use the default sut folder name supplied if cm-sut-json is not there #537
• Added dependency graph for mlperf-inference runs by @arjunsuresh in Added dependency graph for mlperf-inference runs #573
• Add get-rocm-devices for AMD GPUs by @Henryfzh in Add get-rocm-devices for AMD GPUs #544
• Add aarch64-Compatible Base Image for MLPerf Inference & Fix TensorRT Version Matching by @Leonard226 in Add aarch64-Compatible Base Image for MLPerf Inference & Fix TensorRT Version Matching  #478
• Sync <- Mlperf inference by @arjunsuresh in Sync <- Mlperf inference #557
• additional tag for dataset sampling - mixtral by @anandhu-eng in additional tag for dataset sampling - mixtral #576
• match test query count with sampled data by @anandhu-eng in match test query count with sampled data #577
• Update nvidia mlperf inference sdxl configs by @arjunsuresh in Update nvidia mlperf inference sdxl configs #578
• Improve docker detached mode error capture by @arjunsuresh in Improve docker detached mode error capture #579
• Use absolute paths for docker mounts by @arjunsuresh in Use absolute paths for docker mounts #580
• Issue Rearrange files - Submission generation #542 - folder structure rearrange - submission generation by @anandhu-eng in Issue #542 - folder structure rearrange - submission generation #574
• Fixes for SDXL accuracy run by @arjunsuresh in Fixes for SDXL accuracy run #582
• Fix a typo in benchmark-program by @arjunsuresh in Fix a typo in benchmark-program #583
• _cm.json -> _cm.yaml by @arjunsuresh in _cm.json -> _cm.yaml #584
• Sync <- Mlperf inference for November release by @arjunsuresh in Sync <- Mlperf inference for November release #585
• Support for platform details in submission generation by @anandhu-eng in Support for platform details in submission generation #581
• Add remaining gh actions for cm submission generation by @anandhu-eng in Add remaining gh actions for cm submission generation #587
• Sync main <- Mlperf inference for November 2024 release by @arjunsuresh in Sync main <- Mlperf inference for November 2024 release #588

New Contributors

• @Oseltamivir made their first contribution in Fixes #261 partially #426
• @Submandarine made their first contribution in fix get_cudnn and get_tensorrt not detecting multi-digit version numbers correctly #565
• @Henryfzh made their first contribution in Add get-rocm-devices for AMD GPUs #544
• @Leonard226 made their first contribution in Add aarch64-Compatible Base Image for MLPerf Inference & Fix TensorRT Version Matching  #478

Full Changelog: r20241005a...cm4mlperf-v2.3.5

---

This discussion was created from the release cm4mlperf November 2024.