Merge branch 'develop' into develop

Author: hgreebe

CHANGELOG.md

@@ -17,6 +17,20 @@ CHANGELOG
 - The build-image command now deploys a global role that is used to automatically delete the build-image stack after images either succeed or fail the build. 
   The role is meant to exists even after the stack has been deleted. This is to prevent build-image stack deletion failures, reported in https://github.com/aws/aws-parallelcluster/issues/5914
 - Add the configuration parameter `HeadNode/SharedStorageEfsSettings/Encrypted` to enable encryption on the EFS file system used for the head node internal shared storage.
+- Upgrade Slurm to version 24.11.6 (from 24.05.8).
+- Upgrade EFA installer to 1.42.0 (from 1.41.0).
+  - Efa-driver: efa-2.15.3-1
+  - Efa-config: efa-config-1.18-1
+  - Efa-profile: efa-profile-1.7-1
+  - Libfabric-aws: libfabric-aws-2.1.0-3
+  - Rdma-core: rdma-core-57.0-1
+  - Open MPI: openmpi40-aws-4.1.7-2 and openmpi50-aws-5.0.6-11
+- Upgrade Cinc Client to version to 18.4.12 from 18.2.7.
+- Upgrade NVIDIA driver to version 570.172.08 (from 570.86.15) for all OSs except AL2.
+- Upgrade CUDA Toolkit to version 12.8.1 (from 12.8.0) for all OSs except AL2.
+- Upgrade DCGM to version 4.2.3 (from 3.3.6) for all OSs except AL2.
+- Upgrade Python to 3.12.11 (from 3.12.8) for all OSs except AL2.
+- Upgrade Intel MPI Library to 2021.16.0 (from 2021.13.1).
 
 **BUG FIXES**
 - Fix an issue where Security Group validation failed when a rule contained both IPv4 ranges (IpRanges) and security group references (UserIdGroupPairs).

cli/src/pcluster/resources/imagebuilder/parallelcluster.yaml

@@ -5,7 +5,7 @@ schemaVersion: 1.0
 constants:
   - ChefVersion:
       type: string
-      value: 18.4.12
+      value: 18.7.10
   - FailExitCode:
       type: string
       value: 1
@@ -57,7 +57,7 @@ phases:
               COOKBOOK_URL="https://${AWS::Region}-aws-parallelcluster.s3.${AWS::Region}.${AWS::URLSuffix}/parallelcluster/${CfnParamCookbookVersion}/cookbooks/${!COOKBOOK_NAME}.tgz"
               [ -n "${CfnParamChefCookbook}" ] && COOKBOOK_URL="${CfnParamChefCookbook}"
               
-              CINC_URL="https://${AWS::Region}-aws-parallelcluster.s3.${AWS::Region}.${AWS::URLSuffix}/archives/cinc/cinc-install-1.3.0.sh"
+              CINC_URL="https://${AWS::Region}-aws-parallelcluster.s3.${AWS::Region}.${AWS::URLSuffix}/archives/cinc/cinc-install-1.4.0.sh"
               [ -n "${CfnParamCincInstaller}" ] && CINC_URL="${CfnParamCincInstaller}"
               
               # Output all variables to persistent location
@@ -186,7 +186,11 @@ phases:
                 apt-get -y --only-upgrade install ca-certificates
               fi
               
-              curl --retry 3 -L $CINC_URL | bash -s -- -v {{ ChefVersion }}
+              OS="{{ build.OperatingSystemName.outputs.stdout }}"
+              ChefVersion={{ ChefVersion }}
+              [[ $OS == alinux2 ]] && ChefVersion=18.4.12
+              
+              curl --retry 3 -L $CINC_URL | bash -s -- -v $ChefVersion
               
               [[ -e $CA_CERTS_FILE ]] && {
                 mkdir -p /opt/cinc/embedded/ssl/certs