Skip to content

bug: redfish create_client fails when DGX H100 host is powered off #5418

Description

@ericpretzel

Version

v2.1.0-rc.3

Describe the bug.

When a DGX H100 host is powered off, the BMC responds with an error when you try to access Systems/HGX_Baseboard_0. I think this is normal behavior for the BMC. However, in libredfish it seems like it's necessary for this endpoint to respond with no error in order for a redfish client to be created, so the machine effectively gets stuck in any state that requires it to power off. If someone manually turns it on (causing the redfish client creation to start working), NICo will just turn it off again.

This was observed when re-ingesting a DGX H100 machine. It got stuck in DPUInitializing/WaitingForPlatformPowerOff
with state handler error:
Failed redfish operation: create_client. Details: failed redfish request HTTP 503 Service Unavailable at https://[redacted]/redfish/v1/Systems/HGX_Baseboard_0: {"error":{"@Message.ExtendedInfo":[{"@odata.type":"#Message.v1_0_8.Message","Message":" Indicates that HOSTBMC fails to communicate with HMC ","MessageId":"Ami.1.0.HMCCommuicationFailed","Resolution":"Makesure HMC configuration details is valid, if not correct the HMC Configuation in JSON File and restart the redfish server.","Severity":"Warning"}],"code":"Ami.1.0.HMCCommuicationFailed","message":" Indicates that HOSTBMC fails to communicate with HMC "}}.
I confirmed the host was indeed powered off.

Minimum reproducible example

Relevant log output

Other/Misc.

No response

Code of Conduct

  • I agree to follow NVIDIA Infra Controller's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report

Metadata

Metadata

Assignees

Labels

bugA defect in existing software (deprecated - use issue type, but it's needed for reporting now)

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions