enable device index check for all device types

[garfield1997]

enable device index check for all device types for grad setter.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/126767

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (3 Unrelated Failures)

As of commit 7e8d1d8 with merge base 00f675b ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / linux-focal-py3.8-clang10 / test (dynamo, 3, 3, linux.2xlarge) (gh) (similar failure)
  test_quantization.py::TestQuantizePT2EQATModels::test_qat_mobilenet_v2
• trunk / macos-py3-arm64-mps / test (mps, 1, 1, macos-m1-13) (gh) (similar failure)
  test_mps.py::TestMPS::test_mps_allocator_module
• trunk / macos-py3-arm64-mps / test (mps, 1, 1, macos-m1-14) (gh) (similar failure)
  test_mps.py::TestMPS::test_mps_allocator_module

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[janeyx99]

Looks fine to me, needs test though

[albanD]

The blast radius is pretty large but that sounds ok to me as any device with single hw should just return 0 all the time. Will let @soulitzer give the final stamp if he's happy with it.

Also agreed that we need test to make sure it works fine.

[garfield1997]

I believe there are already some tests in place. https://github.com/pytorch/pytorch/blob/main/test/test_autograd.py#L11299

# Tests cross-device assignment raises
        if len(devices) > 1:
            x = torch.randn(5, 5, device=devices[0])
            with self.assertRaises(RuntimeError):
                x.grad = torch.randn(5, 5, device=devices[1])

Do we need additional testing? @albanD @janeyx99

[albanD]

Do we run this test on non-cuda devices? I would expect that this was failing before and would be fixed with this PR for non-cuda devices?

[garfield1997]

I am adding a custom backend for PyTorch. This test case used to fail and will be fixed by this PR. @albanD

[garfield1997]

If you have the time, could you give me some feedback on this PR? @albanD @janeyx99

[soulitzer]

Sounds reasonable to me

Though also on testing, @albanD, do you know what our general testing strategy is for non-cuda devices, for things like this

[albanD]

The best Plan On Record for testing these is what is described in this rfc: pytorch/rfcs#64 but it is not done yet I'm afraid.

[albanD]

Sounds good then!

[albanD]

@pytorchbot merge

[pytorchmergebot]

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

[albanD]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: 1 jobs have failed, first few of them are: trunk / linux-focal-cuda12.1-py3.10-gcc9 / test (nogpu_NO_AVX2, 1, 1, linux.2xlarge)

Details for Dev Infra team

Raised by workflow job

[garfield1997]

The CI failure seems unrelated to the changes in the PR, and I am unable to reproduce it locally.

[FFFrog]

The best Plan On Record for testing these is what is described in this rfc: pytorch/rfcs#64 but it is not done yet I'm afraid.

cc @albanD @soulitzer @garfield1997

pytorch/rfcs#64 is currently in the design and development stage. We will do our best to speed up the development and open source it as soon as possible.

As stated in the RFC, we plan to abstract and standardize the third-party device access mechanism and in theory, it can provide the following benefits:

1. Code abstract reuse to accelerate the integration of new elements
2. Standardized integration to improve integration quality of third-party devices
3. Streamlined backend and unified test coverage ensure the availability of third-party device integration mechanisms
4. End-to-end full process documentation and official DEMO

[garfield1997]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: 3 mandatory check(s) failed. The first few are:

• BC Lint
• pull
• Lint

Dig deeper by viewing the failures on hud

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

[garfield1997]

Could you please help trigger the merge again? @albanD

[garfield1997]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased remove_grad_cuda_check onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout remove_grad_cuda_check && git pull --rebase)

[garfield1997]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here