[Detector Support]: Yolo-NAS notebook from documentation fails to run with 404 error

[kidhasmoxy]

Describe the problem you are having

The pretrained weights for YOLO-NAS are no longer available with a 404 error. The domain they are hosted under is for sale, so any automated attempts to download files from the model zoo will fail. This makes the documentation impossible to follow for OpenVino Yolo-NAS setup.

Deci, the company behind the super-gradients library was acquired by Nvidia earlier this year. It's unclear whether the weights will be moved to a new location or whether the library will continue with the license it had.

Version

14

Frigate config file

! pip install -q super_gradients==3.7.1
     

from super_gradients.common.object_names import Models
from super_gradients.conversion import DetectionOutputFormatMode
from super_gradients.training import models

model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")
     

# export the model for compatibility with Frigate

model.export("yolo_nas_s.onnx",
             output_predictions_format=DetectionOutputFormatMode.FLAT_FORMAT,
             max_predictions_per_image=20,
             confidence_threshold=0.4,
             input_image_shape=(320,320),
            )
     

from google.colab import files

files.download('yolo_nas_s.onnx')

docker-compose file or Docker CLI command

N/A

Relevant Frigate log output

[2024-10-20 13:16:58] INFO - crash_tips_setup.py - Crash tips is enabled. You can set your environment variable to CRASH_HANDLER=FALSE to disable it
[2024-10-20 13:17:09] INFO - utils.py - NumExpr defaulting to 2 threads.
[2024-10-20 13:17:17] WARNING - checkpoint_utils.py - :warning: The pre-trained models provided by SuperGradients may have their own licenses or terms and conditions derived from the dataset used for pre-training.
 It is your responsibility to determine whether you have permission to use the models for your use case.
 The model you have requested was pre-trained on the coco dataset, published under the following terms: https://cocodataset.org/#termsofuse
[2024-10-20 13:17:17] INFO - checkpoint_utils.py - License Notification: YOLO-NAS pre-trained weights are subjected to the specific license terms and conditions detailed in 
https://github.com/Deci-AI/super-gradients/blob/master/LICENSE.YOLONAS.md
By downloading the pre-trained weight files you agree to comply with these terms.
Downloading: "https://sghub.deci.ai/models/yolo_nas_s_coco.pth" to /root/.cache/torch/hub/checkpoints/yolo_nas_s_coco.pth
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
<ipython-input-1-c4c919ec4cd2> in <cell line: 2>()
      1 from super_gradients.training import models
----> 2 model = models.get("yolo_nas_s", pretrained_weights="coco")

11 frames
/usr/lib/python3.10/urllib/request.py in http_error_default(self, req, fp, code, msg, hdrs)
    641 class HTTPDefaultErrorHandler(BaseHandler):
    642     def http_error_default(self, req, fp, code, msg, hdrs):
--> 643         raise HTTPError(req.full_url, code, msg, hdrs, fp)
    644 
    645 class HTTPRedirectHandler(BaseHandler):

HTTPError: HTTP Error 404: Not Found

Install method

HassOS Addon

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

Any other information that may be helpful

No response

[kidhasmoxy]

For this running into this, there is a temporary workaround. First you need to change the first line of the notebook to import from the latest github commit.

! pip install -U git+https://github.com/Deci-AI/super-gradients

When you run it, you'll get an error about Index out of range referencing checkpoint_utils.py. From there, you'll need to edit checkpoint_utils.py to change line 1592 from
unique_filename = url.split("https://sghub.deci.ai/models/")[1].replace("/", "_").replace(" ", "_")

to

unique_filename = url.split("https://sg-hub-nv.s3.amazonaws.com/models/")[1].replace("/", "_").replace(" ", "_")

[partytimeexcellent]

I changed the first step as you suggested, but instead of an error regarding checkpoint_utils.py, I get a dependency conflict error:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ipython 7.34.0 requires jedi>=0.16, which is not installed.

[partytimeexcellent]

And when running the second step, after changing the line in checkpoint_utils.py, I still get an index error:

[2024-10-24 08:13:17] WARNING - checkpoint_utils.py - :warning: The pre-trained models provided by SuperGradients may have their own licenses or terms and conditions derived from the dataset used for pre-training.
 It is your responsibility to determine whether you have permission to use the models for your use case.
 The model you have requested was pre-trained on the coco dataset, published under the following terms: https://cocodataset.org/#termsofuse
[2024-10-24 08:13:17] INFO - checkpoint_utils.py - License Notification: YOLO-NAS pre-trained weights are subjected to the specific license terms and conditions detailed in 
https://github.com/Deci-AI/super-gradients/blob/master/LICENSE.YOLONAS.md
By downloading the pre-trained weight files you agree to comply with these terms.

---------------------------------------------------------------------------

IndexError                                Traceback (most recent call last)

[<ipython-input-5-c847f4ac59f2>](https://localhost:8080/#) in <cell line: 5>()
      3 from super_gradients.training import models
      4 
----> 5 model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")

3 frames

[/usr/local/lib/python3.10/dist-packages/super_gradients/training/utils/checkpoint_utils.py](https://localhost:8080/#) in load_pretrained_weights(model, architecture, pretrained_weights)
   1590         pretrained_state_dict = torch.load(url.replace("file://", ""), map_location="cpu")
   1591     else:
-> 1592         unique_filename = url.split("https://sg-hub-nv.s3.amazonaws.com/models/")[1].replace("/", "_").replace(" ", "_")
   1593         map_location = torch.device("cpu")
   1594         with wait_for_the_master(get_local_rank()):

IndexError: list index out of range

[dreaming0512]

you should change "url = MODEL_URLS[model_url_key]" to url = "https://sg-hub-nv.s3.amazonaws.com/models/yolo_nas_s_coco.pth" in checkpoint_utils.py.

It's on the 1571th line in checkpoint_utils.py.

[kidhasmoxy]

That usually means you need to restart the Runtime to get it to take the changes to the environment. That's required anytime you change the cached python files.

…

On Thu, Oct 24, 2024 at 4:15 AM partytimeexcellent ***@***.***> wrote: And when running the second step, after changing the line in checkpoint_utils.py, I still get an index error: [2024-10-24 08:13:17] WARNING - checkpoint_utils.py - :warning: The pre-trained models provided by SuperGradients may have their own licenses or terms and conditions derived from the dataset used for pre-training. It is your responsibility to determine whether you have permission to use the models for your use case. The model you have requested was pre-trained on the coco dataset, published under the following terms: https://cocodataset.org/#termsofuse [2024-10-24 <https://cocodataset.org/#termsofuse[2024-10-24> 08:13:17] INFO - checkpoint_utils.py - License Notification: YOLO-NAS pre-trained weights are subjected to the specific license terms and conditions detailed in https://github.com/Deci-AI/super-gradients/blob/master/LICENSE.YOLONAS.md By downloading the pre-trained weight files you agree to comply with these terms. --------------------------------------------------------------------------- IndexError Traceback (most recent call last) [<ipython-input-5-c847f4ac59f2>](https://localhost:8080/#) in <cell line: 5>() 3 from super_gradients.training import models 4 ----> 5 model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco") 3 frames [/usr/local/lib/python3.10/dist-packages/super_gradients/training/utils/checkpoint_utils.py](https://localhost:8080/#) in load_pretrained_weights(model, architecture, pretrained_weights) 1590 pretrained_state_dict = torch.load(url.replace("file://", ""), map_location="cpu") 1591 else: -> 1592 unique_filename = url.split("https://sg-hub-nv.s3.amazonaws.com/models/")[1].replace("/", "_").replace(" ", "_") 1593 map_location = torch.device("cpu") 1594 with wait_for_the_master(get_local_rank()): IndexError: list index out of range — Reply to this email directly, view it on GitHub <#14457 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAJCARMVYOOGOJBWMMLVH3Z5CUC7AVCNFSM6AAAAABQIQXJ46VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMBTG44TGMQ> . You are receiving this because you authored the thread.Message ID: ***@***.*** com>

[partytimeexcellent]

Ok yes! this was the missing key.

For any noobs like me here is a slightly more detailed tutorial:

1. Change the first line of the notebook to import from the latest github commit:
   replace: ! pip install -q super_gradients==3.7.1
   with: ! pip install -U git+https://github.com/Deci-AI/super-gradients

2. Run Cell 1
   You will probably receive the following error, (which can be ignored) and be prompted to restart the environment, which you should do.

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
ipython 7.34.0 requires jedi>=0.16, which is not installed.

3. Run Cell 2
   You will receive the following error:

IndexError                                Traceback (most recent call last)

[<ipython-input-1-c847f4ac59f2>](https://localhost:8080/#) in <cell line: 5>()
      3 from super_gradients.training import models
      4 
----> 5 model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")

3 frames

[/usr/local/lib/python3.10/dist-packages/super_gradients/training/utils/checkpoint_utils.py](https://localhost:8080/#) in load_pretrained_weights(model, architecture, pretrained_weights)
   1590         pretrained_state_dict = torch.load(url.replace("file://", ""), map_location="cpu")
   1591     else:
-> 1592         unique_filename = url.split("https://sghub.deci.ai/models/")[1].replace("/", "_").replace(" ", "_")
   1593         map_location = torch.device("cpu")
   1594         with wait_for_the_master(get_local_rank()):

IndexError: list index out of range

4. The error references /usr/local/lib/python3.10/dist-packages/super_gradients/training/utils/checkpoint_utils.py
   Click on this link to open the file editor and locate line 1592

5. Replace unique_filename = url.split("https://sghub.deci.ai/models/")[1].replace("/", "_").replace(" ", "_")
   With unique_filename = url.split("https://sg-hub-nv.s3.amazonaws.com/models/")[1].replace("/", "_").replace(" ", "_")

6. Navigate to "Runtime" menu and click "Restart Session"

7. Run Cell 2 again, this time there should be no error

8. Run Cell 3

9. Run Cell 4 and save the file to local storage

I tested this on both Chrome and Firefox, working on both browsers as of 10/24/2024

[numminorih-git]

29/10/2024 work. Thx!

[distinctjuggle]

@NickM-27 The documentation for the ONNX detector in the beta doesn't say anything about setting the input_tensor, so if it is required, the documentation should be updated to reflect that.

For anyone else with the same problem, the correct setting seems to be: input_tensor: nchw underneath model:

[NickM-27]

yep, will be updated in next PR

[Korenchkin]

@NickM-27 The documentation for the ONNX detector in the beta doesn't say anything about setting the input_tensor, so if it is required, the documentation should be updated to reflect that.

For anyone else with the same problem, the correct setting seems to be: input_tensor: nchw underneath model:

not only onnx,openvino too,same problem with yolonas and nchw (n?c?Height Width probably)..not that i know what i'm doing :)

[NickM-27]

That is incorrect, openvino has always had the correct config shown https://docs.frigate.video/configuration/object_detectors#yolo-nas

[Korenchkin]

That is incorrect, openvino has always had the correct config shown https://docs.frigate.video/configuration/object_detectors#yolo-nas

Ah,sorry about that,too much info and variants of config i'm getting lost in it

[distinctjuggle]

I'm getting a completely different error here than others. I've changed the line 1 endpoint, ran it, restarted the environment, then ran block 2:

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

[<ipython-input-1-c847f4ac59f2>](https://localhost:8080/#) in <cell line: 1>()
----> 1 from super_gradients.common.object_names import Models
      2 from super_gradients.conversion import DetectionOutputFormatMode
      3 from super_gradients.training import models
      4 
      5 model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")

12 frames

[/usr/lib/python3.10/abc.py](https://localhost:8080/#) in __subclasscheck__(cls, subclass)
    121         def __subclasscheck__(cls, subclass):
    122             """Override for issubclass(subclass, cls)."""
--> 123             return _abc_subclasscheck(cls, subclass)
    124 
    125         def _dump_registry(cls, file=None):

TypeError: issubclass() arg 1 must be a class

If I rerun this block 2 with no other changes or steps being made/taken, the error will change?

---------------------------------------------------------------------------

ImportError                               Traceback (most recent call last)

[<ipython-input-2-c847f4ac59f2>](https://localhost:8080/#) in <cell line: 3>()
      1 from super_gradients.common.object_names import Models
      2 from super_gradients.conversion import DetectionOutputFormatMode
----> 3 from super_gradients.training import models
      4 
      5 model = models.get(Models.YOLO_NAS_S, pretrained_weights="coco")

1 frames

[/usr/local/lib/python3.10/dist-packages/super_gradients/training/__init__.py](https://localhost:8080/#) in <module>
      1 # PACKAGE IMPORTS FOR EXTERNAL USAGE
----> 2 import super_gradients.training.utils.distributed_training_utils as distributed_training_utils
      3 from super_gradients.common import MultiGPUMode, StrictLoad, EvaluationType
      4 from super_gradients.training.datasets import datasets_utils, DataAugmentation
      5 from super_gradients.training.pre_launch_callbacks import modify_params_for_qat

ImportError: cannot import name 'utils' from partially initialized module 'super_gradients.training' (most likely due to a circular import) (/usr/local/lib/python3.10/dist-packages/super_gradients/training/__init__.py)


---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

I've tried resetting the environment by Runtime -> Disconnect and Delete Runtime but that doesn't seem to fix the issue.

[distinctjuggle]

@ProsjektX (tagging for your benefit)

I found a workaround using a specific commit version to get the same error as documented here:

Change the first line/block to: pip install -U git+https://github.com/Deci-AI/super-gradients@e0ccacf8868ffa1296fa4f8407c03d2bc227312c

Then you can follow all other steps as outlined.

[Devchili]

pip install -U git+https://github.com/Deci-AI/super-gradients@e0ccacf8868ffa1296fa4f8407c03d2bc227312c

@distinctjuggle are you installing supervision?

[distinctjuggle]

I'm not knowingly doing anything other than as described. I don't recall supervision being anything I explicitly touched one way or the other.

[ProsjektX]

The procedure by partytimeexcellent worked a few days ago, but now I get the same fault as in the previous post. Have no clue though..

[jurasjo]

03/11/2024 procedure works.

[distinctjuggle]

What steps are you following? Because I'm still getting the same error that I posted yesterday, which is not resolved by any procedure posted here.

[jurasjo]

Those posted by partytimeexcellent

[distinctjuggle]

Can't reproduce your success. I'm still getting the same error as I posted a few days ago.

[NickM-27]

where do you see that? by the way you should create your own discussion, this is not the right place to discuss this

[victpork]

Hi everyone,
Stumbled upon this thread when trying to generate a Yolo-NAS model. In case anyone still struggling to generate the model, here is what I got after trying the methods above with no success:

1. Open the original notebook bundled with Frigate in Google Colab
2. Add a new code cell after the first one, and paste the code below into it:

! sed -i 's/sghub.deci.ai/sg-hub-nv.s3.amazonaws.com/' /usr/local/lib/python3.10/dist-packages/super_gradients/training/pretrained_models.py
! sed -i 's/sghub.deci.ai/sg-hub-nv.s3.amazonaws.com/' /usr/local/lib/python3.10/dist-packages/super_gradients/training/utils/checkpoint_utils.py

3. Run the code blocks one by one in sequence.

[sakcaj]

@blakeblackshear can you please update the notebook linked under https://docs.frigate.video/configuration/object_detectors#yolo-nas to the above? I've struggled with the this for ~1h, finally found this thread and the above worked flawlessly.

[victpork]

I created PR #15087 for the change, cheers :)

[midwesternrodent]

Hey, for anyone finding this discussion on google before the PR with victpork's changes is out of master. Follow their comment first.

If you've tried other solutions here before scrolling down this far and are now getting a bunch of errors about something being wrong in abc.py or are having issues importing modules due to circular imports, and you're a newbie to google colab (like me). EVEN IF, you've closed the tab, reopened it, reset the runtime, etc... unless you specifically go to runtime > disconnect and delete runtime there still might be some residual things waiting around from a previous run and causing all sorts of issues for you.

Disconnect and delete your runtime, open the notebook again, follow victpork's suggestion and it should work just fine.