0.15.0 Beta 1

[blakeblackshear]

Beta Documentation: https://deploy-preview-13787--frigate-docs.netlify.app/

Images

• ghcr.io/blakeblackshear/frigate:0.15.0-beta1
• ghcr.io/blakeblackshear/frigate:0.15.0-beta1-standard-arm64
• ghcr.io/blakeblackshear/frigate:0.15.0-beta1-tensorrt
• ghcr.io/blakeblackshear/frigate:0.15.0-beta1-tensorrt-jp4
• ghcr.io/blakeblackshear/frigate:0.15.0-beta1-tensorrt-jp5
• ghcr.io/blakeblackshear/frigate:0.15.0-beta1-rk
• ghcr.io/blakeblackshear/frigate:0.15.0-beta1-rocm
• ghcr.io/blakeblackshear/frigate:0.15.0-beta1-h8l

Major Changes for 0.15.0

Breaking Changes

There are several breaking changes in this release, Frigate will attempt to update the configuration automatically. In some cases manual changes may be required. It is always recommended to back up your current config and database before upgrading:

1. Simply copy your current config file to a new location
2. Stop Frigate and make a copy of the frigate.db file

• Rewritten SHM system
  • The SHM system has been rewritten to be more stable. This means that more shared memory (SHM) will be used by default but the total usage will not fluctuate, and Frigate will not fail from SHM overflowing. A new calculation is done on startup, so if your shm_size is too low then a warning will be printed in the log stating that it needs to be increased.
• Recording configuration changed
  • The record config has been refactored to allow for direct control of how long alerts and detections are retained. These values will be automatically populated from your current config, but you may want to adjust the values that are set after updating. See the updated docs here.
• ffmpeg updates
  • ffmpeg has been updated to version 7. Some Intel users may need to change what hwaccel preset they are using (preset-vaapi may need to now be preset-intel-qsv-h264 or preset-intel-qsv-h265).
  • ffmpeg is no longer part of $PATH. In most cases this is handled automatically, but go2rtc exec streams will need to add the full path for ffmpeg which in most cases will be /usr/lib/ffmpeg/7.0/bin/ffmpeg

Explore

The new Explore pane in Frigate 0.15 makes it easy to explore every object tracked by Frigate. It offers a variety of filters and supports keyword and phrase-based text search, searching for similar images, and searching through descriptive text generated by AI models.

AI/ML Search

Frigate 0.15 introduces two powerful search features: Semantic Search and GenAI Search. Semantic Search can be enabled on its own, while GenAI Search works in addition to Semantic Search.

Semantic Search

Semantic Search uses a CLIP model to generate embeddings (numerical representations of images) for the thumbnails of your tracked objects, enabling searches based on text descriptions or visual similarity. This is all done locally.

For instance, if Frigate detects and tracks a car, you can use similarity search to see other instances where Frigate detected and tracked that same car. You can also quickly search your tracked objects using an "image caption" approach. Searching for "red car driving on a residential street" or "person in a blue shirt walking on the sidewalk at dawn" or even "a person wearing a black t-shirt with the word 'SPORT' on it" will produce some stunning results.

Semantic Search works by running an AI model locally on your system. Small or underpowered systems like a Raspberry Pi will not run Semantic Search reliably or at all. A dedicated GPU and 16GB of RAM is recommended for best performance.

See the Semantic Search docs for system requirements, setup instructions, and usage tips.

Generative AI

GenAI Search employs generative AI models to create descriptive text for the thumbnails of your tracked objects, which are stored in the Frigate database to enhance future searches. Supported providers include Google Gemini, Ollama, and OpenAI, so you can choose whether you want to send data to the cloud or use a locally hosted provider.

See the GenAI docs for setup instructions and use case suggestions.

Improved Tools for Debugging

Review Item Details Pane

A new Review Item Details pane can be viewed by clicking / tapping on the gray chip on a review item in the Review pane. This shows more information about the review item as well as thumbnails or snapshots for individual objects (if enabled). The pane also provides links to share the review item, download it, submit images to Frigate+, view object lifecycles, and more.

Object Lifecycle Pane

The Recordings Timeline from Frigate 0.13 has been improved upon and returns to 0.15 as the Object Lifecycle, viewable in the Review Details pane as well as the new Explore page. The new pane shows the significant moments during the object's lifecycle: when it was first seen, when it entered a zone, became stationary, etc. It also provides information about the object's area and size ratio to assist in configuring Frigate to tune out false positives.

Native Notifications

Frigate now supports notifications using the WebPush protocol. This allows Frigate to deliver notifications to devices that have registered to receive notifications in the Frigate settings, delivering them in a timely and secure manner. Currently, notifications will be delivered for all review items marked as alerts. More options for native notifications will be supported in the future.

See the notifications docs.

New Object Detectors

ONNX

ONNX is an open model standard which allows for a single model format that can run on different types of GPUs. The default, tensorrt, and rocm Frigate build variants include GPU support for efficient object detection via ONNX models, simplifying configuration and support more models. There are no default included ONNX models for object detection.

AMD MiGraphX

Support has been added for AMD GPUs via ROCm and MiGraphX. Currently there is no default included model for this detector.

Hailo-8

Support has been added for the Hailo8 and Hailo-8L hardware for object detection on both arm64 and amd64 platforms.

Other UI Changes

• Rework the Storage metrics pane to be more clear how much storage each camera is using
• Camera stream info button has been added to the camera metrics page allowing easy access to see camera codec information
• Add button for downloading the full set of logs
• The timeline handlebar will now automatically scroll while viewing recordings
• Review items and exports can now be shared via your browser's sharing pane or via direct link url. A secure context is required for the browser/OS sharing pane to function
• Add the ability to rename camera groups
• Add middle mouse button support in the Review pane to open full recordings view in a new tab
• Add Nvidia GPU encode and decode stats to system metrics
• New desktop keyboard shortcuts. In addition to the numerous keyboard shortcuts available with 0.14, 0.15 adds several more. In Live view, the numeric keys (1-9) will move PTZ cameras to presets. In Review, Page Up and Page Down will scroll through review items
• Accessibility updates for users with screen readers

Other Backend Changes

• FFMPEG updated to 7.0
• OpenVINO updated to 2024.3
• Use FastAPI instead of Flask for the Frigate API
• Add service manager infrastructure
• Install onevpl tools and update drivers for Intel
• If record is disabled for a camera then that camera will not save frames to the preview during that time
• Fix stalled connections to certain onvif cameras
• Fix autotracking absolute zoom interpolation logic

---

This discussion was created from the release 0.15.0 Beta 1.

[SigHunter]

impressive pace over the last months! Frigate is now the best NVR tool

[pyrodex]

Thank you so much for the hard work the Frigate team!

I just upgraded, yes I've been stalking this upgrade for a while now and see a few issues.

Seeing the following errors occasionally in the logs and didn't see anything like this in 0.14.1:

2024-10-29 07:35:54.874458654  [2024-10-29 07:35:54] frigate.video                  ERROR   : Charlotte: Unable to read frames from ffmpeg process.
2024-10-29 07:35:54.874501456  [2024-10-29 07:35:54] frigate.video                  ERROR   : Charlotte: ffmpeg process is not running. exiting capture thread...
2024-10-29 07:35:56.891616481  [2024-10-29 07:35:56] watchdog.Charlotte             ERROR   : Ffmpeg process crashed unexpectedly for Charlotte.
2024-10-29 07:35:56.891658856  [2024-10-29 07:35:56] watchdog.Charlotte             ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
2024-10-29 07:35:56.891698064  [2024-10-29 07:35:56] ffmpeg.Charlotte.detect        ERROR   : [AVHWFramesContext @ 0x7ff398053000] Failed to sync surface 0xe: 1 (operation failed).
2024-10-29 07:35:56.891734307  [2024-10-29 07:35:56] ffmpeg.Charlotte.detect        ERROR   : [hwdownload @ 0x7ff3a8003980] Failed to download frame: -5.
2024-10-29 07:35:56.891760967  [2024-10-29 07:35:56] ffmpeg.Charlotte.detect        ERROR   : [vf#0:0 @ 0x560ee3d1d100] Error while filtering: Input/output error
2024-10-29 07:35:56.891788631  [2024-10-29 07:35:56] ffmpeg.Charlotte.detect        ERROR   : [vf#0:0 @ 0x560ee3d1d100] Task finished with error code: -5 (Input/output error)
2024-10-29 07:35:56.891823146  [2024-10-29 07:35:56] ffmpeg.Charlotte.detect        ERROR   : [vf#0:0 @ 0x560ee3d1d100] Terminating thread with return code -5 (Input/output error)

When I look at the cameras they are fine. The cameras reporting this are all Amcrests and I run all my cameras through go2rtc and Frigate consumes those of course via go2rtc.

Going to the review section when I hover over some of the events, seems to be the most recent ones, I see a blank square but others show the mini preview image.

Looking forward to helping out and making this best possible. Let me know if you need anything else to help troubleshoot.

[pyrodex]

What CPU do you have?

13th Gen Intel(R) Core(TM) i7-13700

[NickM-27]

You need to use qsv

[pyrodex]

You need to use qsv

So change:

ffmpeg:
  hwaccel_args: preset-vaapi
  input_args: preset-rtsp-restream
  output_args:
    record: preset-record-generic-audio-aac

to

ffmpeg:
  hwaccel_args: preset-intel-qsv-h265
  input_args: preset-rtsp-restream
  output_args:
    record: preset-record-generic-audio-aac

?

[NickM-27]

Depends what codec your detect streams are, but in general yes

[pyrodex]

Oh see I thought those settings were always related to the record stream not detect. My cameras are configured with main stream as 265 and sub-stream (detect) for 264. I made the qsv 264 tweak and will watch now for issues.

[leccelecce]

Fantastic progress as ever!

One point on Semantic Search - interested as to where the 16GB RAM requirement comes from for the CLIP model. I've been running 0.15 dev builds for a few weeks and my Frigate container (which is set to 16GB max RAM on a 32GB system) has never exceeded 4.5GB memory use. I also only have an Intel 10700 iGPU and don't see any real performance problems (I think it took about 20 minutes to re-encode 7,000 snapshots). Appreciate you might not want to lower the requirement, but just sharing my experience to avoid putting anyone else off...

Of course, 0.16 sounds like a GPU could become more mandatory for face recognition, so whether the long term view is that it's necessary would be a fair one.

[leccelecce]

16GB is not a requirement, but a recommendation. We want to make it clear that smaller and underpowered devices won't perform well (or at all) with some of the new and upcoming features.

This is part of why I see merit in a roadmap of planned features (not a timeline, but a feature list) - so users can make the right hardware choices.

I've been pondering this a bit so will make a separate post on the subject.

[NickM-27]

The thing is it is not really planned, and the supported hardware is also not known. The face recognition / alpr was not planned for 0.16 until we started talking about it literally a week ago. And it was not clear what hardware would be supported until we actually tried it out, found a model, etc.

Things could certainly still change with it too.

[hawkeye217]

As we've indicated elsewhere, there is no roadmap, no official development team, and no funding for Frigate beyond what users give via Github Sponsors. Frigate+ is exclusively Blake's project, Nick and I don't contribute or receive benefit from it. Nick and I are just volunteer contributors, and like he said, it was just last week that both of us worked on separate projects (face detection and license plate recognition) and decided to contribute them to the Frigate codebase.

For a list of planned features, you can always take a look at the pinned issues. These are things users have requested that we feel are worthwhile to eventually have within Frigate.

[leccelecce]

I guess I'm as much asking from the perspective that you guys are probably as well-informed as anybody on the deployment of AI models relevant to the security space! Also appreciate that you're not sat there with the equipment or time to test across loads of different hardware configurations.

Perhaps I could create a simple "benchmark.py" script that can be run as a non-default docker image command by Frigate, or a UI button, that executes a set of AI activities against some pre-supplied images and outputs the time taken

[NickM-27]

it's possible things like semantic search can have inference statistics just like the object detectors as a way to gauge how things are running.

[conorlap]

Is there an update to https://github.com/blakeblackshear/frigate-hass-integration or https://github.com/dermotduffy/frigate-hass-card required?

I can see the clips in the card, but when I click one to play it just shows a loading spinner and doesn't play any clip.

[NickM-27]

No, nothing has changed in these regards

[SeanChengN]

Do you have plans to accept multiple languages?

[NickM-27]

It would likely make sense at some point, that would be a very large effort though to be clear

[SeanChengN]

It would likely make sense at some point, that would be a very large effort though to be clear

Thank you to the team's hard work and dedication.
Fortunately, frigate has an active community, and we can use Crowdin to solicit translations like other projects.

[dan-r]

First of all thank you to the team (and contributors) for all the time spent making Frigate so incredible!

I was having some issues after upgrading, related to having birdseye enabled with restream.

2024-10-29 12:41:20.389690+00:00Traceback (most recent call last):
2024-10-29 12:41:20.389747+00:00File "/usr/local/go2rtc/create_config.py", line 168, in <module>
2024-10-29 12:41:20.389700+00:002024-10-29 12:41:20.389626385  [INFO] Not injecting WebRTC candidates into go2rtc config as it has been set manually
2024-10-29 12:41:20.389870+00:00ffmpeg_cmd = f"exec:{parse_preset_hardware_acceleration_encode(ffmpeg_path, config.get('ffmpeg', {}).get('hwaccel_args'), input, '-rtsp_transport tcp -f rtsp {output}')}"
2024-10-29 12:41:20.389935+00:00File "/opt/frigate/frigate/ffmpeg_presets.py", line 227, in parse_preset_hardware_acceleration_encode
2024-10-29 12:41:20.390020+00:00return arg_map["default"].format(input, output)
2024-10-29 12:41:20.390042+00:00IndexError: Replacement index 2 out of range for positional args tuple

Looks like as I don't have an ffmpeg>hwaccel_args config key, config.get('ffmpeg', {}).get('hwaccel_args') is coming back as None and causing a crash loop in go2rtc. Although a good prompt for me to sort out hardware acceleration, the ffmpeg section is marked as optional.

Seems like amending this line to the below (adding , 'auto') does the job fixing it. Happy to submit a PR/create an issue with this if it seems reasonable and I'm not missing something else obvious?

ffmpeg_cmd = f"exec:{parse_preset_hardware_acceleration_encode(ffmpeg_path, config.get('ffmpeg', {}).get('hwaccel_args', 'auto'), input, '-rtsp_transport tcp -f rtsp {output}')}"

[NickM-27]

auto as a default doesn't really make sense because the go2rtc config is created before frigate starts. Realistically you will need to set the hwaccel_args globally to ensure birdseye is restreamed efficiently.

But yes setting the default as an empty string there will solve the issue so feel free to put up a PR

[dan-r]

Thanks for the quick response, raised under #14658

[pyrodex]

Found another issue related to Tracked Object Details interface, the left and right arrows at the bottom don't do anything so having to click on the icons in the middle section works but not the left or right:

[pyrodex]

No, because if you look at your example the buttons are disabled. If I then make my browser small enough to hide some of the middle icons the arrows then become active and clickable.

To be honest that is kind of confusing, should they be hidden if large enough or just active them for a more linear progression?

[NickM-27]

I'm only explaining that the behavior appears to be expected and not a bug in the buttons. Hiding them might be preferable

[hawkeye217]

This is using two carousel UI elements - the first is the one your camera frame is displayed in, which can be dragged on desktop and swiped on mobile. The second carousel is the one with multiple squares/icons. The arrows correspond to that carousel and will be active/not grayed out when there are many steps in the object's life cycle. I can see how this may be confusing. We'll improve it for the next beta.

[hawkeye217]

Just put up a PR to cause the buttons to move through the object lifecycle elements #14662

[OnFreund]

I think it might be less confusing if the arrows are right next to the icons, rather than being spread wide.

[YBonline]

So upgrading from 0.14.1 to this, for the first 10 minutes or so, the cameras seemed to load and then keep crashing over and over again. It seems to have stopped doing it for the next 10 minutes, so not sure yet if its just a temporary problem on startup/first upgrade, I can keep you updated... here's logs during the issue (we got the old "Unable to keep up with recording segments in cache for . Keeping the 6 most recent segments out of 8 and discarding the rest...":
frigate_logs.txt

I haven't yet tried the new features, this was just a simple upgrade

[NathanVanCleave]

I also noticed this with a few cameras. I restarted Frigate after about 10 minutes and it hasn't come back. I've also restarted it a few more times since then and it hasn't happened again.

[mimainka]

Spun up my container using the beta and now i don't have the icon for frigate+ any longer on the left side of my screen.
When I go into the review section however, i can send frames to frigate+ and they do arrive on plus.frigate.video.
Also cleared cache and so on.
Anything i'm missing? Had the same behavior in previous dev-containers but not in stable releases nor the 0.14 betas.

[uncle-buzz]

This is expected, use the explore view to submit to Frigate+

Would it be possible to have an icon or any other visual effect on the explore view to see if an item was already sent to frigate+? We have to open a review item and go to the snapshot to see if it was already sent or if we can send it, could be good to know it without having to open it.
Or maybe I missed a point?
Thanks

[NickM-27]

The send to plus button shows on the card if the object has not been sent and has a snapshot. You can also use the 3 dots

[almeidainf]

I do appreciate the new explore view a lot with all the features packed into it.

As for Frigate+, let me know if I'm missing something (just installed 0.15 beta 1 today) but I do feel like it was much easier to skim through the snapshots before in the dedicated view, where the color coded bounding boxes would immediately tell me where to look, what was detected and the score. It was quick to spot a false positive, or badly placed bounding box, for example.

In the new explore view I get something close to the old view when in Unfiltered Grid, but I need to look at the small icon to know what was detected and the score, and the thumbnail doesn't show the bounding box. It can happen that, for example, the icon says " 85%", there's a dog in the thumbnail, but it was actually a false positive in another part of the image. I would not suspect this false positive and would not open it to verify.

I do miss some color coding, bigger confidence score/percentage and immediately viewable bounding boxes. The missing button as an indication that an imagine has been sent is also hard to quickly spot.

I feel like this UI serves its main purpose, but not the Frigate+ purpose, as it's a lot harder to mentally filter through the thumbnails. Perhaps a dedicated Frigate+ view is still valid.

Thanks a lot! Happy to continue testing the 0.15 betas.

[hawkeye217]

In addition to the score filter, there have been some small changes to recent dev versions that will improve the Frigate+ workflow, including a "Submitted to Frigate+" filter for snapshots. After filtering, open up an item in Explore's grid view and navigate to the Snapshot tab where you'll see the familiar snapshot with the bounding box. The left/right arrow keys on your keyboard will move between each tracked object's snapshot for quick review and submission to Frigate+.

This filter is available in the latest dev build and will also be in the next beta.

[almeidainf]

I did test the left/right arrow keys to move between snapshots but thought it was slow, as before I would just visually skim through many of those while scrolling. The new filter will help, and using other filters more aggressively too. I guess it takes some adjusting to the new workflow.

Will keep an eye out for the next beta. Thanks!

[MaBeniu]

great to see new release!!!
can you help to understand where I need to add ffmpeg path??? I dont really understand this:

but go2rtc exec streams will need to add the full path for ffmpeg which in most cases will be /usr/lib/ffmpeg/7.0/bin/ffmpeg

is it in my main config under each camera stream I need to add?
I believe the follow error is related;

2024-10-29 18:13:08.354976804  18:13:08.354 ERR [exec] error="exec: \"ffmpeg\": executable file not found in $PATH" url="exec:ffmpeg -hide_banner -v error -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i 

[MaBeniu]

ok now I get the following error for camera "durys" (reolink doorbell)

2024-10-29 18:32:26.848477998  [2024-10-29 18:32:26] ffmpeg.durys.detect            ERROR   : [h264_qsv @ 0x60bc4a61af80] video_get_buffer: image parameters invalid

2024-10-29 18:32:26.848503332  [2024-10-29 18:32:26] ffmpeg.durys.detect            ERROR   : [h264_qsv @ 0x60bc4a61af80] get_buffer() failed

2024-10-29 18:32:26.848530093  [2024-10-29 18:32:26] ffmpeg.durys.detect            ERROR   : [vist#0:0/h264 @ 0x60bc4a5a28c0] [dec:h264_qsv @ 0x60bc4a661480] Decoding error: Invalid argument

I double checked streaming from doorbell is using h264

[NickM-27]

try using preset-vaapi for that camera

[MaBeniu]

I changed global ffmpeg setting and error is gone.
Could you help and edit "durys" camera config for me? sorry for being so dumb

[NickM-27]

if there are no errors then there should be no issue

[MaBeniu]

anyway big thanks Nick. You are the best!!!
super great job on keeping the updates and development. will go now to test what's new

[tbrasser]

Hmm, after fixing the birdseye restream ffmpeg config I now hit this:

2024-10-29 16:46:16.261783499 File "/opt/frigate/frigate/util/config.py", line 32, in migrate_frigate_config 
2024-10-29 16:46:16.261793160 previous_version = str(config.get("version", "0.13")) 
2024-10-29 16:46:16.261794091 AttributeError: 'NoneType' object has no attribute 'get'

[NickM-27]

that would seem to indicate yaml is unable to load your config for some reason, perhaps it is invalid yaml

[tbrasser]

Fixed! My file was called frigate.yaml, and in addition frigate created an empty frigate.yml

now a new issue, all streams show perfectly in go2rtc, but not in frigate...

[NickM-27]

you should create a support discussion, but none of this seems related to the beta

[tbrasser]

That's weird because everything worked perfectly on 0.14.1

go2rtc:
  streams:
    tablet1: rtsp://10.9.8.137:8080/h264_pcm.sdp
    tablet2: rtsp://10.9.8.109:8080/h264_pcm.sdp
    tablet3: rtsp://10.9.8.64:8080/h264_pcm.sdp
    tablet4: rtsp://10.9.8.163:8080/h264_pcm.sdp
audio: # <- enable audio events for all camera
  enabled: true
snapshots: 
  enabled: true 
cameras: 
  tablet1: 
    mqtt: 
      timestamp: false 
      bounding_box: false 
      crop: true 
      quality: 100 
      height: 500 
    ffmpeg: 
      inputs: 
        - path: "rtsp://127.0.0.1:8554/tablet1?video&audio"
          input_args: preset-rtsp-restream
          roles: 
            - audio 
            - detect 
            - record
  tablet2: 
    mqtt: 
      timestamp: false 
      bounding_box: false
      crop: true 
      quality: 100 
      height: 500 
    ffmpeg: 
      inputs: 
        - path: "rtsp://127.0.0.1:8554/tablet2?video&audio" 
          input_args: preset-rtsp-restream 
          roles: 
            - record 
            - detect 
            - audio 
  tablet3: 
    mqtt: 
      timestamp: false 
      bounding_box: false 
      crop: true 
      quality: 100 
      height: 500 
    ffmpeg: 
      inputs: 
        - path: "rtsp://127.0.0.1:8554/tablet3?video&audio" 
          input_args: preset-rtsp-restream 
          roles: 
            - record 
            - detect 
            - audio 
  tablet4: 
    mqtt: 
      timestamp: false 
      bounding_box: false 
      crop: true 
      quality: 100 
      height: 500 
    ffmpeg: 
      inputs: 
        - path: "rtsp://127.0.0.1:8554/tablet4?video&audio" 
          input_args: preset-rtsp-restream 
          roles: 
            - record 
            - detect 
            - audio
detectors: 
  coral: 
    type: edgetpu 
    device: usb 
database: 
  path: /media/frigate/frigate.db 
birdseye: 
  enabled: true 
  restream: true 
  mode: motion 
motion: 
  threshold: 100 
  contour_area: 30 
  objects: 
    track: 
      - person 
      - face 
      - cat 
ffmpeg: 
  hwaccel_args: preset-intel-qsv-h264 
record: 
  enabled: true 
  expire_interval: 60 
  retain: 
    days: 1 
    mode: motion 
  alerts: 
    retain: 
      days: 7 
  detections: 
    retain: 
      days: 7 
version: 0.15-0

[tbrasser]

Fixed by setting ffmpeg section to:

ffmpeg:
  hwaccel_args: '""'

[Mincka]

Impressive update, congratulations!

I think I found a bug for the annotation offset.
You cannot "Apply" if you want to set it back to 0.

Otherwise, two suggestions:

1. Ability to zoom with mouse in the video of the Tracked Object Details view.
   Not a big issue since there's a shortcut to view it in History view, where you can zoom.

2. More responsiveness for the number of items shown in:

• /explore (always 10 currently)
  The previews become too small for low resolutions

• explore?labels=xxx (1 column for small screens, 4 columns for bigger screens).
  It should be more than 4 columns for larger resolutions (with a max height?), especially wide ones, otherwise it's just 4 giant snapshots per line.

[NickM-27]

Thanks for the report

1. we had this but a user found that it did not work well with the built in player controls and caused issues so it was removed.
2. on desktop you can already customize the number of columns shown using the settings

[hawkeye217]

I think I found a bug for the annotation offset.
You cannot "Apply" if you want to set it back to 0.

Thanks, fixed in #14667

[Mincka]

Great, thank you for the heads up and the fix!

[DanGarion]

Was there a breaking change? I did the update and tried to start Frigate and now I'm getting the following within the logs. I update almost every time there is one so I had whatever was the most recent before this morning.

2024-10-29 18:25:08.792682098  *************************************************************
2024-10-29 18:25:08.792694460  *************************************************************
2024-10-29 18:25:08.792700708  ***    Your config file is not valid!                     ***
2024-10-29 18:25:08.792711121  ***    Please check the docs at                           ***
2024-10-29 18:25:08.792769307  ***    https://docs.frigate.video/configuration/          ***
2024-10-29 18:25:08.792774314  *************************************************************
2024-10-29 18:25:08.792777835  *************************************************************
2024-10-29 18:25:08.792781403  ***    Config Validation Errors                           ***
2024-10-29 18:25:08.792784849  *************************************************************
2024-10-29 18:25:08.792857547  environment_vars: Value error, too many values to unpack (expected 2)
2024-10-29 18:25:08.792862642  *************************************************************
2024-10-29 18:25:08.792866013  ***    End Config Validation Errors                       ***
2024-10-29 18:25:08.792869332  *************************************************************

[hawkeye217]

This is a known issue that will be fixed for the next beta.

[DanGarion]

Ok, I will restore and ignore for now. :)

[Mincka]

I had this issue and that was because I had Frigate+ API key configured in both config file and as environment variable.
I just removed it from the configuration file.

[hexatrim]

Ok so semantic search is really cool.
There seems to be a keydown listener preventing the search box from receiving the Home and End keys though, wonder if that could be adjusted?
Using them to jump to the start/end or select a big chunk of text is too ingrained to unlearn.

[hawkeye217]

The Home/End keys are bound to the suggestions list that appears below when entering filters through the keyboard. It does seem reasonable to have the keys work in the input when you're typing a normal search term without any filters in the input, though. I'll put up a PR to add this to the next beta.

[hexatrim]

Ah so they are! I can see why they would be useful there too. Yeah that seems like a good solution, thanks :)

[bagobones]

Does Semantic Search detect intel iGPU support automatically or does a detector for OpenVINO need to be configured for ONIX support? If OpenVINO needs to be configured is there a way to specify which tasks to to which accelerator if I have a Coral for normal object detection already configured as well?

[NickM-27]

it will detect openvino automatically assuming you are using the main build and the GPU device is present. You can check this by using intel_gpu_top

[wilcomir]

Hello,
thanks for this release, it really is awesome.

I am fiddling a bit with semantic search and genai - I have an (old) nvidia card available on my setup, so I started messing with ollama.

My card is used also for stream encoding/deconding, and since it only has 4 GB of vram, I first started testing with moondream which does fit on GPU. All works as expected.
I then pumped up my VM, now I am at 48GB ram, 16 cores i9 14th gen, and started testing out llava 13b and 34b. Both models work via the command line, i.e. I can chat with them - they are a bit slow, but it works - within frigate though I press regenerate and nothing happens.

I suspect there might be some timeout happening somewhere - I did not have the time to test the command line with vision, just chat, but I left the system going on its own for quite some time and none of the new events have a caption. I also monitor CPU usage when I request a caption, it pins to 100% for some time and then goes back to idle, so something is happening.

Can anyone give any pointers as to how to debug/investigate this? Should I open an issue? Up to now it is not clear whether this is frigate timing out, ollama timing out, my machine shitting the bed or whatever, so I thought to ask here.

Again thanks for this awesome piece of software.

[wilcomir]

You are correct - I am testing llava not llama - my bad :) currently on llava:7b and it does work albeit it is a bit hit and miss unfortunately, meaning not all events are getting a caption.
I kind of get lots of events as I have indoor cams & two cats, and when everyone is outside of the house frigate is actively monitoring them.

[hawkeye217]

You'll want to check out the docs for using Ollama: https://deploy-preview-13787--frigate-docs.netlify.app/configuration/genai/#ollama

Parallel requests come with some caveats, and multi-modal parallel requests are currently not supported by Ollama. Depending on your hardware and the number of requests made to the Ollama API, these limitations may prevent Ollama from being an optimal solution for many users.

[wilcomir]

Thanks @hawkeye217 - you are correct I most definitely need to fiddle with the options there. Perhaps requests are just dropped because they come while the one before is still being processed.

I'll test it out and reply here with hopefully a solution, to help others who might find themselves in a similar predicament.

[hawkeye217]

Perhaps requests are just dropped because they come while the one before is still being processed.

That's most likely the case. Other users have seem the same thing. It's a limitation of the llava model (and any Ollama multi-modal model) at this point.

[hawkeye217]

FYI, we've updated the Ollama genai docs to include some Ollama environment variables you can set to improve the experience of using Ollama with Frigate. If you can, test it and let us know how it's working for you.

[chrisuthe]

Not sure what it means yet, but just had 15b1 crash two days in a row with logs saying it's out of space for /dev/shm/log which obviously got changed a bit for this release. I did not get a notification SHM was too small which is why I bring it up here. I was setting it to 512mb of memory backed volume, upped it to 1024 for now. Is there a way I run the calculation manually to see if what was calculated really was not enough or if I missed the warning to make it bigger?

[tpjanssen]

@NickM-27 when will beta 2 will be released?

[chrisuthe]

the logs have been adjusted for the next beta, but in general there is a known issue that only some users are seeing that is causing file descriptors to get stuck in /dev/shm when the frame is deleted

Awesome, I will not attempt further digging unless I can be helpful to your work, let me know. Until then I'll just bounce the container when needed. Appreciate the note!!

[hawkeye217]

@NickM-27 when will beta 2 will be released?

There is no time frame available. We have a number of bugs still to work through.

[NickM-27]

if you want you can use crzynik/frigate as your docker image and it will have logs about shm transactions. Once lsof +D /dev/shm | grep deleted returns a list of items / you see SHM usage increasing, getting that output and the shm logs will tell us where something might have gone wrong.

[chrisuthe]

if you want you can use crzynik/frigate as your docker image and it will have logs about shm transactions. Once lsof +D /dev/shm | grep deleted returns a list of items / you see SHM usage increasing, getting that output and the shm logs will tell us where something might have gone wrong.

I can do that if it'd help, not a problem.

[Blueion76]

Are you able to add Groq to this Frigate version by chance? I tried using it with the OpenAI provider and I get the following:

2024-11-12 22:42:44.019595140  [2024-11-12 22:42:44] httpx                          INFO    : HTTP Request: POST https://api.groq.com/openai/v1/chat/completions "HTTP/1.1 400 Bad Request"

2024-11-12 22:42:44.022721908    File "/opt/frigate/frigate/genai/openai.py", line 30, in _send

2024-11-12 22:42:44.022724422    File "/usr/local/lib/python3.9/dist-packages/openai/_utils/_utils.py", line 274, in wrapper

2024-11-12 22:42:44.022726687    File "/usr/local/lib/python3.9/dist-packages/openai/resources/chat/completions.py", line 742, in create

2024-11-12 22:42:44.022741334    File "/usr/local/lib/python3.9/dist-packages/openai/_base_client.py", line 1277, in post

2024-11-12 22:42:44.022744160    File "/usr/local/lib/python3.9/dist-packages/openai/_base_client.py", line 954, in request

2024-11-12 22:42:44.022746354    File "/usr/local/lib/python3.9/dist-packages/openai/_base_client.py", line 1058, in _request

2024-11-12 22:42:44.022751303  openai.BadRequestError: Error code: 400 - {'error': {'message': "'messages.0' : for 'role:user' the following must be satisfied[('messages.0.content' : one of the following must be satisfied[('messages.0.content' : value must be a string) OR ('messages.0.content.1' : one of the following must be satisfied[('messages.0.content.1' : value must be an object) OR ('messages.0.content.1' : value must be an object)])])]", 'type': 'invalid_request_error'}}

[bagobones]

There are a few open ai api compatible projects out there some acting as proxies for other services to enhance results, generalizing to allow custom URL etc would be useful for those. an example would be https://github.com/codelion/optillm

Maybe call it openai_custom an and let the base URL be set and have a few other option parameters?

[hawkeye217]

There is a PR up to make this possible, once merged you'll just need to set the base_url for the OpenAI provider to https://api.x.ai/v1 and use your xAI API key that you generated on https://console.x.ai.

[Blueion76]

There is a PR up to make this possible, once merged you'll just need to set the base_url for the OpenAI provider to https://api.x.ai/v1 and use your xAI API key that you generated on https://console.x.ai.

Thanks! Groq is actually different from Grok. Would it work the same since it's OpenAI compatible?

[hawkeye217]

Sorry, I misread your original message. Any AI that fully supports the OpenAI API should work, though we'll only officially support the providers listed in the docs.

[jahkas]

Hi all,

Does anyone know why I can't playback my video recording from the Media saved recording?

Thank you

[NickM-27]

this is a known bug in HA 2024.11

[jahkas]

Great., thank you

[bozzio83]

Hello

I've increased my dev/shm from 256 to 512. Frigate ran fine for a couple days and I got the message about running out of space. So I upped to 1024 and the message went away for a couple days until today.. how high should this be set? I have 5 cameras.

[NickM-27]

see #14653 (comment)

[bozzio83]

Thanks Nick I did see that. I'm just not very good with this stuff so changing an image I would probably break something in my setup. I was just curious if there was a quick fix or recommended number. I'll wait for beta 2 patiently. Thanks for all you guys do on this great project

[bagobones]

So I am not sure exactly what is happing but on a LARGE number of my life cycle reviews the boxes and images don't come close.

I was trouble shotting some false positives and found the boxes not clearly attached to anything. In others it looks like the boxes match the historic path of the object but the frames shown don't even include the object or it isn't even close.

If I had to guess this got worse when I increased my detect FPS from 5 to 10 FPS to try and capture faster objects better however I think it has over all made tracking worse for all object?

[hawkeye217]

Have you adjusted the annotation offset?

[bagobones]

The object isn't even still visible to adjust to in some of the shots, and some of the objects are not fast moving objects.

[bm55]

I use nodered to mange notification frequency via HomeAssistant which includes following action to play the video clip related to the notification object. I am curious if similar link can be created so it will work like the native notification on 0.15 beta. I like how the native notification plays the recording inside the frigate explorer's review page
"actions": [
{
"action": "PLAY_CLIP",
"title": "Play Clip",
"uri": "https://frigate.example.org/api/events/{{payload.after.id}}/clip.mp4",
"icon": "sfsymbols:play.circle.fill"
},

[bagobones]

Looks like running out of SHM is still an issue, I am at a shm_size: "512mb" currently and things where running fine for a bit but I was doing lots of configs and restarts. left it for a day and sat down to find the UI hung and this going on again in the docker logs.

2024-11-14 20:42:04.937346417  [2024-11-14 20:42:04] frigate.record.maintainer      WARNING : Too many unprocessed recording segments in cache for front_yard_cam. This likely indicates an issue with the detect stream, keeping the 6 most recent segments out of 7 and discarding the rest...

s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device

s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device

s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device

s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device

s6-log: warning: unable to write to /dev/shm/logs/nginx/current: No space left on device

[bagobones]

We are aware of an issue that only affects some users and have a potential fix that is being tested

Is there a specific cause you need more confirmation on or is it more it just happens over time?

[hawkeye217]

We believe we've found the bug and fixed it, feel free to pull the latest dev and test.

[bagobones]

We believe we've found the bug and fixed it, feel free to pull the latest dev and test.

Is it a condition that can be forced? How far off are we from a beta 2?

[bagobones]

We believe we've found the bug and fixed it, feel free to pull the latest dev and test.

How do I pull the dev docker? is it just something simple like adding :dev to my docker config?

I have increased from 256 to 512 now to 1024 and it still happens every 24-48hrs depending on the last time frigate was restarted.

[NickM-27]

just use the latest image https://github.com/users/blakeblackshear/packages/container/package/frigate

[tpjanssen]

Somebody else noticed that cars where accidently detected as Amazon or DPD? I've now had this on two occasions, on two different cameras. Looking in the debug screen gives a score of 96+%, the score the cars normally would have when being detected as actual cars. Restarting Frigate solves this; after the restart the cars were detected properly.

[tpjanssen]

OK. I will enable alerts for cars on my camera's for debugging. Do I have to explicitly add all sub logos for this as well?

Is there a way to delete multiple events in the explore section with a single go? Ctrl-A doesn't work, while it does so in the review section.

[hawkeye217]

No, you can't multi-delete tracked objects in Explore. To be clear, deleting in Explore only removes the snapshot, any saved embeddings, and any associated object lifecycle entries. Recorded footage of tracked objects in History view will not be deleted.

[tpjanssen]

This is exactly what I want. Will this do the job?

snapshots:
  retain:
    objects:
      car: 2

[bagobones]

Somebody else noticed that cars where accidently detected as Amazon or DPD?

I noticed the same, in my case I had to go through several of the reviews to see it was delivery trucks driving past parked cars and the logo migrating to the parked car.

[tpjanssen]

@NickM-27 saw your last PR. No action needed, unless it happens again in beta 2?

[bozzio83]

I'm getting a lot of hanging car tracking.. like 45min to hour long clips of parked cars in driveway.

objects:
  track:
    - person
    - face
    - dog
    - car
    - amazon
    - fedex
    - ups
    - package
    - purolator
    - bird
detect:
  stationary:
    interval: 70
    threshold: 70

A car will drive by which will trigger the detection but then it focuses on another parked car in the background and stays there. How could I eliminate this?

[bozzio83]

No I don't have cars in the road set for review.. Does tracking cause a big strain on the gpu? The reason I saw this was bottom right of my frigate it said driveway camera using high fffmpg gpu usage, so when I looked at the object page one of these cars were being tracked..

[NickM-27]

I think you are mixing things up. The bottom right will only have a message that ffmpeg is using high CPU. And, that has nothing to do with tracking

[bozzio83]

It's quite probable I'm mixing things up lol. Well thanks for taking the time to answer my questions. I'll leave it alone if all is working as intended.

[bozzio83]

Sorry to bring this up again.. But this is what's happening. I'll attach pictures. It's been 4 hours since this started..

If it really doesn't matter that its still processing it even though nothing has moved in hours fine. I just wanted to show what I meant earlier.

[NickM-27]

Yeah that's correct

[J4yDubsHome]

What's the normal cadence (if there is one) for beta releases?

[NickM-27]

there isn't one, just when they're ready

[tpjanssen]

Is there a way to configure retention in such away to make an object go away from the explore section? It seems I can make the snapshot itself go away, but I think the object itself only goes when the actual footage is removed from disk?

[tpjanssen]

That looks the same as my config. In 0.14 we could this:

record:
  events:
    retain:
      objects:
        cat: 3

That cause them to complete remove cats from the event list, while still possible to have 24/7 recordings of that period.

[NickM-27]

You can do the same by setting the detection record retention to 3 days

[tpjanssen]

You can do the same by setting the detection record retention to 3 days

Also on object level? Can you give an example in yaml?

[NickM-27]

Detections are inherently "less important", there is no object specific config for that, and that is by design

[tpjanssen]

Meaning that there is no way (yet) to make a specific object type go away in the explore section while keeping 24/7 recordings of that period?

[bagobones]

/explore API or Custom Events should NOT have the "This is a" or "This is not a" buttons for submitting to frigate+ as the labels do not match anything in frigate+.

Instead they should just have a "Submit" button.

[bagobones]

Also any chance we can get access to download the CLEAN snapshot if that feature is enabled?

I am currently doing some testing outside of frigate and it is very handy there is a download snapshot button but it is the one with the bounding boxes, if I have clean snapshots also enabled I would love to be able to download those from the UI.

[ilkkaolavas]

My frigate instance is running well but randomly after 12-18 hours it crashes.
Config file unchanged from 0.14.1 which worked well without problems. Just upgraded with same conf:

detectors:
coral:
type: edgetpu
device: usb
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
go2rtc:
streams:
jarvelle_hires:
- rtsp://xxxxx:{xxxxx}@192.168.x.x/stream1
....

---

Processor: Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz
Coral USB Accelerator
Description: Ubuntu 24.04.1 LTS
Running Frigate on Docker

Logs:
unknown 2024-11-17 02:00:03 unknown Fatal Python error: Bus error unknown 2024-11-17 06:38:08 unknown 24-11-17 02:00:03.018929627 unknown 2024-11-17 02:00:03 unknown Thread 0x00007c28ef4006c0 (most recent call first): unknown 2024-11-17 02:00:03 unknown File "/usr/lib/python3.9/threading.py", line 312 in wait unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/multiprocessing/queues.py", line 233 in _feed unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 892 in run unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap unknown 2024-11-17 06:38:07 unknown 24-11-17 02:00:05.024036279 unknown 2024-11-17 02:00:05 unknown Current thread 0x00007c28eea006c0 (most recent call first): unknown 2024-11-17 02:00:05 unknown File "/opt/frigate/frigate/video.py", line 119 in capture_frames unknown 2024-11-17 02:00:05 unknown File "/opt/frigate/frigate/video.py", line 372 in run unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap unknown 2024-11-17 06:38:07 unknown 24-11-17 02:00:05.024041908 unknown 2024-11-17 02:00:05 unknown Thread 0x00007c28ee0006c0 (most recent call first): unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 261 in wait unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/multiprocessing/synchronize.py", line 349 in wait unknown 2024-11-17 02:00:05 unknown File "/opt/frigate/frigate/video.py", line 208 in run unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap unknown 2024-11-17 06:38:07 unknown 24-11-17 02:00:05.024067269 unknown 2024-11-17 02:00:05 unknown Thread 0x00007c28ed0006c0 (most recent call first): unknown 2024-11-17 02:00:05 unknown File "/opt/frigate/frigate/log.py", line 92 in run unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 954 in _bootstrap_inner unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 912 in _bootstrap unknown 2024-11-17 06:38:07 unknown 24-11-17 02:00:05.024084066 unknown 2024-11-17 02:00:05 unknown Thread 0x00007c2944955740 (most recent call first): unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 1049 in _wait_for_tstate_lock unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/threading.py", line 1033 in join unknown 2024-11-17 02:00:05 unknown File "/opt/frigate/frigate/video.py", line 412 in capture_camera unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/multiprocessing/process.py", line 108 in run unknown 2024-11-17 02:00:05 unknown File "/opt/frigate/frigate/util/process.py", line 41 in run_wrapper unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/multiprocessing/process.py", line 315 in _bootstrap unknown 2024-11-17 02:00:05 unknown File "/usr/lib/python3.9/multiprocessing/popen_fork.py", line 71 in _launch unknown 2024-11-17 02:00:05 unknown etc. More on logfile:
frigate-logs.txt

[hawkeye217]

unknown 2024-11-17 02:00:03 unknown Fatal Python error: Bus error

You need to increase your shm_size. This is due to the changes to the shm system as discussed in the 0.15 beta 1 release notes. Changes have been made to improve this further that will be available in beta 2.

[ofcedwards]

For semantic search, is the thumbnail found in the explore view what's fed into the model? On desktop, I've noticed that these thumbnails are pretty small and the quality isn't great, even compared to a cropped 300x300 image at 70 quality that I can get from the snapshot api.

I think the thumbnails are fine for general use and it's nice that they're lightweight, but if they're fed into the model I'd be willing to have larger files and higher resource usage. Assuming that a higher quality thumbnail actually produces better results with semantic search, of course.

Either way, I've really been enjoying the quality of life upgrades in 0.15, especially semantic search!

[NickM-27]

The semantic search embedding are generated from a direct crop of the decoded frame, which is also before the snapshots are compressed and saved.

[ofcedwards]

Thanks! I kind of figured it wasn't the thumbnails since it's reading text pretty well, but just wanted to make sure.