Real-time inference?

[FedericoVasile1]

Hi, I congrats for your work.
I would like to know whether at inference time is it possible to use this model as part of a framework that performs the online action detection task? (i.e. let's suppose I have an input stream video, is the model able to compute optical-flow features as the frames arrive, with real-time speed ?)

Thank you!

[vitusleng]

DIS optical flow has already been integrated into opencv, you can give it a try, for images not too large, the realtime performance seems pretty good.

[tikroeger]

The operation points as described in the paper show that real-time performance is easily within range of the existing code. But it largely depends on the required accuracy. The OpenCV implementation (which I have not written or tested) seems to achieve that as well. The main computational bottleneck of this method in general (this repo or OpenCV) is the computation of X and Y gradients for both images. If the gradients are computed only once for each frame, as opposed to once for a frame-pair as in the current code, large speed-ups are possible. The spatial gradient code itself can easily be sped-up as well. Relatedly, this algorithm has repeatedly been re-implement by corporate partners, occasionally with GPU usage: One the worlds best well known video/photo-story app is using this algorithm for live virtual augmentation of video-feeds.

…

On Sun, Jan 24, 2021 at 01:35 vitusleng ***@***.***> wrote: DIS optical flow has already been integrated into opencv, you can have a try, for images no too large, the realtime performance seems pretty good. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIVIVXWAXEOFVWCURAFMADS3PSXDANCNFSM4PSXMNLQ> .