Video-MAE-Quantized: Sports and human action recognition in videos

Video MAE (Masked Auto Encoder) is a network for doing video classification that uses the ViT (Vision Transformer) backbone.

This is based on the implementation of Video-MAE-Quantized found here. This repository contains scripts for optimized on-device export suitable to run on Qualcomm® devices. More details on model performance accross various devices, can be found here.

Sign up to start using Qualcomm AI Hub and run these models on a hosted Qualcomm® device.

Example & Usage

Install the package via pip:

pip install "qai-hub-models[video-mae-quantized]"

Once installed, run the following simple CLI demo:

python -m qai_hub_models.models.video_mae_quantized.demo

More details on the CLI tool can be found with the --help option. See demo.py for sample usage of the model including pre/post processing scripts. Please refer to our general instructions on using models for more usage instructions.

Export for on-device deployment

This repository contains export scripts that produce a model optimized for on-device deployment. This can be run as follows:

python -m qai_hub_models.models.video_mae_quantized.export

Additional options are documented with the --help option. Note that the above script requires access to Deployment instructions for Qualcomm® AI Hub.

License

• The license for the original implementation of Video-MAE-Quantized can be found here.
• The license for the compiled assets for on-device deployment can be found here

References

• Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training
• Source Model Implementation

Community

• Join our AI Hub Slack community to collaborate, post questions and learn more about on-device AI.
• For questions or feedback please reach out to us.