axinc-ai · Shysto · Apr 9, 2024 · Apr 9, 2024 · Apr 15, 2024
diff --git a/README.md b/README.md
@@ -13,9 +13,10 @@ The collection of pre-trained, state-of-the-art AI models.
 [ailia MODELS tutorial 日本語版](TUTORIAL_jp.md)
 
 # Supported models
-320 models as of March 29th, 2024
+321 models as of April 9th, 2024
 
 # Latest update
+- 2024.04.09 Add fuseformer
 - 2024.03.29 Add falcon-adapter-network-packet, pyannote-audio
 - 2024.03.08 Add did_m3d, damo_yolo, deep-music-enhancer, tusimple-DUC
 - 2024.03.05 Add polylanenet, rnnoise
@@ -570,6 +571,12 @@ The collection of pre-trained, state-of-the-art AI models.
 | [<img src="vehicle_recognition/vehicle-attributes-recognition-barrier/demo.png" width=64px>](/vehicle_recognition/vehicle-attributes-recognition-barrier/) |[vehicle-attributes-recognition-barrier](/vehicle_recognition/vehicle-attributes-recognition-barrier) | [vehicle-attributes-recognition-barrier-0042](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/vehicle-attributes-recognition-barrier-0042) | OpenVINO | 1.2.5 and later | [EN](https://medium.com/axinc-ai/vehicleattributerecognitionbarrier-a-machine-learning-model-for-detecting-car-attributes-fe8fda7649ff) [JP](https://medium.com/axinc/vehicleattributerecognitionbarrier-%E8%BB%8A%E3%81%AE%E5%B1%9E%E6%80%A7%E3%82%92%E6%A4%9C%E5%87%BA%E3%81%99%E3%82%8B%E6%A9%9F%E6%A2%B0%E5%AD%A6%E7%BF%92%E3%83%A2%E3%83%87%E3%83%AB-ee26d1a3e00b) |
 | [<img src="vehicle_recognition/vehicle-license-plate-detection-barrier/output.png" width=128px>](vehicle_recognition/vehicle-license-plate-detection-barrier/) | [vehicle-license-plate-detection-barrier](/vehicle_recognition/vehicle-license-plate-detection-barrier/) | [vehicle-license-plate-detection-barrier-0106](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/intel/vehicle-license-plate-detection-barrier-0106) | OpenVINO | 1.2.5 and later | |
 
+## Video inpainting
+
+| | Model | Reference | Exported From | Supported Ailia Version | Blog |
+|:-----------|------------:|:------------:|:------------:|:------------:|:------------:|
+| [<img src="video_inpainting/fuseformer/misc/bmx_trees_output.gif" width=128px>](video_inpainting/fuseformer) |[fuseformer](video_inpainting/fuseformer) | [FuseFormer](https://github.com/ruiliu-ai/FuseFormer) | PyTorch | Not supported yet | |
+
 ## Commercial model
 
 | Model | Reference | Exported From | Supported Ailia Version | Blog |

diff --git a/scripts/download_all_models.sh b/scripts/download_all_models.sh
@@ -323,4 +323,5 @@ cd ../../text_recognition/easyocr; python3 easyocr.py ${OPTION}
 cd ../../text_recognition/ndlocr_text_recognition; python3 ndlocr_text_recognition.py ${OPTION}
 cd ../../time_series_forecasting/informer2020; python3 informer2020.py ${OPTION}
 cd ../../vehicle_recognition/vehicle-attributes-recognition-barrier; python3 vehicle-attributes-recognition-barrier.py ${OPTION}
-cd ../../vehicle_recognition/vehicle-license-plate-detection-barrier; python3 vehicle-license-plate-detection-barrier.py ${OPTION}
+cd ../../vehicle_recognition/vehicle-license-plate-detection-barrier; python3 vehicle-license-plate-detection-barrier.py ${OPTION}
+cd ../../video_inpainting/fuseformer; python3 fuseformer.py ${OPTION}
diff --git a/video_inpainting/fuseformer/README.md b/video_inpainting/fuseformer/README.md
@@ -0,0 +1,62 @@
+# FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting
+
+## Input
+
+![Input video](misc/bmx_trees.gif)
+
+(Video from https://github.com/sczhou/ProPainter/tree/main/inputs/object_removal/bmx-trees)
+
+![Input mask example](bmx_trees_mask/00000.png)
+
+(Image from https://github.com/sczhou/ProPainter/blob/main/inputs/object_removal/bmx-trees_mask/00000.png)
+
+## Output
+
+![Output video](misc/bmx_trees_output.gif)
+
+## Usage
+Automatically downloads the onnx and prototxt files on the first run.
+It is necessary to be connected to the Internet while downloading.
+
+For the sample video,
+``` bash
+$ python3 fuseformer.py
+```
+
+If you want to specify the input video or input frames, put the video path or the input frames folder after the `--video` option. <br>
+If you want to specify the input masked frames, put the input masked frames folder after the `--mask` option. <br>
+You can use `--savepath` option to change the name of the output folder to save.
+```bash
+$ python3 fuseformer.py --video VIDEO_OR_FRAMES_PATH --mask MASKED_FRAMES_PATH --savepath SAVE_FOLDER_PATH
+```
+
+You can use `--width` and `--height`, or the `--resize_ratio` options to change the video resolution.
+```bash
+$ python3 fuseformer.py --width 432 --height 240
+```
+
+By adding the `--save_frames` option, you can save the output video frames.
+```bash
+$ python3 fuseformer.py --save_frames
+```
+
+By adding the `--onnx` option, you can run the inference using OnnxRuntime.
+```bash
+$ python3 fuseformer.py --onnx
+```
+
+## Reference
+
+- [FuseFormer](https://github.com/ruiliu-ai/FuseFormer)
+
+## Framework
+
+Pytorch
+
+## Model Format
+
+ONNX opset = 18
+
+## Netron
+
+- [fuseformer.onnx.prototxt](https://netron.app/?url=https://storage.googleapis.com/ailia-models/fuseformer/fuseformer.onnx.prototxt)
diff --git a/video_inpainting/fuseformer/bmx_trees.mp4 b/video_inpainting/fuseformer/bmx_trees.mp4
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00000.png b/video_inpainting/fuseformer/bmx_trees_mask/00000.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00001.png b/video_inpainting/fuseformer/bmx_trees_mask/00001.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00002.png b/video_inpainting/fuseformer/bmx_trees_mask/00002.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00003.png b/video_inpainting/fuseformer/bmx_trees_mask/00003.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00004.png b/video_inpainting/fuseformer/bmx_trees_mask/00004.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00005.png b/video_inpainting/fuseformer/bmx_trees_mask/00005.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00006.png b/video_inpainting/fuseformer/bmx_trees_mask/00006.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00007.png b/video_inpainting/fuseformer/bmx_trees_mask/00007.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00008.png b/video_inpainting/fuseformer/bmx_trees_mask/00008.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00009.png b/video_inpainting/fuseformer/bmx_trees_mask/00009.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00010.png b/video_inpainting/fuseformer/bmx_trees_mask/00010.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00011.png b/video_inpainting/fuseformer/bmx_trees_mask/00011.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00012.png b/video_inpainting/fuseformer/bmx_trees_mask/00012.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00013.png b/video_inpainting/fuseformer/bmx_trees_mask/00013.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00014.png b/video_inpainting/fuseformer/bmx_trees_mask/00014.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00015.png b/video_inpainting/fuseformer/bmx_trees_mask/00015.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00016.png b/video_inpainting/fuseformer/bmx_trees_mask/00016.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00017.png b/video_inpainting/fuseformer/bmx_trees_mask/00017.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00018.png b/video_inpainting/fuseformer/bmx_trees_mask/00018.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00019.png b/video_inpainting/fuseformer/bmx_trees_mask/00019.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00020.png b/video_inpainting/fuseformer/bmx_trees_mask/00020.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00021.png b/video_inpainting/fuseformer/bmx_trees_mask/00021.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00022.png b/video_inpainting/fuseformer/bmx_trees_mask/00022.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00023.png b/video_inpainting/fuseformer/bmx_trees_mask/00023.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00024.png b/video_inpainting/fuseformer/bmx_trees_mask/00024.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00025.png b/video_inpainting/fuseformer/bmx_trees_mask/00025.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00026.png b/video_inpainting/fuseformer/bmx_trees_mask/00026.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00027.png b/video_inpainting/fuseformer/bmx_trees_mask/00027.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00028.png b/video_inpainting/fuseformer/bmx_trees_mask/00028.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00029.png b/video_inpainting/fuseformer/bmx_trees_mask/00029.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00030.png b/video_inpainting/fuseformer/bmx_trees_mask/00030.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00031.png b/video_inpainting/fuseformer/bmx_trees_mask/00031.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00032.png b/video_inpainting/fuseformer/bmx_trees_mask/00032.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00033.png b/video_inpainting/fuseformer/bmx_trees_mask/00033.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00034.png b/video_inpainting/fuseformer/bmx_trees_mask/00034.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00035.png b/video_inpainting/fuseformer/bmx_trees_mask/00035.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00036.png b/video_inpainting/fuseformer/bmx_trees_mask/00036.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00037.png b/video_inpainting/fuseformer/bmx_trees_mask/00037.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00038.png b/video_inpainting/fuseformer/bmx_trees_mask/00038.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00039.png b/video_inpainting/fuseformer/bmx_trees_mask/00039.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00040.png b/video_inpainting/fuseformer/bmx_trees_mask/00040.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00041.png b/video_inpainting/fuseformer/bmx_trees_mask/00041.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00042.png b/video_inpainting/fuseformer/bmx_trees_mask/00042.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00043.png b/video_inpainting/fuseformer/bmx_trees_mask/00043.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00044.png b/video_inpainting/fuseformer/bmx_trees_mask/00044.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00045.png b/video_inpainting/fuseformer/bmx_trees_mask/00045.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00046.png b/video_inpainting/fuseformer/bmx_trees_mask/00046.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00047.png b/video_inpainting/fuseformer/bmx_trees_mask/00047.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00048.png b/video_inpainting/fuseformer/bmx_trees_mask/00048.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00049.png b/video_inpainting/fuseformer/bmx_trees_mask/00049.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00050.png b/video_inpainting/fuseformer/bmx_trees_mask/00050.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00051.png b/video_inpainting/fuseformer/bmx_trees_mask/00051.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00052.png b/video_inpainting/fuseformer/bmx_trees_mask/00052.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00053.png b/video_inpainting/fuseformer/bmx_trees_mask/00053.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00054.png b/video_inpainting/fuseformer/bmx_trees_mask/00054.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00055.png b/video_inpainting/fuseformer/bmx_trees_mask/00055.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00056.png b/video_inpainting/fuseformer/bmx_trees_mask/00056.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00057.png b/video_inpainting/fuseformer/bmx_trees_mask/00057.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00058.png b/video_inpainting/fuseformer/bmx_trees_mask/00058.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00059.png b/video_inpainting/fuseformer/bmx_trees_mask/00059.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00060.png b/video_inpainting/fuseformer/bmx_trees_mask/00060.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00061.png b/video_inpainting/fuseformer/bmx_trees_mask/00061.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00062.png b/video_inpainting/fuseformer/bmx_trees_mask/00062.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00063.png b/video_inpainting/fuseformer/bmx_trees_mask/00063.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00064.png b/video_inpainting/fuseformer/bmx_trees_mask/00064.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00065.png b/video_inpainting/fuseformer/bmx_trees_mask/00065.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00066.png b/video_inpainting/fuseformer/bmx_trees_mask/00066.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00067.png b/video_inpainting/fuseformer/bmx_trees_mask/00067.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00068.png b/video_inpainting/fuseformer/bmx_trees_mask/00068.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00069.png b/video_inpainting/fuseformer/bmx_trees_mask/00069.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00070.png b/video_inpainting/fuseformer/bmx_trees_mask/00070.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00071.png b/video_inpainting/fuseformer/bmx_trees_mask/00071.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00072.png b/video_inpainting/fuseformer/bmx_trees_mask/00072.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00073.png b/video_inpainting/fuseformer/bmx_trees_mask/00073.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00074.png b/video_inpainting/fuseformer/bmx_trees_mask/00074.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00075.png b/video_inpainting/fuseformer/bmx_trees_mask/00075.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00076.png b/video_inpainting/fuseformer/bmx_trees_mask/00076.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00077.png b/video_inpainting/fuseformer/bmx_trees_mask/00077.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00078.png b/video_inpainting/fuseformer/bmx_trees_mask/00078.png
diff --git a/video_inpainting/fuseformer/bmx_trees_mask/00079.png b/video_inpainting/fuseformer/bmx_trees_mask/00079.png