Skip to content

Commit 0add946

Browse files
theAIGuysCodetheAIGuysCode
authored
Initial upload
1 parent b81c8cb commit 0add946

36 files changed

+47527
-0
lines changed

Diff for: ‎README.md

+308
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,308 @@
1+
# tensorflow-yolov4-tflite
2+
[![license](https://img.shields.io/github/license/mashape/apistatus.svg)](LICENSE)
3+
4+
YOLOv4, YOLOv4-tiny Implemented in Tensorflow 2.0.
5+
Convert YOLO v4, YOLOv3, YOLO tiny .weights to .pb, .tflite and trt format for tensorflow, tensorflow lite, tensorRT.
6+
<p align="center"><img src="data/helpers/demo.gif"\></p>
7+
8+
## Getting Started
9+
### Conda (Recommended)
10+
11+
```bash
12+
# Tensorflow CPU
13+
conda env create -f conda-cpu.yml
14+
conda activate yolov4-cpu
15+
16+
# Tensorflow GPU
17+
conda env create -f conda-gpu.yml
18+
conda activate yolov4-gpu
19+
```
20+
21+
### Pip
22+
```bash
23+
# TensorFlow CPU
24+
pip install -r requirements.txt
25+
26+
# TensorFlow GPU
27+
pip install -r requirements-gpu.txt
28+
```
29+
### Nvidia Driver (For GPU, if you are not using Conda Environment and haven't set up CUDA yet)
30+
Make sure to use CUDA Toolkit version 10.1 as it is the proper version for the TensorFlow version used in this repository.
31+
https://developer.nvidia.com/cuda-10.1-download-archive-update2
32+
33+
### Performance
34+
Check out how YOLOv4 compares to other object detection systems.
35+
36+
<p align="center"><img src="data/helpers/performance.png" width="640"\></p>
37+
38+
## Downloading Official Pre-trained Weights
39+
YOLOv4 comes pre-trained and able to detect 80 classes. For easy demo purposes we will use the pre-trained weights.
40+
Download pre-trained yolov4.weights file: https://drive.google.com/open?id=1cewMfusmPjYWbrnuJRuKhPMwRe_b9PaT
41+
42+
Copy and paste yolov4.weights from your downloads folder into the 'data' folder of this repository.
43+
44+
If you want to use yolov4-tiny.weights, a smaller model that is faster at running detections but less accurate, download file here: https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v4_pre/yolov4-tiny.weights
45+
46+
## Using Custom Trained YOLOv4 Weights
47+
<strong>Learn How To Train Custom YOLOv4 Weights here: https://www.youtube.com/watch?v=mmj3nxGT2YQ </strong>
48+
49+
Copy and paste your custom .weights file into the 'data' folder and copy and paste your custom .names into the 'data/classes/' folder.
50+
51+
The only change within the code you need to make in order for your custom model to work is on line 14 of 'core/config.py' file.
52+
Update the code to point at your custom .names file as seen below. (my custom .names file is called custom.names but yours might be named differently)
53+
<p align="center"><img src="data/helpers/custom_config.png" width="640"\></p>
54+
55+
<strong>Note:</strong> If you are using the pre-trained yolov4 then make sure that line 14 remains <strong>coco.names</strong>.
56+
57+
## YOLOv4 Using Tensorflow (tf, .pb model)
58+
To implement YOLOv4 using TensorFlow, first we convert the .weights into the corresponding TensorFlow model files and then run the model.
59+
```bash
60+
# Convert darknet weights to tensorflow
61+
## yolov4
62+
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4-416 --input_size 416 --model yolov4
63+
64+
# yolov4-tiny
65+
python save_model.py --weights ./data/yolov4-tiny.weights --output ./checkpoints/yolov4-tiny-416 --input_size 416 --model yolov4 --tiny
66+
67+
# custom yolov4
68+
python save_model.py --weights ./data/custom.weights --output ./checkpoints/custom-416 --input_size 416 --model yolov4
69+
70+
# Run yolov4 tensorflow model
71+
python detect.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --images ./data/images/kite.jpg
72+
73+
# Run yolov4-tiny tensorflow model
74+
python detect.py --weights ./checkpoints/yolov4-tiny-416 --size 416 --model yolov4 --images ./data/images/kite.jpg --tiny
75+
76+
# Run custom yolov4 tensorflow model
77+
python detect.py --weights ./checkpoints/custom-416 --size 416 --model yolov4 --images ./data/images/car.jpg
78+
79+
# Run yolov4 on video
80+
python detect_video.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --video ./data/video/video.mp4 --output ./detections/results.avi
81+
82+
# Run yolov4 on webcam
83+
python detect_video.py --weights ./checkpoints/yolov4-416 --size 416 --model yolov4 --video 0 --output ./detections/results.avi
84+
```
85+
If you want to run yolov3 or yolov3-tiny change ``--model yolov3`` and .weights file in above commands.
86+
87+
<strong>Note:</strong> You can also run the detector on multiple images at once by changing the --images flag like such ``--images "./data/images/kite.jpg, ./data/images/dog.jpg"``
88+
89+
### Result Image(s) (Regular TensorFlow)
90+
You can find the outputted image(s) showing the detections saved within the 'detections' folder.
91+
#### Pre-trained YOLOv4 Model Example
92+
<p align="center"><img src="data/helpers/result.png" width="640"\></p>
93+
94+
#### Custom YOLOv4 Model Example (see video link above to train this model)
95+
<p align="center"><img src="data/helpers/custom_result.png" width="640"\></p>
96+
97+
### Result Video
98+
Video saves wherever you point --output flag to. If you don't set the flag then your video will not be saved with detections on it.
99+
<p align="center"><img src="data/helpers/demo.gif"\></p>
100+
101+
## YOLOv4 Using TensorFlow Lite (.tflite model)
102+
Can also implement YOLOv4 using TensorFlow Lite. TensorFlow Lite is a much smaller model and perfect for mobile or edge devices (raspberry pi, etc).
103+
```bash
104+
# Save tf model for tflite converting
105+
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4-416 --input_size 416 --model yolov4 --framework tflite
106+
107+
# yolov4
108+
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416.tflite
109+
110+
# yolov4 quantize float16
111+
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416-fp16.tflite --quantize_mode float16
112+
113+
# yolov4 quantize int8
114+
python convert_tflite.py --weights ./checkpoints/yolov4-416 --output ./checkpoints/yolov4-416-int8.tflite --quantize_mode int8 --dataset ./coco_dataset/coco/val207.txt
115+
116+
# Run tflite model
117+
python detect.py --weights ./checkpoints/yolov4-416.tflite --size 416 --model yolov4 --images ./data/images/kite.jpg --framework tflite
118+
```
119+
### Result Image (TensorFlow Lite)
120+
You can find the outputted image(s) showing the detections saved within the 'detections' folder.
121+
#### TensorFlow Lite int8 Example
122+
<p align="center"><img src="data/helpers/result-int8.png" width="640"\></p>
123+
124+
Yolov4 and Yolov4-tiny int8 quantization have some issues. I will try to fix that. You can try Yolov3 and Yolov3-tiny int8 quantization
125+
126+
## YOLOv4 Using TensorRT
127+
Can also implement YOLOv4 using TensorFlow's TensorRT. TensorRT is a high-performance inference optimizer and runtime that can be used to perform inference in lower precision (FP16 and INT8) on GPUs. TensorRT can allow up to 8x higher performance than regular TensorFlow.
128+
```bash# yolov3
129+
python save_model.py --weights ./data/yolov3.weights --output ./checkpoints/yolov3.tf --input_size 416 --model yolov3
130+
python convert_trt.py --weights ./checkpoints/yolov3.tf --quantize_mode float16 --output ./checkpoints/yolov3-trt-fp16-416
131+
132+
# yolov3-tiny
133+
python save_model.py --weights ./data/yolov3-tiny.weights --output ./checkpoints/yolov3-tiny.tf --input_size 416 --tiny
134+
python convert_trt.py --weights ./checkpoints/yolov3-tiny.tf --quantize_mode float16 --output ./checkpoints/yolov3-tiny-trt-fp16-416
135+
136+
# yolov4
137+
python save_model.py --weights ./data/yolov4.weights --output ./checkpoints/yolov4.tf --input_size 416 --model yolov4
138+
python convert_trt.py --weights ./checkpoints/yolov4.tf --quantize_mode float16 --output ./checkpoints/yolov4-trt-fp16-416
139+
python detect.py --weights ./checkpoints/yolov4-trt-fp16-416 --model yolov4 --images ./data/images/kite.jpg --framework trt
140+
```
141+
142+
## Command Line Args Reference
143+
144+
```bash
145+
save_model.py:
146+
--weights: path to weights file
147+
(default: './data/yolov4.weights')
148+
--output: path to output
149+
(default: './checkpoints/yolov4-416')
150+
--[no]tiny: yolov4 or yolov4-tiny
151+
(default: 'False')
152+
--input_size: define input size of export model
153+
(default: 416)
154+
--framework: what framework to use (tf, trt, tflite)
155+
(default: tf)
156+
--model: yolov3 or yolov4
157+
(default: yolov4)
158+
159+
detect.py:
160+
--images: path to input images as a string with images separated by ","
161+
(default: './data/images/kite.jpg')
162+
--output: path to output folder
163+
(default: './detections/')
164+
--[no]tiny: yolov4 or yolov4-tiny
165+
(default: 'False')
166+
--weights: path to weights file
167+
(default: './checkpoints/yolov4-416')
168+
--framework: what framework to use (tf, trt, tflite)
169+
(default: tf)
170+
--model: yolov3 or yolov4
171+
(default: yolov4)
172+
--size: resize images to
173+
(default: 416)
174+
--iou: iou threshold
175+
(default: 0.45)
176+
--score: confidence threshold
177+
(default: 0.25)
178+
179+
detect_video.py:
180+
--video: path to input video (use 0 for webcam)
181+
(default: './data/video/video.mp4')
182+
--output: path to output video (remember to set right codec for given format. e.g. XVID for .avi)
183+
(default: None)
184+
--output_format: codec used in VideoWriter when saving video to file
185+
(default: 'XVID)
186+
--[no]tiny: yolov4 or yolov4-tiny
187+
(default: 'false')
188+
--weights: path to weights file
189+
(default: './checkpoints/yolov4-416')
190+
--framework: what framework to use (tf, trt, tflite)
191+
(default: tf)
192+
--model: yolov3 or yolov4
193+
(default: yolov4)
194+
--size: resize images to
195+
(default: 416)
196+
--iou: iou threshold
197+
(default: 0.45)
198+
--score: confidence threshold
199+
(default: 0.25)
200+
```
201+
202+
## Evaluate on COCO 2017 Dataset
203+
```bash
204+
# run script in /script/get_coco_dataset_2017.sh to download COCO 2017 Dataset
205+
# preprocess coco dataset
206+
cd data
207+
mkdir dataset
208+
cd ..
209+
cd scripts
210+
python coco_convert.py --input ./coco/annotations/instances_val2017.json --output val2017.pkl
211+
python coco_annotation.py --coco_path ./coco
212+
cd ..
213+
214+
# evaluate yolov4 model
215+
python evaluate.py --weights ./data/yolov4.weights
216+
cd mAP/extra
217+
python remove_space.py
218+
cd ..
219+
python main.py --output results_yolov4_tf
220+
```
221+
#### mAP50 on COCO 2017 Dataset
222+
223+
| Detection | 512x512 | 416x416 | 320x320 |
224+
|-------------|---------|---------|---------|
225+
| YoloV3 | 55.43 | 52.32 | |
226+
| YoloV4 | 61.96 | 57.33 | |
227+
228+
## Benchmark
229+
```bash
230+
python benchmarks.py --size 416 --model yolov4 --weights ./data/yolov4.weights
231+
```
232+
#### TensorRT performance
233+
234+
| YoloV4 416 images/s | FP32 | FP16 | INT8 |
235+
|---------------------|----------|----------|----------|
236+
| Batch size 1 | 55 | 116 | |
237+
| Batch size 8 | 70 | 152 | |
238+
239+
#### Tesla P100
240+
241+
| Detection | 512x512 | 416x416 | 320x320 |
242+
|-------------|---------|---------|---------|
243+
| YoloV3 FPS | 40.6 | 49.4 | 61.3 |
244+
| YoloV4 FPS | 33.4 | 41.7 | 50.0 |
245+
246+
#### Tesla K80
247+
248+
| Detection | 512x512 | 416x416 | 320x320 |
249+
|-------------|---------|---------|---------|
250+
| YoloV3 FPS | 10.8 | 12.9 | 17.6 |
251+
| YoloV4 FPS | 9.6 | 11.7 | 16.0 |
252+
253+
#### Tesla T4
254+
255+
| Detection | 512x512 | 416x416 | 320x320 |
256+
|-------------|---------|---------|---------|
257+
| YoloV3 FPS | 27.6 | 32.3 | 45.1 |
258+
| YoloV4 FPS | 24.0 | 30.3 | 40.1 |
259+
260+
#### Tesla P4
261+
262+
| Detection | 512x512 | 416x416 | 320x320 |
263+
|-------------|---------|---------|---------|
264+
| YoloV3 FPS | 20.2 | 24.2 | 31.2 |
265+
| YoloV4 FPS | 16.2 | 20.2 | 26.5 |
266+
267+
#### Macbook Pro 15 (2.3GHz i7)
268+
269+
| Detection | 512x512 | 416x416 | 320x320 |
270+
|-------------|---------|---------|---------|
271+
| YoloV3 FPS | | | |
272+
| YoloV4 FPS | | | |
273+
274+
## Traning your own model in TensorFlow
275+
```bash
276+
# Prepare your dataset
277+
# If you want to train from scratch:
278+
In config.py set FISRT_STAGE_EPOCHS=0
279+
# Run script:
280+
python train.py
281+
282+
# Transfer learning:
283+
python train.py --weights ./data/yolov4.weights
284+
```
285+
The training performance is not fully reproduced yet, so I recommended to use Alex's [Darknet](https://github.com/AlexeyAB/darknet) to train your own data, then convert the .weights to tensorflow or tflite.
286+
287+
<strong>Use this video to train your own model easily in Google Colab: https://www.youtube.com/watch?v=mmj3nxGT2YQ </strong>
288+
289+
### TODO
290+
* [x] Convert YOLOv4 to TensorRT
291+
* [x] YOLOv4 tflite on android
292+
* [ ] YOLOv4 tflite on ios
293+
* [x] Training code
294+
* [x] Update scale xy
295+
* [ ] ciou
296+
* [ ] Mosaic data augmentation
297+
* [x] Mish activation
298+
* [x] yolov4 tflite version
299+
* [x] yolov4 in8 tflite version for mobile
300+
301+
### References
302+
303+
* YOLOv4: Optimal Speed and Accuracy of Object Detection [YOLOv4](https://arxiv.org/abs/2004.10934).
304+
* [darknet](https://github.com/AlexeyAB/darknet)
305+
306+
My project is inspired by these previous fantastic YOLOv3 implementations:
307+
* [Yolov3 tensorflow](https://github.com/YunYang1994/tensorflow-yolov3)
308+
* [Yolov3 tf2](https://github.com/zzh8829/yolov3-tf2)

Diff for: ‎conda-cpu.yml

+15
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
name: yolov4-cpu
2+
3+
dependencies:
4+
- python==3.7
5+
- pip
6+
- matplotlib
7+
- opencv
8+
- pip:
9+
- opencv-python==4.1.1.26
10+
- lxml
11+
- tqdm
12+
- tensorflow==2.3.0rc0
13+
- absl-py
14+
- easydict
15+
- pillow

Diff for: ‎conda-gpu.yml

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
name: yolov4-gpu
2+
3+
dependencies:
4+
- python==3.7
5+
- pip
6+
- matplotlib
7+
- opencv
8+
- cudnn
9+
- cudatoolkit==10.1.243
10+
- pip:
11+
- tensorflow-gpu==2.3.0rc0
12+
- opencv-python==4.1.1.26
13+
- lxml
14+
- tqdm
15+
- absl-py
16+
- easydict
17+
- pillow

0 commit comments

Comments
 (0)