Skip to content

Commit 1f88ff4

Browse files
kimishpatelfacebook-github-bot
authored andcommitted
Update doc and code to run quantized model (#157)
Summary: Pull Request resolved: #157 - Fix doc to separte 1) generating quantized model and b) running it with executor_runner - Include <tuple> in chose_qparams - Include quantized ops by default in executor_runner Reviewed By: larryliu0820, guangy10 Differential Revision: D48752106 fbshipit-source-id: 30f4e7ba121abeb01b7b97020c2fef0f5d2ac891
1 parent b9f37cc commit 1f88ff4

File tree

4 files changed

+18
-6
lines changed

4 files changed

+18
-6
lines changed

examples/README.md

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,9 @@ buck2 run examples/executor_runner:executor_runner -- --model_path mv2.pte
5858
## Quantization
5959
Here is the [Quantization Flow Docs](/docs/website/docs/tutorials/quantization_flow.md).
6060

61-
You can run quantization test with the following command:
61+
### Generating quantized model
62+
63+
You can generate quantized model with the following command (following example is for mv2, aka MobileNetV2):
6264
```bash
6365
python3 -m examples.quantization.example --model_name "mv2" --so-library "<path/to/so/lib>" # for MobileNetv2
6466
```
@@ -80,6 +82,16 @@ you can also find the valid quantized example models by running:
8082
buck2 run executorch/examples/quantization:example -- --help
8183
```
8284

85+
### Running quantized model
86+
87+
Quantized model can be run via executor_runner, similar to floating point model, via, as shown above:
88+
89+
```bash
90+
buck2 run examples/executor_runner:executor_runner -- --model_path mv2.pte
91+
```
92+
93+
Note that, running quantized model, requires various quantized/dequantize operators, available in [quantized kernel lib](/kernels/quantized).
94+
8395
## XNNPACK Backend
8496
Please see [Backend README](backend/README) for XNNPACK quantization, export, and run workflow.
8597

examples/executor_runner/targets.bzl

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,13 @@ def define_common_targets():
2828

2929
register_custom_op = native.read_config("executorch", "register_custom_op", "0")
3030
register_quantized_ops = native.read_config("executorch", "register_quantized_ops", "0")
31-
custom_ops_lib = []
31+
32+
# Include quantized ops to be able to run quantized model with portable ops
33+
custom_ops_lib = ["//executorch/kernels/quantized:generated_lib"]
3234
if register_custom_op == "1":
3335
custom_ops_lib.append("//executorch/examples/custom_ops:lib_1")
3436
elif register_custom_op == "2":
3537
custom_ops_lib.append("//executorch/examples/custom_ops:lib_2")
36-
if register_quantized_ops == "1":
37-
custom_ops_lib.append("//executorch/kernels/quantized:generated_lib")
3838

3939
# Test driver for models, uses all portable kernels and a demo backend. This
4040
# is intended to have minimal dependencies. If you want a runner that links

examples/quantization/test_quantize.sh

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,8 +32,7 @@ test_buck2_quantization() {
3232
${PYTHON_EXECUTABLE} -m "examples.quantization.example" --so_library="$SO_LIB" --model_name="$1"
3333

3434
echo 'Running executor_runner'
35-
buck2 run //examples/executor_runner:executor_runner \
36-
--config=executorch.register_quantized_ops=1 -- --model_path="./$1.pte"
35+
buck2 run //examples/executor_runner:executor_runner -- --model_path="./$1.pte"
3736
# should give correct result
3837

3938
echo "Removing $1.pte"

kernels/quantized/cpu/op_choose_qparams.cpp

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
#include <algorithm>
1212
#include <cinttypes>
1313
#include <cmath>
14+
#include <tuple>
1415
/**
1516
* For an input tensor, use the scale and zero_point arguments to quantize it.
1617
*/

0 commit comments

Comments
 (0)