Repository navigation
Add in-process entry points to gen_all_oplist and codegen.gen - #24704
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24704
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 7758683 with merge base 4411b8d ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
acdb002 to
6a1e995
Compare
6a1e995 to
54a5479
Compare
54a5479 to
164f8f6
Compare
JakeStevens
left a comment
There was a problem hiding this comment.
fyi claude is nervous about internal ci with this, keep an eye i guess
164f8f6 to
b0e7c0b
Compare
gen_oplist, gen_selected_prim_ops and gen_max_kernel_num each have a function their main() calls, but gen_all_oplist and codegen.gen do all their work in main(), so a Python caller has to start a subprocess or fake a command line to use them. Their work moves into gen_all_oplist.gen_all_oplist() and codegen.gen.generate(), and main() keeps only the command-line handling: argument parsing, the @-prefixed list file, the BUCK checks and gen_all_oplist's single-yaml mode, which still leaves debug_info alone. generate() takes main()'s flags as keyword arguments (outputs is --generate; the unused --static-dispatch-backend is left out) and builds its FileManager directly instead of through make_file_manager(options). Its tags.yaml and native_functions.yaml default to the copies packaged with torchgen, the ones tools/cmake/Codegen.cmake passes, so callers need not know where torchgen keeps them; --tags-path and --aten-yaml-path default the same way. The command lines behave as before: a selective build of two models plus an op list makes byte-identical sources with the old and new tools. The ESP-IDF component's gen_kernels.py calls both functions in the next change. Authored with Claude Code.
b0e7c0b to
7758683
Compare
gen_oplist, gen_selected_prim_ops and gen_max_kernel_num each have a
function their main() calls, but gen_all_oplist and codegen.gen do all
their work in main(), so a Python caller has to start a subprocess or
fake a command line to use them. Their work moves into
gen_all_oplist.gen_all_oplist() and codegen.gen.generate(), and main()
keeps only the command-line handling: argument parsing, the @-prefixed
list file, the BUCK checks and gen_all_oplist's single-yaml mode, which
still leaves debug_info alone. generate() takes main()'s flags as keyword
arguments (outputs is --generate; the unused --static-dispatch-backend is
left out) and builds its FileManager directly instead of through
make_file_manager(options). Its tags.yaml and native_functions.yaml
default to the copies packaged with torchgen, the ones
tools/cmake/Codegen.cmake passes, so callers need not know where torchgen
keeps them; --tags-path and --aten-yaml-path default the same way.
The command lines behave as before: a selective build of two models plus
an op list makes byte-identical sources with the old and new tools. The
ESP-IDF component's gen_kernels.py calls both functions in the next
change.
Authored with Claude Code.