Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get build <1s with CacheDir #1529

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

BBBmau
Copy link

@BBBmau BBBmau commented Nov 26, 2024

Did a few implementations before getting to the Final result.

First I tried PreCompiled Headers but found that it just wasn't producing significant performance boosts if at all. I was getting at most 0.2 seconds of improvement with added on complexity to the codebase.

I then looked into some other methods and found that we can include ccache into our environment in SConstruct that brought it down to 1.4s

... rest of file ...
env = Environment(
  ENV=os.environ,
  CC='cache gcc',
  CXX='ccache g++',
... rest of file ...
)

This then led to looking into CacheDir which significantly improved the performance since it directly pulls the results from .scons_cache and checks if targets require rebuilding.

└─(20:57:47 on build-under-1s)──> scons -u -j$(nproc) --debug=time                                                                                       ──(Mon,Nov25)─┘
scons: Reading SConscript files ...
SConscript:/Users/mau/Dev/opendbc/opendbc/can/SConscript  took 2.591 ms
SConscript:/Users/mau/Dev/opendbc/opendbc/dbc/SConscript  took 3.831 ms
SConscript:/Users/mau/Dev/opendbc/SConscript  took 6.946 ms
SConscript:/Users/mau/Dev/opendbc/SConstruct  took 358.092 ms
scons: done reading SConscript files.
scons: Building targets ...
Retrieved `opendbc/can/common.os' from cache
Retrieved `opendbc/can/dbc.os' from cache
Command execution time: opendbc/can/common.os: 0.003291 seconds
Command execution time: opendbc/can/dbc.os: 0.002676 seconds
Retrieved `opendbc/can/parser.os' from cache
Command execution time: opendbc/can/parser.os: 0.001277 seconds
Retrieved `opendbc/can/packer.os' from cache
Command execution time: opendbc/can/packer.os: 0.001059 seconds
Retrieved `opendbc/can/packer_pyx.cpp' from cache
Command execution time: opendbc/can/packer_pyx.cpp: 0.002474 seconds
Retrieved `opendbc/can/libdbc.dylib' from cache
Command execution time: opendbc/can/libdbc.dylib: 0.001193 seconds
Retrieved `opendbc/can/packer_pyx.o' from cache
Command execution time: opendbc/can/packer_pyx.o: 0.002424 seconds
Retrieved `opendbc/can/parser_pyx.cpp' from cache
Command execution time: opendbc/can/parser_pyx.cpp: 0.001076 seconds
Retrieved `opendbc/can/packer_pyx.so' from cache
Command execution time: opendbc/can/packer_pyx.so: 0.000941 seconds
Retrieved `opendbc/can/parser_pyx.o' from cache
Command execution time: opendbc/can/parser_pyx.o: 0.003859 seconds
Retrieved `opendbc/dbc/tesla_radar_bosch_generated.dbc' from cache
Retrieved `opendbc/dbc/tesla_radar_continental_generated.dbc' from cache
Retrieved `opendbc/dbc/hyundai_kia_mando_corner_radar_generated.dbc' from cache
Retrieved `opendbc/dbc/hyundai_kia_mando_front_radar_generated.dbc' from cache
Retrieved `opendbc/can/parser_pyx.so' from cache
Retrieved `opendbc/dbc/toyota_tnga_k_pt_generated.dbc' from cache
Command execution time: opendbc/can/parser_pyx.so: 0.001665 seconds
Retrieved `opendbc/dbc/toyota_new_mc_pt_generated.dbc' from cache
Command execution time: opendbc/can: 0.000002 seconds
Retrieved `opendbc/dbc/toyota_nodsu_pt_generated.dbc' from cache
Retrieved `opendbc/dbc/toyota_rav4_prime_generated.dbc' from cache
Retrieved `opendbc/dbc/nissan_leaf_2018_generated.dbc' from cache
Retrieved `opendbc/dbc/nissan_x_trail_2017_generated.dbc' from cache
Retrieved `opendbc/dbc/subaru_forester_2017_generated.dbc' from cache
Retrieved `opendbc/dbc/subaru_outback_2019_generated.dbc' from cache
Retrieved `opendbc/dbc/subaru_global_2017_generated.dbc' from cache
Retrieved `opendbc/dbc/subaru_global_2020_hybrid_generated.dbc' from cache
Retrieved `opendbc/dbc/subaru_outback_2015_generated.dbc' from cache
Retrieved `opendbc/dbc/gm_global_a_powertrain_generated.dbc' from cache
Retrieved `opendbc/dbc/chrysler_ram_hd_generated.dbc' from cache
Retrieved `opendbc/dbc/chrysler_pacifica_2017_hybrid_generated.dbc' from cache
Retrieved `opendbc/dbc/chrysler_ram_dt_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_pilot_2023_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_crv_ex_2017_can_generated.dbc' from cache
Retrieved `opendbc/dbc/acura_rdx_2018_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_fit_hybrid_2018_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_crv_touring_2016_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_accord_2018_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_insight_ex_2019_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_fit_ex_2018_can_generated.dbc' from cache
Retrieved `opendbc/dbc/acura_ilx_2016_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_clarity_hybrid_2018_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_crv_ex_2017_body_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_odyssey_extreme_edition_2018_china_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_crv_executive_2016_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_civic_hatchback_ex_2017_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_civic_ex_2022_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_civic_touring_2016_can_generated.dbc' from cache
Retrieved `opendbc/dbc/acura_rdx_2020_can_generated.dbc' from cache
Retrieved `opendbc/dbc/honda_odyssey_exl_2018_generated.dbc' from cache
Retrieved `opendbc/dbc/tesla_radar_continental_generated.dbc' from cache
Retrieved `opendbc/dbc/tesla_radar_bosch_generated.dbc' from cache
Retrieved `opendbc/dbc/hyundai_kia_mando_front_radar_generated.dbc' from cache
Retrieved `opendbc/dbc/hyundai_kia_mando_corner_radar_generated.dbc' from cache
Command execution time: opendbc/dbc/acura_ilx_2016_can_generated.dbc: 0.028246 seconds
Command execution time: opendbc/dbc: 0.000002 seconds
Command execution time: opendbc: 0.000002 seconds
Command execution time: .: 0.000001 seconds
scons: done building targets.
Total SConsign sync time: 0.004131 seconds
Total build time: 0.534332 seconds
Total SConscript file execution time: 0.358252 seconds
Total SCons execution time: 0.054623 seconds
Total command execution time: 0.121457 seconds

This is ran on my M1 Max so I can imagine that when running on ThreadRipper we'll seen potentially builds at 0.3 seconds

Fixes #1528

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for contributing to opendbc! In order for us to review your PR as quickly as possible, check the following:

  • Convert your PR to a draft unless it's ready to review
  • Read the contributing docs
  • Before marking as "ready for review", ensure:
    • the goal is clearly stated in the description
    • all the tests are passing
    • include a route or your device' dongle ID if relevant

@adeebshihadeh
Copy link
Contributor

The issue is for an uncached build, so you need to do a scons -c first to properly evaluate this. I'll add that to the original description.

Also this PR only makes a difference in the case that you git clean -xdf then rebuild; opendbc is so small that it's not worth having a cache dir.

@adeebshihadeh adeebshihadeh marked this pull request as draft November 28, 2024 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Get build times under 1s
2 participants