Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable unusedFunction checks #1970

Draft
wants to merge 14 commits into
base: master
Choose a base branch
from
Draft

Enable unusedFunction checks #1970

wants to merge 14 commits into from

Conversation

dzid26
Copy link
Contributor

@dzid26 dzid26 commented Jun 4, 2024 •
edited
Loading

Find unusedFunctions for Panda program. Allows to:

  • treat F4 and H7 variants as a part of same program
  • differentiate between functions used in Panda and e.g. PandaJungle

Links: #1954 #1794 #1878

@dzid26 dzid26 force-pushed the unusedFunction branch 3 times, most recently from 31ff4a0 to b9e64f8 Compare June 5, 2024 16:53
@dzid26
Copy link
Contributor Author

dzid26 commented Jun 5, 2024 •
edited
Loading

  • I enabled --project compile_commands.json for Misra and unusedFunctions now.
    • it uses scons, so it can check macros we are interested in.
    • for CI I think we may run twice. With and without RELEASE env
    • it checks all main() files, so it found 104 violations (to be fixed)
  • I still kept per build checks for unusedFunctions but only for Panda and bootstubs builds - I think this best practise and I think this similar will be need to be achieved for misra 8.x rules.

@dzid26 dzid26 force-pushed the unusedFunction branch 6 times, most recently from ee1eeb3 to f8d5b49 Compare June 7, 2024 14:03
@dzid26
Copy link
Contributor Author

dzid26 commented Jun 7, 2024 •
edited
Loading

@adeebshihadeh is SPI ever used for Panda F4 besides the CI test?
I removed the stm32f4/llspi.h header since normal scons job doesn't use it and mutation test was failing sometimes.

I can:

  • remove building with ENABLE_SPI from CI test
  • or add ENABLE_SPI in misra tests

@adeebshihadeh
Copy link
Contributor

@adeebshihadeh is SPI ever used for Panda F4 besides the CI test? I removed the stm32f4/llspi.h header since normal scons job doesn't use it and so mutation test was failing sometimes.

I can:

  • remove building with ENABLE_SPI from CI test
  • or add ENABLE_SPI in misra tests

Let's add it to the tests. We don't use it yet, but might want to in the future.

@dzid26 dzid26 force-pushed the unusedFunction branch 6 times, most recently from df66b33 to 60c3a42 Compare June 9, 2024 08:41
@dzid26
Copy link
Contributor Author

dzid26 commented Jun 9, 2024 •
edited
Loading

I noticed that cppcheck doesn't report system level misra violation when used with --project= compile_commands.json argument.
I will see if I can fix it, because otherwise, it is the most robust way to call whole program analysis.

@dzid26 dzid26 force-pushed the unusedFunction branch 2 times, most recently from ebf561a to dac2b4d Compare June 10, 2024 21:15
dzid26 added 8 commits June 11, 2024 01:02
@dzid26
Add gnu.cfg library. Make Cppcheck work without system headers (faster).
7786a0d 
- gnu.cfg replaces __typeof__ with typeof() and triggered some misra-c2012-10.4 - fixed by making both function args the same essential type
@dzid26
simplify stm32 header include
15e70df
@dzid26
cppcheck panda whole program analysis
084fb51 
- combines panda build configurations

Increase mutation test timeout
@dzid26
enable unusedFunction check for Panda program
f604fc6 
remove Panda specific functions
@dzid26
removed globally unused functions
a4f9e2a 
STMF4 doesn't use uart interrupt
DAC and watchdog are not used
@dzid26
moved hundai canfd checksum into canfd header
9390319
@dzid26
memset() - suppress "unusedFunction"
2cb7be1 
memset is needed by the linker
@dzid26
ENABLE_SPI is only used in Dos config and nowhere in H7
29acdd6
@dzid26
Copy link
Contributor Author

dzid26 commented Jun 11, 2024 •
edited
Loading

--project=compile_commands.json is definitely the way to go in the future, but I needed to give up on it until cppcheck bugs (resulting in false negatives on system level misra checks) get fixed.

In the meantime, I figured out a way to check different macro configurations together using --force and force-included dummy/helper header which specifies possible combinations for F4 & H7. This should make checking for unused macros (rule 2.5) fairly straightforward as well.

@dzid26 dzid26 marked this pull request as ready for review June 12, 2024 02:00
adeebshihadeh
adeebshihadeh requested changes Jun 12, 2024
View reviewed changes
tests/misra/test_mutation.py Outdated Show resolved Hide resolved
tests/misra/checkers.txt
@@ -193,440 +193,7 @@ Yes CheckType::checkSignConversion
Yes CheckType::checkTooBigBitwiseShift
Yes CheckUninitVar::check
Yes CheckUninitVar::valueFlowUninit
No CheckUnusedFunctions::check require:unusedFunction
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did all of this get removed?

Copy link
Contributor Author

@dzid26 dzid26 Jun 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The diff is a bit weird - the second variant used to start at line 440 and now is removed.
Basically now, instead of checking F4 and H7 separately, they are checked together using cppcheck configuration combination capability (which is enforced with... --force argument ).

tests/misra/test_mutation.py Outdated Show resolved Hide resolved
@dzid26 dzid26 force-pushed the unusedFunction branch 2 times, most recently from 99f0254 to e7fc9db Compare June 13, 2024 02:13
@dzid26 dzid26 requested a review from adeebshihadeh June 19, 2024 23:54
@dzid26 dzid26 mentioned this pull request Jul 8, 2024
Merged
@dzid26
Copy link
Contributor Author

dzid26 commented Jul 26, 2024 •
edited
Loading

Converting to draft.
It seems that Cppcheck bails out if it encounters [unknownMacro] violation. But since I had --suppress=unknownMacro it wouldn't report any remaining violations. https://sourceforge.net/p/cppcheck/discussion/general/thread/769381a303/#f0dc

We need to add --safety flag to avoid quiet bail-outs like this. danmar/cppcheck#5777 (comment)

Underlying issue is related to how --force option works:
https://sourceforge.net/p/cppcheck/discussion/general/thread/5c740643e6/?limit=25#7278

@dzid26 dzid26 marked this pull request as draft July 26, 2024 13:56
@dzid26
Squashed commit of the following:
624e2b1 
commit 8545c68
Author: Radek <[email protected]>
Date:   Fri Sep 13 15:12:35 2024 -0700

    Fix USB errors (commaai#2011)

    Fix LIBUSB_ERROR_PIPE [-9] when resetting over USB hubs or jungle V2

commit b8a2a86
Author: Maxime Desroches <[email protected]>
Date:   Tue Sep 10 21:49:03 2024 -0700

    fix python setup (commaai#2018)

    * fix

    * comment

commit fcccbb3
Author: dzid26 <[email protected]>
Date:   Sat Sep 7 19:57:19 2024 -0700

    revert shorter MISRA mutation CI timeout
    - test_misra.sh only got faster locally

commit 2aaab40
Author: Radek <[email protected]>
Date:   Fri Sep 6 13:30:05 2024 -0700

    fix test_mutation.py performace (commaai#2012)

    stop pytest from executing test_misra.sh in test collections
    - in each test_mutation.py worker

commit 6f0d1ef
Author: dzid26 <[email protected]>
Date:   Thu Sep 5 21:49:30 2024 -0700

    stricter misra config check - prevent quiet bail out

commit 93776fd
Author: Greg Hogan <[email protected]>
Date:   Tue Sep 3 20:42:12 2024 -0700

    CCP: fix upload response size and getting seed (commaai#2009)

commit aac60b8
Author: commaci-public <[email protected]>
Date:   Sun Sep 1 09:51:30 2024 -0700

    pre-commit: autoupdate hooks (commaai#2010)

    Update pre-commit hook versions

    Co-authored-by: adeebshihadeh <[email protected]>

commit 9ec02e7
Author: Adeeb Shihadeh <[email protected]>
Date:   Sat Aug 31 13:17:55 2024 -0700

    socketcan: skip install in CI for now

commit 160ea08
Author: Adeeb Shihadeh <[email protected]>
Date:   Sat Aug 31 13:09:58 2024 -0700

    misra: bump to cppcheck 2.15.0 (commaai#2008)

    * misra: bump to latest cppcheck

    * update checkers

commit a36ca22
Author: MarinkoMagla <[email protected]>
Date:   Sat Aug 31 22:01:28 2024 +0200

    Defining types in return dictionaries (commaai#1923)

    * Defining types in return dictionaries

    * Correcting indent

    * Returning the class instead of dict

    * Fixing failing linter

    * Fixing whitespace error

commit bd6cec3
Author: Robbe Derks <[email protected]>
Date:   Fri Aug 30 15:35:50 2024 +0200

    cuatro LED (commaai#2005)

    * Revert "cuatro: disable LED for now"

    This reverts commit 866bd9c.

    * re-enable and make sure to use open-drain

    * use gpio function

    ---------

    Co-authored-by: Comma Device <[email protected]>

commit 8587ae3
Author: Shane Smiskol <[email protected]>
Date:   Mon Aug 26 17:56:46 2024 -0700

    drivers: log can core reset when switching multiplexing (commaai#1972)

    fdcan: log lost tx messages when resetting can core for multiplexing

commit 0a09122
Author: Andrei Radulescu <[email protected]>
Date:   Tue Aug 27 03:25:15 2024 +0300

    update bash scripts to #!/usr/bin/env bash (commaai#1987)

    * update bash shebang for better compatibility including macOS

    * can't have anything after #bash

commit 2b94e4f
Author: dzid26 <[email protected]>
Date:   Tue Aug 27 00:15:29 2024 +0000

    Enable CAN3 printer (commaai#1991)

    * panda has 3 can peripherals

    * enable printing CAN3

commit 866bd9c
Author: Adeeb Shihadeh <[email protected]>
Date:   Fri Aug 23 20:41:53 2024 -0700

    cuatro: disable LED for now

commit 30d0434
Author: Comma Device <[email protected]>
Date:   Fri Aug 23 04:57:34 2024 +0000

    cuatro: disable SOM reset

commit e53c802
Author: Adeeb Shihadeh <[email protected]>
Date:   Thu Aug 22 21:55:02 2024 -0700

    tres: fix SOM reset line blinking (commaai#2003)

    Co-authored-by: Comma Device <[email protected]>

commit 080e53f
Author: Robbe Derks <[email protected]>
Date:   Sun Aug 18 17:43:27 2024 -0700

    Cuatro fan (commaai#1999)

    * cuatro fan debugging

    * fix

    * faster fan, faster interrupts

    * fix comment

    ---------

    Co-authored-by: Comma Device <[email protected]>

commit cfa8b79
Author: Adeeb Shihadeh <[email protected]>
Date:   Sun Aug 18 12:01:17 2024 -0700

    cleanup dependencies (commaai#2001)

    * cleanup dependencies

    * in the real spot

    * fix jenkins

commit c4e75ee
Author: Adeeb Shihadeh <[email protected]>
Date:   Sun Aug 18 11:40:54 2024 -0700

    remove cereal dependence (commaai#2000)

    * remove cereal

    * rm more shit

    * add back make

    * python3

    * python3

    * fix that

    * put that back

    * rename

    * fix that

    * can't move it

commit e7a04b5
Author: Shane Smiskol <[email protected]>
Date:   Fri Aug 16 23:01:54 2024 -0700

    make uds and isotp general imports (commaai#1998)

commit 0234e75
Author: Robbe Derks <[email protected]>
Date:   Thu Aug 15 21:33:07 2024 -0700

    STM32H7: add missing interrupt handlers (commaai#1997)

    add missing interrupts

    Co-authored-by: Comma Device <[email protected]>

commit 87572e0
Author: geeth345 <[email protected]>
Date:   Tue Aug 13 18:34:46 2024 +0100

    Update README.md to fix dead link (commaai#1996)

    Changed C++ library link to reflect boardd -> pandad change

commit 1cbcc13
Author: Shane Smiskol <[email protected]>
Date:   Mon Aug 5 14:51:56 2024 -0700

    use and configure custom logger (commaai#1994)

    * prepare to log logs

    * utils

    * discoverability

    * forgot

commit 3c1def1
Author: commaci-public <[email protected]>
Date:   Fri Aug 2 15:51:37 2024 -0700

    pre-commit: autoupdate hooks (commaai#1979)

    Update pre-commit hook versions

    Co-authored-by: adeebshihadeh <[email protected]>

commit 5d81523
Author: Adeeb Shihadeh <[email protected]>
Date:   Fri Aug 2 13:52:28 2024 -0700

    1min is all you need

commit b70ba5e
Author: Robbe Derks <[email protected]>
Date:   Fri Aug 2 20:26:16 2024 +0200

    Detect H723 in firmware (commaai#1988)

    * determine H723 based on package type

    * wrong way round

    * misra...

    * enable SYSCFG clock before reading

    * whitelist certain package types

commit ee7b385
Author: Adeeb Shihadeh <[email protected]>
Date:   Fri Aug 2 11:02:38 2024 -0700

    CI: add jenkins-trigger

commit daa739e
Author: Shane Smiskol <[email protected]>
Date:   Wed Jul 31 14:30:53 2024 -0700

    missed a busTime (commaai#1990)

    * whoops

    * addr dat src

commit 8c3bb01
Author: Shane Smiskol <[email protected]>
Date:   Tue Jul 30 21:20:48 2024 -0700

    deprecate busTime (commaai#1989)

    * deprecate busTime

    * fix test_comms.py

    * update opendbc

    * way more than i thought

    * and here

    * oops

commit f637584
Author: Hoang Bui <[email protected]>
Date:   Wed Jul 17 23:49:10 2024 -0400

    add `set_gas_pressed_prev()` helper (commaai#1986)

    add set_gas_pressed_prev helper

commit 5ee262f
Author: Adeeb Shihadeh <[email protected]>
Date:   Wed Jul 17 11:19:24 2024 -0700

    H7 CR3 notes (commaai#1985)

commit b4e3d5c
Author: Adeeb Shihadeh <[email protected]>
Date:   Sun Jul 14 15:56:18 2024 -0700

    Revert "H7: detect board type with second ID pin group (commaai#1975)"

    This reverts commit 376408b.

commit 481d2be
Author: pantew869 <[email protected]>
Date:   Sat Jul 13 23:34:25 2024 +0300

    Use numpy instead of numpy-stubs in pre-commit (commaai#1984)

    * delete numpy-stubs from additional_dependencies(it's deprecated and causes git issues)

    * use numpy directly

commit 376408b
Author: Adeeb Shihadeh <[email protected]>
Date:   Wed Jun 26 18:20:51 2024 -0700

    H7: detect board type with second ID pin group (commaai#1975)

    * H7: detect board type based on package

    * detect smps

    * misra fix

    * jungle?

    * pkg isn't reliable, need syscfg enabled

    ---------

    Co-authored-by: Comma Device <[email protected]>

commit 29f2f30
Author: Adeeb Shihadeh <[email protected]>
Date:   Wed Jun 26 17:08:35 2024 -0700

    jenkins: run jungles first (commaai#1977)

    * jenkins: run jungles first

    * inside stage

commit ee1d5ce
Author: Adeeb Shihadeh <[email protected]>
Date:   Wed Jun 26 16:59:08 2024 -0700

    add assert_fatal function (commaai#1976)

    * add assert function

    * fix

    * flip

    * rename

    * assert fatal

commit cac94e3
Author: JJ <[email protected]>
Date:   Tue Jun 25 06:53:31 2024 -0700

    Update README.md (commaai#1974)

    small spelling fix

commit 7287ff0
Author: Shane Smiskol <[email protected]>
Date:   Wed Jun 12 19:21:38 2024 -0700

    query_fw_versions: ignore 29 bit addresses option (commaai#1973)

    fix
@dzid26
Merge commit '93aedd987bda7e267210689b877eb87f288fc6ab' into unusedFu…
fd6fe90 
…nction
@dzid26 dzid26 force-pushed the unusedFunction branch 3 times, most recently from 2721b14 to 3493596 Compare September 23, 2024 03:05
@dzid26
cppcheck with --project
0c67ceb 
CI: test misra with RELEASE=1
Generates compile db without ALLOW_DEBUG
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@adeebshihadeh adeebshihadeh Awaiting requested review from adeebshihadeh

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@dzid26 @adeebshihadeh