Address binskim flag for compiler in driver build

[ShaojieZhuIntel]

Summary

Since the compiler in the driver build cannot pass the Binskim scan and shows [Explicitly disabled warnings: 4146;4244;4267], it is necessary to remove the suppression of these three warnings. LLVM code like:

npu-plugin-llvm/llvm/cmake/modules/HandleLLVMOptions.cmake

Line 692 in dfee500

-wd4146 # Suppress 'unary minus operator applied to unsigned type, result still unsigned'

     -wd4146 # Suppress 'unary minus operator applied to unsigned type, result still unsigned'
     -wd4244 # Suppress ''argument' : conversion from 'type1' to 'type2', possible loss of data'
     -wd4267 # Suppress ''var' : conversion from 'size_t' to 'type', possible loss of data'

-wd4146 means disable the /w4146 warning
-wd4244 means disable the /w4244 warning
-wd4267 means disable the /w4267 warning

So we need enable these three warnings to pass the Binskim

Since it is not possible to directly enable specific warnings for the LLVM repo within the compiler repo, we use an option here to control whether the specified warnings are suppressed.

JIRA ticket

• EISW-130780

Related PR in NPU Compiler and/or OpenVINO repository with sub-module update

• PR-205

Other related tickets

List tickets for additional work, eg, something was found during review but you agreed to address it in another Jira

• E-xxxxx

[ShaojieZhuIntel]

Apply the current changes from PR99 in PR15919, and perform compiler in driver build with PR205. The build was successful and passed the binskim scan.
The CiD build job is recorded in PR15919.

https://github.com/intel-innersource/applications.ai.vpu-accelerators.vpux-plugin/pull/15919 PR-checker passed

[AlexandruLorinti]

I confirm there are no errors regarding suppressed warnings in BinSkim scan

[ggladilo]

nitpick: I'm not quite familiar with LLVM CMake code involved here, but I'd consider updating global CMAKE variables, especially CMAKE_C/CXX_* not the best practice, reason is there's a chance it'd affect other targets you didn't intended to change
I saw that case in our project already, when OpenVINO did this and then propagated not always desired compilation options to our targets

If you stick with modifying these variables for some reason, could you please clarify why and confirm no unintended target is changed (presumably nothing outside of llvm-project folder)?

Alternative would be just working with target properties, you can specify compilation options per target such it's not intrusive

[ShaojieZhuIntel]

Try to modify and test based on your suggestion.

[ggladilo]

nitpick: if (MSVC AND BUILD_COMPILER_FOR_DRIVER)
Also, since we have to modified LLVM's codebase, I suppose we should have some kind tracker of all such changes

@nikita-kud do we have anything like this already?

[ShaojieZhuIntel]

Fixed, thank you!

[nikita-kud]

@nikita-kud do we have anything like this already?

Yes, we have: E-101222
But I think it can't be upstreamed. This change looks sad. Why we can't suppress these warnings scanner?

[ShaojieZhuIntel]

@nikita-kud do we have anything like this already?

Yes, we have: E-101222 But I think it can't be upstreamed. This change looks sad. Why we can't suppress these warnings scanner?

Because we need to pass binskim. @AlexandruLorinti We are currently fix the binskim scan failures by applying a patch during the CiD build. Since it cannot be upstreamed, can we continue using this method?

[AlexandruLorinti]

@nikita-kud do we have anything like this already?

Yes, we have: E-101222 But I think it can't be upstreamed. This change looks sad. Why we can't suppress these warnings scanner?

Because we need to pass binskim. @AlexandruLorinti We are currently fix the binskim scan failures by applying a patch during the CiD build. Since it cannot be upstreamed, can we continue using this method?

any method would be ok, you can keep the patch in CiD build

[ggladilo]

Consider adding a comment to each modification of compile options
Maybe makes sense to create a CMake function that accepts target and silences all needed warnings, then you could add explanation to just function implementation
In this case all the logic for making binskim happy is reused

[lmielick]

We should be able to do the same in NPU Cmake which includes LLVM, avoiding a patch.

[ggladilo]

Yea, I guess it's true, we could write CMake code/function in our project that processes pre-defined set of targets (from LLVM repo that we know are problematic to one our jobs/processes) and for each given target modifies a property of it that corresponds to compile options used to build the target accordingly
This way we get clean separation - this warnings disablement is not a problem of LLVM (since they can build their targets they probably just do not provide support for these-warnings-clean-build), but our problem since we seem to place more requirements on LLVM codebase than its authors/maintainers

[ggladilo]

Move comments to the function implementation, no need to repeat every time you call the function
Function name is confusing, do you disable or enable warnings here? From the values looks like you disable them
Also, I'd keep function specific to your use case, instead of accepting warnings as arguments, just "hardcode" them internally and name the function something along the lines as fix_binksim or something (choose better name than mine!)
Also consider @lmielick's comment to implement solution in compiler repo

[ShaojieZhuIntel]

Ok, thank you for your suggestions! Here we are enabling warnings, and I will modify the function according to your advice. Additionally, I am trying to modify only the compiler repo to make it pass binskim.

[ShaojieZhuIntel]

Firstly, can not directly enable warnings for LLVM within the compiler repo. Secondly, I tried creating a function in the compiler repo and calling this function for each target in LLVM that fails the binskim check. This approach reduces the binskim errors, but it involves modifying many targets. Additionally, if the same errors appear in other targets in the future, further modifications to the LLVM code will be necessary. Therefore, I believe it is more appropriate to add an option in LLVM that allows controlling the enabling of relevant warnings from the compiler repo.

[ShaojieZhuIntel]

Test job: https://jenkins-npu-compiler.ir.intel.com/job/pub/job/IE-Packages/job/Release/job/CiD-Windows10/15966/ passed
binskim passed
Hi @ggladilo @lmielick @nikita-kud @AlexandruLorinti Could you please help to review this change?

[andrey-golubev]

Why do we need it in LLVM? Can we instead do it at the compiler's side around add_subdirectory(LLVM)?
Then the change is local to our stack and doesn't have to be forever cherry-picked between LLVM releases.

Also, removing warnings doesn't look good to me. Are there issues in LLVM otherwise? Can we fix them instead and push that to upstream?

[lmielick]

This approach reduces the binskim errors, but it involves modifying many targets.

@ShaojieZhuIntel did you try using directory level CMAKE variables? I would presume we can append these flags into CFLAGS on directory level. Even if this is applied eagerly for all LLVM targets it should be fine:
a) there is no incentive to fix warnings on LLVM side
b) if it's contingent on BINSKIM flag we can still keep the LLVM warnings on in developer build or so

[ShaojieZhuIntel]

Why do we need it in LLVM? Can we instead do it at the compiler's side around add_subdirectory(LLVM)? Then the change is local to our stack and doesn't have to be forever cherry-picked between LLVM releases.

Also, removing warnings doesn't look good to me. Are there issues in LLVM otherwise? Can we fix them instead and push that to upstream?

We are not removing warnings; we are enabling warnings because some warnings are suppressed in LLVM, which is not allowed by binskim. Like:

  AliasAnalysis.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]
  AliasSetTracker.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]
  AssumeBundleQueries.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]
  AssumptionCache.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]
  BasicAliasAnalysis.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]
  BlockFrequencyInfo.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]
  BlockFrequencyInfoImpl.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]
  BranchProbabilityInfo.cpp.obj (LLVMAnalysis.lib) [Explicitly disabled warnings: 4146;4244;4267]

Even if we handle this around add_subdirectory(LLVM), it will be overridden by the actions in LLVM that disable warnings.

npu-plugin-llvm/llvm/cmake/modules/HandleLLVMOptions.cmake

Line 692 in dfee500

-wd4146 # Suppress 'unary minus operator applied to unsigned type, result still unsigned'

      -wd4146 # Suppress 'unary minus operator applied to unsigned type, result still unsigned'
      -wd4244 # Suppress ''argument' : conversion from 'type1' to 'type2', possible loss of data'
      -wd4267 # Suppress ''var' : conversion from 'size_t' to 'type', possible loss of data'

[ShaojieZhuIntel]

This approach reduces the binskim errors, but it involves modifying many targets.

@ShaojieZhuIntel did you try using directory level CMAKE variables? I would presume we can append these flags into CFLAGS on directory level. Even if this is applied eagerly for all LLVM targets it should be fine: a) there is no incentive to fix warnings on LLVM side b) if it's contingent on BINSKIM flag we can still keep the LLVM warnings on in developer build or so

Yes, I have tried enabling warnings around add_subdirectory(LLVM), but LLVM suppresses some warnings again, causing the enabling of warnings to be ineffective. Like:

npu-plugin-llvm/llvm/cmake/modules/HandleLLVMOptions.cmake

Line 692 in dfee500

-wd4146 # Suppress 'unary minus operator applied to unsigned type, result still unsigned'

      -wd4146 # Suppress 'unary minus operator applied to unsigned type, result still unsigned'
      -wd4244 # Suppress ''argument' : conversion from 'type1' to 'type2', possible loss of data'
      -wd4267 # Suppress ''var' : conversion from 'size_t' to 'type', possible loss of data'

[XinWangIntel]

@ggladilo @lmielick @andrey-golubev @AlexandruLorinti , could you help to review this again? The PR is to open suppressed warnings by remove some compiler flags (which make we can not see warnings in log). And we tried on vpux-plugin repo side, append cmake flag on that level can not work since llvm append flag after that again and invalid our new added flags.

[nikita-kud]

@ShaojieZhuIntel I'm sorry, but I would like to ask you to wait until the LLVM19 update is complete.

[andrey-golubev]

We are not removing warnings; we are enabling warnings because some warnings are suppressed in LLVM, which is not allowed by binskim.

Who should I speak to regarding this? I want to figure out whether we care enough about not-our thirdparty to have this mess forever.

Yes, I have tried enabling warnings around add_subdirectory(LLVM), but LLVM suppresses some warnings again, causing the enabling of warnings to be ineffective.

@ShaojieZhuIntel but what does this do then?

npu-plugin-llvm/llvm/cmake/modules/HandleLLVMOptions.cmake

Lines 749 to 754 in dfee500

	# Enable warnings
	if (LLVM_ENABLE_WARNINGS)
	# Put /W4 in front of all the -we flags. cl.exe doesn't care, but for
	# clang-cl having /W4 after the -we flags will re-enable the warnings
	# disabled by -we.
	set(msvc_warning_flags "/W4 ${msvc_warning_flags}")

Do we need our custom flag at all here? Can we instead use LLVM_ENABLE_WARNINGS=ON?

[andrey-golubev]

Blocking for now: let's figure our what already present LLVM_ENABLE_WARNINGS does

[ShaojieZhuIntel]

We are not removing warnings; we are enabling warnings because some warnings are suppressed in LLVM, which is not allowed by binskim.

Who should I speak to regarding this? I want to figure out whether we care enough about not-our thirdparty to have this mess forever.

Yes, I have tried enabling warnings around add_subdirectory(LLVM), but LLVM suppresses some warnings again, causing the enabling of warnings to be ineffective.

@ShaojieZhuIntel but what does this do then?

npu-plugin-llvm/llvm/cmake/modules/HandleLLVMOptions.cmake

Lines 749 to 754 in dfee500

	# Enable warnings
	if (LLVM_ENABLE_WARNINGS)
	# Put /W4 in front of all the -we flags. cl.exe doesn't care, but for
	# clang-cl having /W4 after the -we flags will re-enable the warnings
	# disabled by -we.
	set(msvc_warning_flags "/W4 ${msvc_warning_flags}")

Do we need our custom flag at all here? Can we instead use LLVM_ENABLE_WARNINGS=ON?

First, we need to address the binskim issue caused by this thirdparty because the target npu_llvm_utils in VPUX is related to this thirdparty. Binskim shows

Secondly, enabling LLVM_ENABLE_WARNINGS can display more warnings when building LLVM, but it won't provide more information when building VPUX, and it won't enable warnings 4146, 4244, 4267, and other suppressed warnings in ${msvc_warning_flags} . However, we can remove all -wd flags from ${msvc_warning_flags} when LLVM_ENABLE_WARNINGS is ON. This approach not only enables the warnings but also resolves the binskim scan errors. The new changes have been pushed and tested to work.

[andrey-golubev]

can we add these in NPU CMake to CMAKE_CXX_FLAGS instead?
something tells me it should be possible to do. and we won't need the patch here.

[andrey-golubev]

(now that we know that ENABLE_WARNINGS works, perhaps we could move it outside?)

[ShaojieZhuIntel]

-w14062 and -we4238 are not the key to solving this issue. The key is to clear the -wdxxxx flags in ${msvc_warning_flags}.
Only enabling LLVM_ENABLE_WARNINGS does not work. We must reset ${msvc_warning_flags} in this code because

npu-plugin-llvm/llvm/cmake/modules/HandleLLVMOptions.cmake

Lines 777 to 780 in 13000db

	foreach(flag ${msvc_warning_flags})
	append("${flag}" CMAKE_C_FLAGS CMAKE_CXX_FLAGS)
	endforeach(flag)
	endif (MSVC)

writes ${msvc_warning_flags} into CMAKE_CXX_FLAGS.
Our goal is to remove all -wdxxxx flags set in

npu-plugin-llvm/llvm/cmake/modules/HandleLLVMOptions.cmake

Lines 689 to 746 in 13000db

	set(msvc_warning_flags
	# Disabled warnings.
	-wd4141 # Suppress ''modifier' : used more than once' (because of __forceinline combined with inline)
	-wd4146 # Suppress 'unary minus operator applied to unsigned type, result still unsigned'
	-wd4244 # Suppress ''argument' : conversion from 'type1' to 'type2', possible loss of data'
	-wd4267 # Suppress ''var' : conversion from 'size_t' to 'type', possible loss of data'
	-wd4291 # Suppress ''declaration' : no matching operator delete found; memory will not be freed if initialization throws an exception'
	-wd4351 # Suppress 'new behavior: elements of array 'array' will be default initialized'
	-wd4456 # Suppress 'declaration of 'var' hides local variable'
	-wd4457 # Suppress 'declaration of 'var' hides function parameter'
	-wd4458 # Suppress 'declaration of 'var' hides class member'
	-wd4459 # Suppress 'declaration of 'var' hides global declaration'
	-wd4503 # Suppress ''identifier' : decorated name length exceeded, name was truncated'
	-wd4624 # Suppress ''derived class' : destructor could not be generated because a base class destructor is inaccessible'
	-wd4722 # Suppress 'function' : destructor never returns, potential memory leak
	-wd4100 # Suppress 'unreferenced formal parameter'
	-wd4127 # Suppress 'conditional expression is constant'
	-wd4512 # Suppress 'assignment operator could not be generated'
	-wd4505 # Suppress 'unreferenced local function has been removed'
	-wd4610 # Suppress '<class> can never be instantiated'
	-wd4510 # Suppress 'default constructor could not be generated'
	-wd4702 # Suppress 'unreachable code'
	-wd4245 # Suppress ''conversion' : conversion from 'type1' to 'type2', signed/unsigned mismatch'
	-wd4706 # Suppress 'assignment within conditional expression'
	-wd4310 # Suppress 'cast truncates constant value'
	-wd4701 # Suppress 'potentially uninitialized local variable'
	-wd4703 # Suppress 'potentially uninitialized local pointer variable'
	-wd4389 # Suppress 'signed/unsigned mismatch'
	-wd4611 # Suppress 'interaction between '_setjmp' and C++ object destruction is non-portable'
	-wd4805 # Suppress 'unsafe mix of type <type> and type <type> in operation'
	-wd4204 # Suppress 'nonstandard extension used : non-constant aggregate initializer'
	-wd4577 # Suppress 'noexcept used with no exception handling mode specified; termination on exception is not guaranteed'
	-wd4091 # Suppress 'typedef: ignored on left of '' when no variable is declared'
	# C4592 is disabled because of false positives in Visual Studio 2015
	# Update 1. Re-evaluate the usefulness of this diagnostic with Update 2.
	-wd4592 # Suppress ''var': symbol will be dynamically initialized (implementation limitation)
	-wd4319 # Suppress ''operator' : zero extending 'type' to 'type' of greater size'
	# C4709 is disabled because of a bug with Visual Studio 2017 as of
	# v15.8.8. Re-evaluate the usefulness of this diagnostic when the bug
	# is fixed.
	-wd4709 # Suppress comma operator within array index expression
	# We'd like this warning to be enabled, but it triggers from code in
	# WinBase.h that we don't have control over.
	-wd5105 # Suppress macro expansion producing 'defined' has undefined behavior
	# Ideally, we'd like this warning to be enabled, but even MSVC 2019 doesn't
	# support the 'aligned' attribute in the way that clang sources requires (for
	# any code that uses the LLVM_ALIGNAS macro), so this is must be disabled to
	# avoid unwanted alignment warnings.
	-wd4324 # Suppress 'structure was padded due to __declspec(align())'
	# Promoted warnings.
	-w14062 # Promote 'enumerator in switch of enum is not handled' to level 1 warning.
	# Promoted warnings to errors.
	-we4238 # Promote 'nonstandard extension used : class rvalue used as lvalue' to error.
	)

, such as -wd4141, -wd4146, and -wd4244. These flags disable some warnings. By removing them, we can enable the corresponding warnings. Removing these -wdxxxx flags while enabling LLVM_ENABLE_WARNINGS aligns with the purpose of this option.

[ggladilo]

Changing CMAKE_CXX_FLAGS is not a good idea since it's global property and will impact all targets built, which is not what we want probably

[ShaojieZhuIntel]

Changing CMAKE_CXX_FLAGS is not a idea since it's global property and will impact all targets built, which is not what we want probably

In fact, I have already tried enabling the specified warnings for each target that reported an error, but this approach has two drawbacks. First, there are too many targets that need modification. Second, if there are other BinSkim errors in the future, further modifications will be required. Additionally, I have tried enabling warnings around add_subdirectory(LLVM), but LLVM suppresses some warnings again, causing the enabling of warnings to be ineffective.
Changing CMAKE_CXX_FLAGS is only to re-enable the previously suppressed warnings, which aligns with the purpose of the LLVM_ENABLE_WARNINGS option. This option is off by default, and we only turn it on during CiD build, so other builds won't be affected.

[ggladilo]

To clarify, my original concern was that with this change we disable some warnings for potentially a lot of targets, but now I see we actually enable warnings, so I'm fine with the change

[ShaojieZhuIntel]

Hi @nikita-kud , @andrey-golubev , @AlexandruLorinti , @ggladilo LLVM19 update was completed, and I have rebase this PR to npu/release/19.x. Testing shows that the BinSkim issue still reproduce with LLVM 19, and the current solution can resolve this issue. Could you please help to review this PR?

[ShaojieZhuIntel]

Sorry for the late reply. There was an issue with the response status, it has just been corrected.