[LLVM] Emit dwarf data for changed-signature and new functions

Add a new pass EmitChangedFuncDebugInfo which will add dwarf for
additional functions including functions with signature change
and new functions.

The previous approach in [1] tries to add debuginfo for those
optimization passes which cause signature changes. Based on
discussion in [1], it is preferred to have a specific pass to
add debuginfo and later on dwarf generation can include those
new debuginfo.

The ultimate goal is to add new information to dwarf like below:

  DW_TAG_compile_unit
    ...
    // New functions with suffix
    DW_TAG_inlined_subroutine
      DW_AT_name      ("foo.1")
      DW_AT_type      (0x0000000000000091 "int")
      DW_AT_artificial (true)
      DW_AT_specificiation (original DW_TAG_subprogram)

      DW_TAG_formal_parameter
        DW_AT_name    ("b")
        DW_AT_type    (0x0000000000000091 "int")

      DW_TAG_formal_parameter
        DW_AT_name    ("c")
        DW_AT_type    (0x0000000000000095 "long")

    ...
    // Functions with changed signatures
    DW_TAG_inlined_subroutine
      DW_AT_name      ("bar")
      DW_AT_type      (0x0000000000000091 "int")
      DW_AT_artificial (true)
      DW_AT_specificiation (original DW_TAG_subprogram)

      DW_TAG_formal_parameter
        DW_AT_name    ("c")
        DW_AT_type    (0x0000000000000095 "unsigned int")

    ...
    // Functions not obtained function changed signatures yet
    // The DW_CC_nocall presence indicates such cases.
    DW_TAG_inlined_subroutine
      DW_AT_name      ("bar" or "bar.1")
      DW_AT_calling_convention        (DW_CC_nocall)
      DW_AT_artificial (true)
      DW_AT_specificiation (original DW_TAG_subprogram)

The parent tag of above DW_TAG_inlined_subroutine is
DW_TAG_compile_unit. This is a new feature for dwarf
so it won't cause issues with existing dwarf related tools.
Total three patterns are introduced as the above.
  . New functions with suffix, e.g., 'foo.1' or 'foo.llvm.<hash>'.
  . Functions with changed signature due to ArgumentPromotion
    or DeadArgumentElimination.
  . Functions the current implementation cannot get proper
    signature. For this case, DW_CC_nocall is set to indicate
    signature is lost. More details in the below.

A special CompileUnit with file name "<artificial>" is created
to hold special DISubprograms for the above three kinds of functions.
During actual dwarf generation, these special DISubprograms
will turn to above to proper DW_TAG_inlined_subroutine tags.

The below are some discussions with not handled cases and
some other alternative things:
 (1) Currently, there are three not handled signature changes.
     . During to ArgumentPromotion, we may have
       foo(..., struct foo *p, ...) => foo(..., int p.0.val, int p.4.val, ...)
     . Struct argument which expands to two actual arguments,
       foo(..., struct foo v, ...) => foo(..., v.coerce0, v.coerce1, ...)
     . Struct argument changed to struct pointer,
       foo(..., struct foo v, ...) => foo(..., struct foo *p, ...)
     I think by utilizing dbg_value/dbg_declare and instructions, we
     might be able to resolve the above and get proper signature.
     But any suggestions are welcome.
 (2) Currently, I am using a special CompileUnit "<artificial>" to hold
     newly created DISubprograms. But there is an alternative.
     For example, "llvm.dbg.cu" metadata is used to hold all CompileUnits.
     We could introduce "llvm.dbg.sp.extra" to hold all new
     DISubprograms instead of a new CompileUnit.

I have tested this patch set by building latest bpf-next linux kernel.
For no-lto case:
  65288 original number of functions
  910   new functions with this patch (including DW_CC_nocall case)
  7     new functions without signatures (with DW_CC_nocall)
For thin-lto case:
  65541 original number of functions
  2324  new functions with this patch (including DW_CC_nocall case)
  14    new functions without signatures (with DW_CC_nocall)

The following are some examples with thinlto with generated dwarf:

  ...
  0x0001707f:   DW_TAG_inlined_subroutine
                  DW_AT_name      ("msr_build_context")
                  DW_AT_type      (0x00004163 "int")
                  DW_AT_artificial        (true)
                  DW_AT_specification     (0x0000440b "msr_build_context")

  0x0001708b:     DW_TAG_formal_parameter
                    DW_AT_name    ("msr_id")
                    DW_AT_type    (0x0000e55c "const u32 *")

  0x00017093:     NULL
  ...
  0x004225e5:   DW_TAG_inlined_subroutine
                  DW_AT_name      ("__die_body.llvm.14794269134614576759")
                  DW_AT_type      (0x00418a14 "int")
                  DW_AT_artificial        (true)
                  DW_AT_specification     (0x00422348 "__die_body")

  0x004225f1:     DW_TAG_formal_parameter
                    DW_AT_name    ("")
                    DW_AT_type    (0x004181f3 "const char *")

  0x004225f9:     DW_TAG_formal_parameter
                    DW_AT_name    ("")
                    DW_AT_type    (0x00419118 "pt_regs *")

  0x00422601:     DW_TAG_formal_parameter
                    DW_AT_name    ("")
                    DW_AT_type    (0x0041af2f "long")

  0x00422609:     NULL
  ...
  0x013f5dac:   DW_TAG_inlined_subroutine
                  DW_AT_name      ("devkmsg_emit")
                  DW_AT_calling_convention        (DW_CC_nocall)
                  DW_AT_artificial        (true)
                  DW_AT_specification     (0x013ef75b "devkmsg_emit")

  [1] #127855

Author: Yonghong Song

llvm/include/llvm/Transforms/Utils/EmitChangedFuncDebugInfo.h

@@ -0,0 +1,33 @@
+//===- EmitChangedFuncDebugInfo.h - Emit Additional Debug Info -*- C++ --*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file
+/// Emit debug info for changed or new funcs.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TRANSFORMS_UTILS_EMITCHANGEDFUNCDEBUGINFO_H
+#define LLVM_TRANSFORMS_UTILS_EMITCHANGEDFUNCDEBUGINFO_H
+
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+
+class Module;
+
+// Pass that emits late dwarf.
+class EmitChangedFuncDebugInfoPass
+    : public PassInfoMixin<EmitChangedFuncDebugInfoPass> {
+public:
+  EmitChangedFuncDebugInfoPass() = default;
+
+  PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
+};
+
+} // end namespace llvm
+
+#endif // LLVM_TRANSFORMS_UTILS_EMITCHANGEDFUNCDEBUGINFO_H

llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp

@@ -1266,11 +1266,83 @@ void DwarfDebug::finishSubprogramDefinitions() {
   }
 }
 
+void DwarfDebug::addChangedSubprograms() {
+  // Generate additional dwarf for functions with signature changed.
+  NamedMDNode *NMD = MMI->getModule()->getNamedMetadata("llvm.dbg.cu");
+  DICompileUnit *ExtraCU = nullptr;
+  for (MDNode *N : NMD->operands()) {
+    auto *CU = dyn_cast<DICompileUnit>(N);
+    if (CU && CU->getFile()->getFilename() == "<artificial>") {
+      ExtraCU = CU;
+      break;
+    }
+  }
+  if (!ExtraCU)
+    return;
+
+  llvm::DebugInfoFinder DIF;
+  DIF.processModule(*MMI->getModule());
+  for (auto *ExtraSP : DIF.subprograms()) {
+    if (ExtraSP->getUnit() != ExtraCU)
+      continue;
+
+    DISubprogram *SP = cast<DISubprogram>(ExtraSP->getScope());
+    DwarfCompileUnit &Cu = getOrCreateDwarfCompileUnit(SP->getUnit());
+    DIE *ScopeDIE =
+        DIE::get(DIEValueAllocator, dwarf::DW_TAG_inlined_subroutine);
+    Cu.getUnitDie().addChild(ScopeDIE);
+
+    Cu.addString(*ScopeDIE, dwarf::DW_AT_name, ExtraSP->getName());
+
+    DITypeRefArray Args = ExtraSP->getType()->getTypeArray();
+
+    if (Args[0])
+      Cu.addType(*ScopeDIE, Args[0]);
+
+    if (ExtraSP->getType()->getCC() == llvm::dwarf::DW_CC_nocall) {
+      Cu.addUInt(*ScopeDIE, dwarf::DW_AT_calling_convention,
+                 dwarf::DW_FORM_data1, llvm::dwarf::DW_CC_nocall);
+    }
+
+    Cu.addFlag(*ScopeDIE, dwarf::DW_AT_artificial);
+
+    // dereference the DIE* for DIEEntry
+    DIE *OriginDIE = Cu.getOrCreateSubprogramDIE(SP);
+    Cu.addDIEEntry(*ScopeDIE, dwarf::DW_AT_specification, DIEEntry(*OriginDIE));
+
+    SmallVector<const DILocalVariable *> ArgVars(Args.size());
+    for (const DINode *DN : ExtraSP->getRetainedNodes()) {
+      if (const auto *DV = dyn_cast<DILocalVariable>(DN)) {
+        uint32_t Arg = DV->getArg();
+        if (Arg)
+          ArgVars[Arg - 1] = DV;
+      }
+    }
+
+    for (unsigned i = 1, N = Args.size(); i < N; ++i) {
+      const DIType *Ty = Args[i];
+      if (!Ty) {
+        assert(i == N - 1 && "Unspecified parameter must be the last argument");
+        Cu.createAndAddDIE(dwarf::DW_TAG_unspecified_parameters, *ScopeDIE);
+      } else {
+        DIE &Arg =
+            Cu.createAndAddDIE(dwarf::DW_TAG_formal_parameter, *ScopeDIE);
+        const DILocalVariable *DV = ArgVars[i - 1];
+        if (DV)
+          Cu.addString(Arg, dwarf::DW_AT_name, DV->getName());
+        Cu.addType(Arg, Ty);
+      }
+    }
+  }
+}
+
 void DwarfDebug::finalizeModuleInfo() {
   const TargetLoweringObjectFile &TLOF = Asm->getObjFileLowering();
 
   finishSubprogramDefinitions();
 
+  addChangedSubprograms();
+
   finishEntityDefinitions();
 
   bool HasEmittedSplitCU = false;

llvm/lib/CodeGen/AsmPrinter/DwarfDebug.h

@@ -565,6 +565,8 @@ class DwarfDebug : public DebugHandlerBase {
 
   void finishSubprogramDefinitions();
 
+  void addChangedSubprograms();
+
   /// Finish off debug information after all functions have been
   /// processed.
   void finalizeModuleInfo();

llvm/lib/Passes/PassBuilder.cpp

@@ -347,6 +347,7 @@
 #include "llvm/Transforms/Utils/DXILUpgrade.h"
 #include "llvm/Transforms/Utils/Debugify.h"
 #include "llvm/Transforms/Utils/DeclareRuntimeLibcalls.h"
+#include "llvm/Transforms/Utils/EmitChangedFuncDebugInfo.h"
 #include "llvm/Transforms/Utils/EntryExitInstrumenter.h"
 #include "llvm/Transforms/Utils/FixIrreducible.h"
 #include "llvm/Transforms/Utils/HelloWorld.h"

llvm/lib/Passes/PassBuilderPipelines.cpp

@@ -133,6 +133,7 @@
 #include "llvm/Transforms/Utils/AssumeBundleBuilder.h"
 #include "llvm/Transforms/Utils/CanonicalizeAliases.h"
 #include "llvm/Transforms/Utils/CountVisits.h"
+#include "llvm/Transforms/Utils/EmitChangedFuncDebugInfo.h"
 #include "llvm/Transforms/Utils/EntryExitInstrumenter.h"
 #include "llvm/Transforms/Utils/ExtraPassManager.h"
 #include "llvm/Transforms/Utils/InjectTLIMappings.h"
@@ -1625,9 +1626,12 @@ PassBuilder::buildModuleOptimizationPipeline(OptimizationLevel Level,
   if (PTO.CallGraphProfile && !LTOPreLink)
     MPM.addPass(CGProfilePass(isLTOPostLink(LTOPhase)));
 
-  // RelLookupTableConverterPass runs later in LTO post-link pipeline.
-  if (!LTOPreLink)
+  // RelLookupTableConverterPass and EmitChangedFuncDebugInfoPass run later in
+  // LTO post-link pipeline.
+  if (!LTOPreLink) {
     MPM.addPass(RelLookupTableConverterPass());
+    MPM.addPass(EmitChangedFuncDebugInfoPass());
+  }
 
   return MPM;
 }
@@ -2355,4 +2359,4 @@ AAManager PassBuilder::buildDefaultAAPipeline() {
 bool PassBuilder::isInstrumentedPGOUse() const {
   return (PGOOpt && PGOOpt->Action == PGOOptions::IRUse) ||
          !UseCtxProfile.empty();
-}
+}

llvm/lib/Passes/PassRegistry.def

@@ -73,6 +73,7 @@ MODULE_PASS("debugify", NewPMDebugifyPass())
 MODULE_PASS("declare-runtime-libcalls", DeclareRuntimeLibcallsPass())
 MODULE_PASS("dfsan", DataFlowSanitizerPass())
 MODULE_PASS("dot-callgraph", CallGraphDOTPrinterPass())
+MODULE_PASS("dwarf-emit-late", EmitChangedFuncDebugInfoPass())
 MODULE_PASS("dxil-upgrade", DXILUpgradePass())
 MODULE_PASS("elim-avail-extern", EliminateAvailableExternallyPass())
 MODULE_PASS("extract-blocks", BlockExtractorPass({}, false))

llvm/lib/Transforms/Utils/CMakeLists.txt

@@ -22,6 +22,7 @@ add_llvm_component_library(LLVMTransformUtils
   Debugify.cpp
   DeclareRuntimeLibcalls.cpp
   DemoteRegToStack.cpp
+  EmitChangedFuncDebugInfo.cpp
   DXILUpgrade.cpp
   EntryExitInstrumenter.cpp
   EscapeEnumerator.cpp