Enable Dev18 Python profiling through Diagnostics Hub

[StellaHuang95]

Context

Python profiling in Dev18 is currently unavailable. The legacy PTVS path depends on VSPerfMon.exe and VSPerfCmd.exe, which were removed from the Dev18 shipping manifest, and its native bridge only understands CPython layouts through Python 3.10. This is the PTVS half of moving basic function instrumentation to the existing Diagnostics Hub Python target and microsoft/python-etwtrace pipeline. It addresses the user-visible failure tracked by #8276.

This PR is not standalone. It requires companion Diagnostics Hub changes to:

• load the PTVS profiling command service without the failing dynamic-MEF handoff;
• treat TargetProperties.Python as script instrumentation instead of opening the native executable/PDB planner;
• resolve script module paths and function IDs correctly in DataWarehouse.

Those companion changes are currently on the DiagnosticsHub branch feature/ptvs-python-profiling.

PTVS changes

• Route Debug > Launch Python Profiling to the Dev18 Diagnostics Hub launcher while preserving the legacy behavior for older VS targets.
• Register the currently hidden PythonProfilingEnabled Diagnostics Hub feature flag for Dev18.
• Expose a package-independent PythonProfilerCommandService that Diagnostics Hub can construct dynamically with the global VS service provider.
• Rework command construction so Diagnostics Hub owns the session/report while PTVS supplies the selected interpreter, script, arguments, working directory, and environment.
• Add diaghub_profile.py, which:
  • launches a dedicated profiled child process to satisfy Diagnostics Hub's child-attach contract;
  • activates etwtrace for the lifetime of the interpreter, including worker threads;
  • supports both script and -m execution;
  • preserves target arguments and import behavior;
  • reports unsupported interpreter/wheel combinations clearly.
• Package coherent etwtrace payloads:
  • 0.1b8 for CPython 3.9-3.11;
  • 0.1b9 for CPython 3.12-3.14.
• Include CPython 3.14 wheels in the fat package restore.
• Force the intended Dev18 System.Text.Json assembly reference to remove the existing MSB3277 conflict warning.

Runtime flow

PTVS target dialog
  -> Diagnostics Hub PythonTarget
  -> Instrumentation collector starts a root Python bootstrap
  -> bootstrap launches a profiled child Python process
  -> etwtrace attaches through DiagnosticsHub.InstrumentationCollector.dll
  -> Python function definitions and enter/exit events are written to .diagsession
  -> DataWarehouse resolves function IDs for the Instrumentation call tree

Manual validation

Validated in a Dev18 Experimental instance with the companion Diagnostics Hub changes:

• the Python Script target opens the PTVS profiling dialog;
• Python 3.9 x64 launches and completes profiling;
• no native executable/PDB selection dialog appears;
• a .diagsession report is produced;
• user functions such as main, calculate_batch, transform, and wait_for_io resolve with call counts, total time, and self time;
• the targeted Dev18 Profiling project and full PTVS traversal build succeed.rn- focused bootstrap tests pass on Python 3.9 and 3.14 and through the existing ProfilingTests MSTest project.

Current limitations

• The launch page does not yet auto-select Python Script and Instrumentation; users must select both.
• Python 3.9 x64 is the manually validated runtime. The intended 3.9-3.14 matrix still needs automated and manual coverage.
• No public x86 etwtrace wheels are available, so 32-bit Python is not supported by this path.
• ARM64 and free-threaded Python builds are not yet validated.
• This does not restore the legacy .vsp/Performance Explorer pipeline.
• Attach-to-process, remote/WSL, multiprocessing descendants, native extension internals, line profiling, and memory profiling remain out of scope.
• Focused bootstrap behavior is covered; cross-repo collector/analyzer integration and the full interpreter/architecture matrix still need dedicated coverage.

Cross-repo rollout note

The DataWarehouse label fix was locally validated with an unsigned Release analyzer copied to both destinations listed by DiagnosticsHub's DataWarehouseManaged.list. Production validation requires the normal signed Diagnostics Hub build/insertion pipeline.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[heejaechang]

🔒 Automated review in progress — @heejaechang is auto-reviewing this PR.

[StellaHuang95]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

Enables Python profiling in VS Dev18 by routing the existing PTVS “Launch Python Profiling” entrypoint to Diagnostics Hub, supplying a new command/input service and bootstrap script so Diagnostics Hub can run python-etwtrace-based instrumentation while preserving legacy behavior for older VS targets.

Changes:

• Route Dev18 profiling launch to Diagnostics Hub and register the PythonProfilingEnabled Diagnostics Hub feature flag.
• Introduce a package-independent PythonProfilerCommandService + reworked command argument construction for Diagnostics Hub ownership of session/report.
• Add diaghub_profile.py plus build/packaging updates to ship coherent etwtrace payloads (legacy + current) and include CPython 3.14 wheels.

Show a summary per file

File	Description
Python/Product/PythonTools/PythonTools.csproj	Pins Dev18 System.Text.Json reference to resolve an assembly conflict.
Python/Product/Profiling/PythonProfilingPackage.cs	Routes Dev18 profiling launch through Diagnostics Hub command.
Python/Product/Profiling/ProvidePythonProfilingFeatureFlagAttribute.cs	Registers Diagnostics Hub PythonProfilingEnabled feature flag for Dev18.
Python/Product/Profiling/Profiling/UserInputDialog.cs	Decouples dialog creation from PythonProfilingPackage by using IServiceProvider.
Python/Product/Profiling/Profiling/PythonProfilerCommandService.cs	Exposes a shared, public command service for Diagnostics Hub to construct and query.
Python/Product/Profiling/Profiling/CommandArgumentBuilder.cs	Reworks argument building so PTVS supplies interpreter/script/env while Diagnostics Hub owns sessions.
Python/Product/Profiling/Profiling.csproj	Ships diaghub_profile.py and includes both current + legacy etwtrace payloads in the VSIX.
Python/Product/Profiling/diaghub_profile.py	Adds the bootstrapper that launches a child process and activates etwtrace for profiling.
Build/PreBuild.ps1	Downloads both current and legacy etwtrace payloads and removes unwanted unsigned test DLLs.
Build/install_pypi_package.py	Extends wheel selection to include CPython 3.14 (cp314).

Review details

Comments suppressed due to low confidence (1)

Python/Product/Profiling/Profiling/PythonProfilerCommandService.cs:62

• The exception handler shows a hard-coded, non-localized dialog title/message ("An unexpected error occurred" / "Error"), and it also swallows critical exceptions (OOM/AV/etc.) that should not be handled. This will produce inconsistent user-facing strings and can mask fatal failures.

            } catch (Exception ex) {
                Debug.Fail($"Error displaying user input dialog: {ex.Message}");
                MessageBox.Show($"An unexpected error occurred: {ex.Message}", "Error", MessageBoxButton.OK, MessageBoxImage.Error);
            }

• Files reviewed: 10/10 changed files
• Comments generated: 0
• Review effort level: Low

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[StellaHuang95]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[heejaechang]

Approved via Review Center.

[Copilot]

Review details

Comments suppressed due to low confidence (1)

Python/Product/Profiling/Profiling/PythonProfilerCommandService.cs:62

• Avoid hard-coded UI strings in product code. This dialog title is the literal "Error" and the message prefix is not localized; elsewhere in the profiling UI, message boxes use resource strings (e.g., Strings.ProductTitle / other Strings.). Consider using Strings.ProductTitle for the title (and ideally a Strings. resource for the message prefix) to keep localization consistent.

            } catch (Exception ex) {
                Debug.Fail($"Error displaying user input dialog: {ex.Message}");
                MessageBox.Show($"An unexpected error occurred: {ex.Message}", "Error", MessageBoxButton.OK, MessageBoxImage.Error);
            }

• Files reviewed: 13/13 changed files
• Comments generated: 0 new
• Review effort level: Low

[StellaHuang95]

PR was also sent on the diagnostics hub side: https://devdiv.visualstudio.com/DevDiv/_git/DiagnosticsHub/pullrequest/763766