Skip to content

Conversation

Copy link

Copilot AI commented Dec 18, 2025

Performance regression runs were executed via per-file subprocesses, providing poor progress visibility and brittle output parsing.

  • Pytest-backed runner: regression_all now builds a transient pytest wrapper to run all regression benchmarks in-process with -s, keeping output visible and enabling richer reporting.
  • Deterministic module loading: Bench files load via hashed, path-aware module names and safe cwd/sys.path context managers, with results aggregated from shared state.
  • Configurable invocation: Optional pytest_args passthrough for custom flags; clearer error if pytest is missing.
  • Tests: Added unit tests that stub tilelang.testing and pytest.main, validating bench execution and wrapper generation.

Example usage:

import tilelang.testing.perf_regression as pr

# Run all discovered regression benchmarks under examples/
pr.regression_all(pytest_args=["-q"])
Original prompt

This branch plans to do performance regression test between two commits, because when a developer give some pull request, we want to test these changes can lead to regression. Please use more robust to do this. Now, we do tests for each file in a subprocess. we want to use pytest etc. framework, so it can show process of tests. Refactor it.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI changed the title [WIP] Refactor performance regression tests for better reporting Refactor perf regression runner to pytest with in-process execution Dec 18, 2025
Copilot AI requested a review from xwhzz December 18, 2025 11:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants