Skip to content

Conversation

@Potterluo
Copy link

Purpose
This PR introduces a comprehensive pytest-based testing framework to enhance our testing capabilities. The new framework supports:

Multi-level testing: Structured test execution across Unit, Smoke, Regression, and Release levels
Flexible tagging: Support for feature and platform-specific test marking (e.g., GPU/NPU)
Performance data collection: Integrated infrastructure for pushing metrics to InfluxDB (implementation pending)
Allure reporting: Framework setup for generating detailed test reports (implementation pending)
The purpose is to establish a standardized, scalable testing architecture that enables targeted test execution, better test organization, and future integration with performance monitoring and reporting tools.

Modifications
This PR introduces user-facing changes in the form of:

New directory structure under tests/ with organized test suites by level
Configuration files (pytest.ini, config.yaml) for test framework settings
Common utility modules for configuration handling, InfluxDB integration, and Allure reporting
Standardized test templates and naming conventions
Command-line options for filtering tests by stage, feature, and platform
These changes establish the foundation for consistent test development and execution across the project.

Test
The framework has been validated through:

Local execution of sample test cases at different levels
Verification of tag-based filtering using --stage, --feature, and --platform parameters
Confirmation that the directory structure and import paths work correctly
Basic CI validation ensuring the test discovery works as expected
The core framework functionality (test discovery, filtering, execution) is operational, while InfluxDB integration and Allure reporting will be implemented in follow-up PRs.

@Potterluo
Copy link
Author

Purpose

This PR introduces a comprehensive pytest-based testing framework to enhance our testing capabilities. The new framework supports:

  • Multi-level testing: Structured test execution across Unit, Smoke, Regression, and Release levels
  • Flexible tagging: Support for feature and platform-specific test marking (e.g., GPU/NPU)
  • Performance data collection: Integrated infrastructure for pushing metrics to InfluxDB (implementation pending)
  • Allure reporting: Framework setup for generating detailed test reports (implementation pending)

The purpose is to establish a standardized, scalable testing architecture that enables targeted test execution, better test organization, and future integration with performance monitoring and reporting tools.

Modifications

This PR introduces user-facing changes in the form of:

  • New directory structure under tests/ with organized test suites by level
  • Configuration files (pytest.ini, config.yaml) for test framework settings
  • Common utility modules for configuration handling, InfluxDB integration, and Allure reporting
  • Standardized test templates and naming conventions
  • Command-line options for filtering tests by stage, feature, and platform

These changes establish the foundation for consistent test development and execution across the project.

Test

The framework has been validated through:

  • Local execution of sample test cases at different levels
  • Verification of tag-based filtering using --stage, --feature, and --platform parameters
  • Confirmation that the directory structure and import paths work correctly
  • Basic CI validation ensuring the test discovery works as expected

The core framework functionality (test discovery, filtering, execution) is operational, while InfluxDB integration and Allure reporting will be implemented in follow-up PRs.

@Potterluo Potterluo force-pushed the feature/tests branch 3 times, most recently from f04ce72 to 3f3e9f0 Compare October 25, 2025 08:40
@Potterluo Potterluo force-pushed the feature/tests branch 3 times, most recently from a602eb5 to a721a18 Compare October 27, 2025 06:12
…l testing, platform tagging, performance data collection, and Allure report generation.
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prevent users from executing and submitting incorrect files in the root directory.

@yuanzhg078
Copy link

Separate the README into language-specific versions. (README.md & README_zh.md)

@Potterluo Potterluo changed the title [Working]implement multi-level testing framework with pytest [Feature]implement multi-level testing framework with pytest Oct 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants