[Feature]implement multi-level testing framework with pytest #313
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
This PR introduces a comprehensive pytest-based testing framework to enhance our testing capabilities. The new framework supports:
Multi-level testing: Structured test execution across Unit, Smoke, Regression, and Release levels
Flexible tagging: Support for feature and platform-specific test marking (e.g., GPU/NPU)
Performance data collection: Integrated infrastructure for pushing metrics to InfluxDB (implementation pending)
Allure reporting: Framework setup for generating detailed test reports (implementation pending)
The purpose is to establish a standardized, scalable testing architecture that enables targeted test execution, better test organization, and future integration with performance monitoring and reporting tools.
Modifications
This PR introduces user-facing changes in the form of:
New directory structure under tests/ with organized test suites by level
Configuration files (pytest.ini, config.yaml) for test framework settings
Common utility modules for configuration handling, InfluxDB integration, and Allure reporting
Standardized test templates and naming conventions
Command-line options for filtering tests by stage, feature, and platform
These changes establish the foundation for consistent test development and execution across the project.
Test
The framework has been validated through:
Local execution of sample test cases at different levels
Verification of tag-based filtering using --stage, --feature, and --platform parameters
Confirmation that the directory structure and import paths work correctly
Basic CI validation ensuring the test discovery works as expected
The core framework functionality (test discovery, filtering, execution) is operational, while InfluxDB integration and Allure reporting will be implemented in follow-up PRs.