From 507e427102f4b7d2019a4ca9b2f9ff24c24eaf2f Mon Sep 17 00:00:00 2001 From: Daniel Wilkowski Date: Sat, 31 Dec 2022 12:55:24 +0100 Subject: [PATCH] Update test\ReadMe.md --- test/ReadMe.md | 99 ++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 75 insertions(+), 24 deletions(-) diff --git a/test/ReadMe.md b/test/ReadMe.md index 21552819d..beac6c199 100644 --- a/test/ReadMe.md +++ b/test/ReadMe.md @@ -1,32 +1,83 @@ -## Where to place new tests +## About tests in T-Regx -Currently, we have 4 roots for automatic tests in T-Regx: +Currently, we have 4 roots for automatic tests in T-Regx: `Feature`, `Legacy`, +`Structure` and `Supposition`. -- `Feature` (a.k.a. "end to end") - This category has two goals: - - Ensure that a certain functionality works "out of the box" (as if used by an end user). - - Ensure that each dependency is integrated properly with other dependencies. - - Is immute to refactoring of internals +In order to minimise misconceptions we don't name the folder `Unit\ `, since the +many users and contributors might bring in their understanding of what "unit test" +is or isn't, we clarify exact criteria T-Regx tests must meet. - Rules: - - Each test-case must be created from `pattern()` function or one of `Pattern` factory methods. +## Overall rules for tests -- `Interaction` - You'd like to test behaviour - then test only the part of your interface that's relevant to the - feature domain, and not each element separately - write the smallest tests possible. Try to keep coupling of tests - with logic as low as possible. +- a test **must not** use or reference a class from `Internal\ ` for any reason. - Rules: - - There should be as little tests in this category as possible (preferably 1 per functionality). More throughout - testcases should be in `Unit` or `Feature`. - - If possible, any dependencies should be real instances, instead of fakes. If instantiating becomes too complicated, - occasional fakes are allowed, to make tests easier to read and edit. +- a test **must** be small: + - an ideal test spans 1-3 lines + - a test up to 4-10 lines is allowed, if the test cannot be split into more + smaller tests + - a test suite class with 300 small tests is preferential to a suite with 50 + larger tests covering the same features. + - increasing quantity of tests **is not** considered a flaw. + - decreasing specifity of tests **is** considered a flaw. -- `Unit` - This is the category that the coverage is measured against (if there are no tests for a class in this - category, it's as thought it's not covered). Rules: - - 100% of dependencies must be faked. Only a change in the tested class can cause the test to fail. +- a test **must** execute fast. + - frequent and rapid execution of tests is desirable + - while developing T-Regx, execution of tests should be possible at any time, in + rapid successions, even after a single character change in a source code. +- a test **must not** alter another test. + - execution of a test **must not** influence in any way the execution of another + test, in particular: by triggering PHP errors (which are preserved between + executions), by setting errors handlers which influence exceution of + error-sensitive tests, set any state between tests. + - a great deal of dedication and care must be put to tests which cover error + handling functionallity in T-Regx. T-Regx library operates on errors and + error handlers, and proper coverage of such functionallity requires an even + greater deal of logic around errors and error handlers, in a way that is + sensitive enough to detect regression, and at the same time is resilient + enough not to influence other tests. -- `Functional` - tests in `Functional` cover T-Regx's contract with PHP methods, errors, warnings, etc. +### Category `Feature\ ` - Rules: - - Tests for breaking changes between PHP versions should be in `Functional` - - Tests for `preg_` and `preg::` should be in `Functional` - - Tests for reaction of T-Regx on warnings, notices, errors, fatals and +Standard T-Regx tests are in `Feature\ ` folder. Tests in `Feature\ ` folder meet +certain criteria that we believe increase the quality of the library. + +Criteria for tests in `Feature\ ` must follow: + +- a test in `Feature\ ` should cover a very narrow part of functionallity. An ideal + test covers as little piece of feature as possibile, provided the piece of feature + is non-trivial. Any test that can be split into two more specific tests, should be + split. + +- tests in `Feature\ ` should be groupped by their relation to the `Pattern` class. +- a test covering a particular function or method should be placed in a directory + with named as the covered function or method (e.g. `flatMap\ `, `match\ `, `stream\ `). +- a test covering other part of functionallity but not pertaining for a particular + method must be placed in a directory prefixed with `_` (e.g. `_trailing_backslash`, + `_noAutoCapture`, `_jit`). + +### Category `Supposition\ ` + +T-Regx handles varying and changing behaviours of PHP, and so certain features of the +library rely on a particular behaviour of a given PHP function. Tests in category +`Supposition\ ` assert that very behaviour, so when the behaviour is prooved to change, +the tests can recognize the improper assumption, as opposed to a simple bug in the +library. + +Criteria for tests in `Supposition\ ` must follow: + +- a test in `Supposition\ ` must not use or reference any T-Regx logic. +- a test in `Supposition\ ` must only assert the current behaviour of PHP. + +### Category `Structure\ ` + +Tests in category `Structure\ ` verify the hierarchy of T-Regx exceptions and whether +the correct relation of the exceptions is followed for the sake of more general +or specific `try`/`catch`. + +### Category `Legacy\ ` + +A previously used category, but now deprecated and planned to be removed in the future. +Tests in category `Legacy\ ` break the encapsulation of T-Regx library and slows down +refactoring of many features. + +No new tests should be added into that category.