|
| 1 | +# pytest plugin for Firebird QA |
| 2 | + |
| 3 | +## Installation |
| 4 | + |
| 5 | +If you plan to use this plugin for personal purposes (not related to Firebird project QA), |
| 6 | +we recommend to use `pipx` to install `pytest` together with `firebird-qa` plugin: |
| 7 | + |
| 8 | +``` |
| 9 | +pipx install pytest |
| 10 | +pipx inject pytest firebird-qa |
| 11 | +``` |
| 12 | + |
| 13 | +## Configuration |
| 14 | + |
| 15 | +### Firebird-driver configuration |
| 16 | + |
| 17 | +The QA plugin uses firebird-driver to access the Firebird servers, and uses driver configuration |
| 18 | +object to set up various driver and server/database connection parameters. The configuration object |
| 19 | +is initialized from `firebird-driver.conf` file, and plugin specifically utilizes server sections |
| 20 | +in this file. When pytest is invoked, you must specify tested server with `–server <name>` option, |
| 21 | +where `<name>` is name of server configuration section in `firebird-driver.conf` file. |
| 22 | + |
| 23 | +This file is stored in firebird-qa repository, and defines default configuration suitable to most QA setups. |
| 24 | + |
| 25 | +Note: |
| 26 | + |
| 27 | +The `firebird-driver.conf` file should be located in QA root directory. In default setup, the QA plugin |
| 28 | +is used to test local Firebird installation with default user name and password (SYSDBA/masterkey) |
| 29 | +via local server (configuration section). |
| 30 | + |
| 31 | +Important: |
| 32 | + |
| 33 | +The firebird-driver currently does not support specification of client library in server sections. |
| 34 | +However, the QA plugin works around that limitation. If server section for tested server contains |
| 35 | +`fb_client_library` option specification, it’s copied to global setting. |
| 36 | + |
| 37 | +See configuration chapter in [driver documentation](https://firebird-driver.readthedocs.io) for details. |
| 38 | + |
| 39 | +### Pytest configuration |
| 40 | + |
| 41 | +While it’s not required, it’s recommended to create pytest configuration file in QA root directory. |
| 42 | +You may use this file to simplify your use of pytest with addopts option, or adjust pytest behaviour. |
| 43 | + |
| 44 | +Suggested options for `pytest.ini`: |
| 45 | +``` |
| 46 | +console_output_style = count |
| 47 | +testpaths = tests |
| 48 | +addopts = --server local --install-terminal |
| 49 | +``` |
| 50 | + |
| 51 | +## Running test for Firebird |
| 52 | + |
| 53 | +To run all tests in suite against local Firebird server, invoke: |
| 54 | +``` |
| 55 | +pytest --server local ./tests |
| 56 | +``` |
| 57 | + |
| 58 | +Tip: If you created `pytest.ini` with recommended values, you can just invoke pytest without additional parameters. |
| 59 | + |
| 60 | +### pytest report header |
| 61 | + |
| 62 | +When pytest is invoked, a report header is printed on terminal before individual tests are executed. |
| 63 | +The QA plugin extend this header with next information: |
| 64 | + |
| 65 | +- Python encodings |
| 66 | + |
| 67 | + - system |
| 68 | + - locale |
| 69 | + - filesystem |
| 70 | + |
| 71 | +- Information about tested Firebird server |
| 72 | + |
| 73 | + - conf. section name |
| 74 | + - version |
| 75 | + - mode |
| 76 | + - architecture |
| 77 | + - home directory |
| 78 | + - tools directory |
| 79 | + - used client library |
| 80 | + |
| 81 | +### pytest switches installed by QA plugin |
| 82 | + |
| 83 | +The QA plugin installs several pytest command-line switches. When you run pytest ``--help``, |
| 84 | +they are listed in Firebird QA section: |
| 85 | +``` |
| 86 | +Firebird QA: |
| 87 | + --server=SERVER Server configuration name |
| 88 | + --bin-dir=PATH Path to directory with Firebird utilities |
| 89 | + --protocol={xnet,inet,inet4,wnet} |
| 90 | + Network protocol used for database attachments |
| 91 | + --runslow Run slow tests |
| 92 | + --save-output Save test std[out|err] output to files |
| 93 | + --skip-deselected={platform,version,any} |
| 94 | + SKIP tests instead deselection |
| 95 | + --extend-xml Extend XML JUnit report with additional information |
| 96 | + --install-terminal Use our own terminal reporter |
| 97 | +``` |
| 98 | + |
| 99 | +**server:** |
| 100 | + |
| 101 | +REQUIRED option. Section name in firebird-driver.conf with connection parameters for tested server. |
| 102 | + |
| 103 | +**bin-dir:** |
| 104 | + |
| 105 | +Normally, the QA plugin detects and properly sets the directory where Firebird tools are installed. |
| 106 | +However, you can set this directory explicitly using the --bin-dir switch. |
| 107 | + |
| 108 | +**protocol:** |
| 109 | + |
| 110 | +Override for network protocol specified in firebird-driver.conf file (or default). |
| 111 | + |
| 112 | +**runslow:** |
| 113 | + |
| 114 | +Tests that run for longer than 10 minutes on equipment used for regular Firebird QA should be |
| 115 | +marked as slow. They are not executed, unless this switch is specified. |
| 116 | + |
| 117 | +**save-output:** |
| 118 | + |
| 119 | +_Experimental switch_ |
| 120 | + |
| 121 | +When this switch is specified, stdout/stderr output of external Firebird tool executed by |
| 122 | +test is stored in `/out` subdirectory. Intended for test debugging. |
| 123 | + |
| 124 | +**skip-deselected:** |
| 125 | + |
| 126 | +Tests that are not applicable to tested server (because they are for specific platform or |
| 127 | +Firebird versions) are deselected during pytest collection phase. It means that they are not |
| 128 | +shown in test session report. This switch changes the routine, so tests are marked to skip |
| 129 | +(with message explaining why) instead deselection, so they show up is session report. |
| 130 | + |
| 131 | +**extend-xml:** |
| 132 | + |
| 133 | +When this switch is used together with `--junitxml` switch, the produced JUnitXML file will |
| 134 | +contain additional metadata for testsuite and testcase elements recorded as property sub-elements. |
| 135 | + |
| 136 | + **Important:** |
| 137 | + |
| 138 | + Please note that using this feature will break schema verifications for the latest JUnitXML schema. |
| 139 | + This might be a problem when used with some CI servers. |
| 140 | + |
| 141 | +**install-terminal:** |
| 142 | + |
| 143 | +This option changes default pytest terminal reporter that displays pytest NODE IDs, to custom |
| 144 | +reporter that displays Firebord QA test IDs. |
| 145 | + |
| 146 | +pytest node IDs are of the form `module.py::class::method` or `module.py::function`. |
| 147 | + |
| 148 | +Firebord QA test IDs are defined in our test metadata. |
| 149 | + |
| 150 | + **Important:** |
| 151 | + |
| 152 | + Right now, the custom terminal is opt-in feature. This will be changed in some future release |
| 153 | + to opt-out using new switch. |
| 154 | + |
| 155 | + |
| 156 | +### Test for use with this plugin |
| 157 | + |
| 158 | +Please read the [plugin documentation](https://firebird-qa.rtfd.io) for instructions how |
| 159 | +to create tests that use special support provided by this plugin. |
0 commit comments