Repository navigation
[Test definitions] Three of the four APIs have no error scenarios #165
Description
Problem description
Three of the four feature files in code/Test_definitions/ contain no error scenario at all — every
scenario asserts a 2xx status, and no scenario anywhere in those three files asserts an error code.
| Feature file | Scenarios | Of which error cases |
|---|---|---|
dedicated-network-accesses.feature |
9 | 0 |
dedicated-network.feature |
5 | 0 |
dedicated-network-profiles.feature |
2 | 0 |
dedicated-network-areas.feature |
11 | 3 |
Counted per operation, the documented 4xx responses with no scenario are:
dedicated-network-accesses.feature— 34 declared 4xx responses across the seven operations, and
none of the 17 errorcodevalues in the response enums is ever asserted.422 MISSING_IDENTIFIER
and404 IDENTIFIER_NOT_FOUNDare both described ininfo.descriptionand neither has a scenario.dedicated-network.feature—listNetworks400/401/403,createNetwork400/401/403,
readNetwork400/401/403/404,deleteNetwork400/401/403/404. The two404s are the most
conspicuous: an unknownnetworkIdis the canonical basic error case for both operations.dedicated-network-profiles.feature—readNetworkProfiles400/401/403 andreadNetworkProfile
400/401/403/404. Two scenarios cover 2 of the 9 declared operation/status combinations.
Two further gaps apply to all four files. The API Testing Guidelines require state coherence around
resource-modifying operations — that a create is followed by a read of the created resource, and that
a delete is followed by a read expecting the documented 404. Neither pattern appears in any of the
four files, although createAccess/readAccess, deleteAccess, createNetwork/readNetwork and
deleteNetwork all declare the responses such scenarios would use.
The API Readiness Checklist defines Test Cases (basic) as "sunny day scenarios and main error cases"
and makes the asset mandatory from rc status. The Testing Guidelines put it more strictly: all errors
explicitly documented in the API specification must be covered by one or more dedicated error
scenarios.
Expected action
For each of the three files, add error scenarios covering the error responses documented for each
operation, asserting the HTTP status and the code value in each case. Add the create-then-read and
delete-then-read-404 scenarios for the operations that support them.
Where a documented code cannot be provoked, or where two documented codes are both defensible for
the same input, that is worth saying on this issue — it points at the API definition rather than at
the test.
Additional context
This was noted in a comment on #138, whose own scope was the missing listDevices scenario. That
scenario now exists and #138 is closed; the error-scenario gap is filed separately here so that it has
a scope of its own.
The operationId mismatches previously reported in #143 are fixed: all 27 operationId references
across the four files now resolve to an operation that exists.
Activity
tlohmar commentedon Aug 21, 2026
Oh, I missed the change in commonalities. I thought that only test cases for sunny day scenarios are needed.
Add error scenarios to dedicated network test definitions