Skip to content

[Test definitions] Three of the four APIs have no error scenarios #165

Closed

Description

@hdamker

Problem description

Three of the four feature files in code/Test_definitions/ contain no error scenario at all — every
scenario asserts a 2xx status, and no scenario anywhere in those three files asserts an error code.

Feature file Scenarios Of which error cases
dedicated-network-accesses.feature 9 0
dedicated-network.feature 5 0
dedicated-network-profiles.feature 2 0
dedicated-network-areas.feature 11 3

Counted per operation, the documented 4xx responses with no scenario are:

  • dedicated-network-accesses.feature — 34 declared 4xx responses across the seven operations, and
    none of the 17 error code values in the response enums is ever asserted. 422 MISSING_IDENTIFIER
    and 404 IDENTIFIER_NOT_FOUND are both described in info.description and neither has a scenario.
  • dedicated-network.feature — listNetworks 400/401/403, createNetwork 400/401/403,
    readNetwork 400/401/403/404, deleteNetwork 400/401/403/404. The two 404s are the most
    conspicuous: an unknown networkId is the canonical basic error case for both operations.
  • dedicated-network-profiles.feature — readNetworkProfiles 400/401/403 and readNetworkProfile
    400/401/403/404. Two scenarios cover 2 of the 9 declared operation/status combinations.

Two further gaps apply to all four files. The API Testing Guidelines require state coherence around
resource-modifying operations — that a create is followed by a read of the created resource, and that
a delete is followed by a read expecting the documented 404. Neither pattern appears in any of the
four files, although createAccess/readAccess, deleteAccess, createNetwork/readNetwork and
deleteNetwork all declare the responses such scenarios would use.

The API Readiness Checklist defines Test Cases (basic) as "sunny day scenarios and main error cases"
and makes the asset mandatory from rc status. The Testing Guidelines put it more strictly: all errors
explicitly documented in the API specification must be covered by one or more dedicated error
scenarios.

Expected action

For each of the three files, add error scenarios covering the error responses documented for each
operation, asserting the HTTP status and the code value in each case. Add the create-then-read and
delete-then-read-404 scenarios for the operations that support them.

Where a documented code cannot be provoked, or where two documented codes are both defensible for
the same input, that is worth saying on this issue — it points at the API definition rather than at
the test.

Additional context

This was noted in a comment on #138, whose own scope was the missing listDevices scenario. That
scenario now exists and #138 is closed; the error-scenario gap is filed separately here so that it has
a scope of its own.

The operationId mismatches previously reported in #143 are fixed: all 27 operationId references
across the four files now resolve to an operation that exists.

Activity

  1. tlohmar commented on Aug 21, 2026

    @tlohmar
    Contributor

    Oh, I missed the change in commonalities. I thought that only test cases for sunny day scenarios are needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions