Response validation

[y-luis-rojo]

I have an OpenAPI v3 specification file with the following (showing just fragments):

paths:
  /global/name:
    get:
    description: Some description
    tags:
      - Global settings
    operationId: getGlobalSettingsName
    responses:
      # Response code
      '200':
        description: Successful response
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/globalSettingsName'

components:
  schemas:
    globalSettingsName:
      type: object
      properties:
        name:
          type: integer
          description: 'ID'
          example: 1
      required:
        - name

but the server response is:

{
  "name": "somestring"
}

Note the name property type is integer and in the server response, it is a string (on purpose) but dredd request passes (success).

Doesn't dredd check for response property types?

I redefined the response as string (not JSON):

responses:
    # Response code
    '200':
      description: Successful response
      content:
        application/json:
          schema:
            type: string

and dredd doesn't complain about either.

I even changed the property of the schema:

    globalSettingsName:
      type: object
      properties:
        www:
          type: string
          description: 'some description'
          example: 'somestring'
      required:
        - www

And same (success) result when it is expected to fail.

Aren't these validation supported by dredd? Am I using specification wrong?

To Reproduce
dredd

Expected behavior
Request fail.

What is in your dredd.yml?

color: true
dry-run: null
hookfiles: "hooks.js"
language: nodejs
require: null
server: null
server-wait: 3
init: false
custom: {}
names: false
only: []
reporter: []
output: []
header: []
sorted: true
user: null
inline-errors: false
details: true
method: []
loglevel: error
path: []
hooks-worker-timeout: 5000
hooks-worker-connect-timeout: 1500
hooks-worker-connect-retry: 500
hooks-worker-after-connect-wait: 100
hooks-worker-term-timeout: 5000
hooks-worker-term-retry: 500
hooks-worker-handler-host: 127.0.0.1
hooks-worker-handler-port: 61321
config: ./dredd.yml
blueprint: docs/user/configuration/openapi.yaml
endpoint: "http://10.22.22.20:9000/server"

What's your dredd --version output?

dredd v8.0.5 (Linux 3.10.0-957.5.1.el7.x86_64; x64)

Does dredd --loglevel=debug uncover something?
No

Can you send us failing test in a Pull Request?
Yes

[honzajavorek]

Thanks for reporting the issue. According to the OAS3 support status (it's still very experimental) the type should be already supported. I wonder whether the OAS3 adapter already correctly generates the schemas for Dredd in this case. @kylef would you be so nice and check where the problem could be? If there is something we don't support yet, I think there would be a warning about it 🤔

[kylef]

The status for Dredd at apiaryio/api-elements.js#71 remains up to date, see this part specifically:

Validation of response body is done from an example value in content. Therefore you should include an example to get response body validation

That said, we have an update coming out soon (https://github.com/apiaryio/api-elements.js/blob/4139afabdc50750380afbe65bf2d01f41ec4466b/packages/fury-adapter-oas3-parser/CHANGELOG.md) which contains more support for examples (referencing Example Object) and inferring an example from a schema (provding the schema is not a reference at the moment). Producing a schema is planned, but this won't happen until our schema parser is improved somewhat. I don't think it's worth us targetting old JSON Schema specs and I think we should go straight for draft-07 (or draft-08 depending on when that comes out) so it will also be worth thinking about making Dredd compatible with the newer versions of JSON Schema.

[y-luis-rojo]

Thanks, @honzajavorek and @kylef, I will modify the specification to satisfy this validation. Looking forward to new updates.