Skip to content

Add file content part support for multimodal chat completions#98

Merged
adambalogh merged 1 commit into
mainfrom
claude/wonderful-cerf-ho2n7m
Jun 8, 2026
Merged

Add file content part support for multimodal chat completions#98
adambalogh merged 1 commit into
mainfrom
claude/wonderful-cerf-ho2n7m

Conversation

@adambalogh

Copy link
Copy Markdown
Contributor

Summary

Adds support for file content parts in multimodal chat completion requests, enabling the gateway to handle PDF attachments and other file inputs alongside text and image content. This is a prerequisite for the "secure attachments" feature where PDF files arrive as OpenAI file content parts.

Changes

  • OpenAPI Schema: Added ChatCompletionRequestMessageContentPartFile schema with nested file object supporting three input modes:

    • file_data: Base64-encoded inline file data (with optional filename)
    • file_id: Reference to a previously uploaded file
    • filename: Name hint for inline data

    Updated the ChatCompletionRequestUserMessageContentPart oneOf union to include the new file part type alongside existing text, image, and audio parts.

  • Test Coverage: Added TestUserMessageContentPartValidation unit test class with schema-validation regression tests:

    • test_image_part_passes_schema_validation: Verifies image content parts pass validation (baseline)
    • test_pdf_file_part_passes_schema_validation: Verifies file content parts with PDF data pass validation

    Tests use a standalone _build_app() helper to build a connexion app directly from the OpenAPI spec, avoiding the optional flask_testing dependency and ensuring validation runs in the lean test environment.

Implementation Details

  • File parts are validated at the connexion request-body validation layer, the same pipeline that processes OHTTP inner requests
  • The schema allows flexible file input modes (inline base64, file ID, or filename) to support various attachment workflows
  • Tests confirm that PDFs no longer receive 400 validation errors (they may fail later with 500 if provider keys are missing, but that's expected in unit tests)

https://claude.ai/code/session_01HF83iVseKjA673AM4n4uN2

PDF attachments were rejected with a 400 while images worked. The secure
attachments feature added handler-side support for OpenAI `file` content
parts (convert_messages / _convert_content_part / validate_attachments) but
never added a matching branch to the request schema.

The OHTTP inner request runs through connexion's full request-body
validation pipeline. The user-message content-part `oneOf` only allowed
text / image_url / input_audio, so a `{type:"file", file:{...}}` part
matched no branch and connexion returned 400 before the handler ran.
image_url parts (JPG/PNG) passed because they had a valid branch.

Add ChatCompletionRequestMessageContentPartFile (and its `file` object with
filename / file_data / file_id) and include it in the user content-part
oneOf, plus a regression test asserting a PDF file part passes validation.
@adambalogh adambalogh marked this pull request as ready for review June 8, 2026 20:00
@adambalogh adambalogh merged commit 35a87f3 into main Jun 8, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants