Skip to content

[Feature]: Support video_url for multimodal messages #259

@genblazejohn

Description

@genblazejohn

Feature Type

Core library enhancement

Provider (if applicable)

No response

Description

The Problem

Right now, req_llm lets us send text and images (using image_url), but there is no built-in way to send a video URL.

Since models can understand video, we need a standard way to pass a video link in our messages list without writing custom upload code every time.

Proposed Solution:

I propose adding a video_url type to the message list, similar to how image_url works.

If the provider supports it (e.g., Gemini): The library should just pass the URL through to the provider's native parameter (like fileData).

If the provider doesn't support it: The library should just raise an error saying "Video URLs are not supported for this provider." It does not need to handle downloading or processing the video file itself.

Why we need this

This keeps the API consistent. Users can send text, images, and videos using the same structure.

Example Usage

messages = [
  %{
    role: "user",
    content: [
      %{"type" => "text", "text" => "What is happening in this video?"},
      %{"type" => "video_url", "video_url" => %{"url" => "[https://example.com/video.mp4](https://example.com/video.mp4)"}}
    ]
  }
]

ReqLLM.chat(messages, model: "gemini-1.5-pro")

Can you contribute this?

Yes, with guidance

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions