images.edit not working when submitting image files (mime type and other issues)

[wdewind]

Hi,

I've been trying to use the images.edit api and having issues. When I submit a single file like this:

    response = openai.images.edit(
        prompt: prompt,
        model: MODEL,
        size: "1024x1024",
        image: open("test_image.jpeg"),
      )

I get the error:

Invalid file 'image': unsupported mimetype ('application/octet-stream'). Supported file formats are 'image/jpeg', 'image/png', and 'image/webp',

If I wrap it in an array:

response = openai.images.edit(
        prompt: prompt,
        model: MODEL,
        size: "1024x1024",
        image: [open("test_image.jpeg")],
      )

I get

Invalid type for 'image': expected one of an array of files or file, but got a string instead.

What am I doing incorrectly here?

I've searched around and it seems others are having similar issues with different libraries, so this may be an open ai api issue not a ruby library issue. Ex: openai/openai-python#2341

[wdewind]

Playing around with the CLI a little and I found that the API complains about jpegs but allows me to use PNGs. The same PNG does not work from the ruby library though. This code works:

curl -s -D >(grep -i x-request-id >&2) 
  -X POST "https://api.openai.com/v1/images/edits" 
  -H "Authorization: Bearer $OPENAI_API_KEY" 
  -F "model=dall-e-2" 
  -F "image=@test_image.png"

[ms-jpq]

hi @wdewind, sorry for the bug! I was able to reproduce it.

It has to do with how we were encoding the content-type here:

openai-ruby/lib/openai/internal/util.rb

Line 500 in a8af268

content_type ||= "application/octet-stream"

We relied too much on "application/octet-stream" when the service requires "text/plain" & the JPG's content_type.

I will have to check with a colleague to identify a fix for this, since I do not want to accidentally break other endpoints when I change the encoding behaviour. Please expect a fix early next week 👍

Cheers!

[ms-jpq]

should be closed by #154