Letterbox of images

[hardikdava]

Search before asking

• I have searched the Supervision issues and found no similar feature requests.

Description

It would be great to have a feature which resize images as per letterbox concept i.e. resize based on maximum size and adding padding to an image. This is very helpful resize method to increase result quality.

Use case

• Best method for preprocessing image before detector, segmentation models.

Additional

def letterbox(im, new_shape=(640, 640), color=(114, 114, 114), auto=True, scaleFill=False, scaleup=True, stride=32) -> np.ndarray:

use case:

import supervision as sv
import cv2

image = cv2.imread(...)
image = sv.letterbox(image)

detection_result = detector(image)

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[SkalskiP]

Hi @hardikdava 👋🏻 Yup! That's an excellent idea. But then your detection_result is in the context of letterbox so we would need to have a utility capable of reversing this process.

[hardikdava]

Hi @SkalskiP 👋 , you are absolutely correct.

[SkalskiP]

@hardikdava, what is the meaning of all of those arguments? Is that function definition copied from YOLOv8?

[hardikdava]

@hardikdava, what is the meaning of all of those arguments? Is that function definition copied from YOLOv8?

Yeah, for a quick info. But we can modify them and use only needed.

[SkalskiP]

We probably should if we don't want licensing issues.

[EduardoPach]

Hey @SkalskiP how are you doing on this? If I'm not mistaken this is the same as used in DETR transformation and we could copy it from the transformers library. Let me know if you have the bandwidth to take care of this issue otherwise I'd love to help out

[SkalskiP]

This issue was implemented via #1028. sv.letterbox_image will be released in supervision-0.20.0.

[hardikdava]

Hello @SkalskiP @LinasKo 👋 , do we have any function which rescale detection to original image after letterbox image?

[LinasKo]

Hi @hardikdava 👋

Good to see you in supervision again :)

Unfortunately not. Letterbox resizes the image and then applies letterboxing, so undoing the effect is tricky without knowing the exact params it used. Exposing those would make the API messy.

[hardikdava]

@LinasKo can't we recalculate them based on knowing letterbox_resolution, input_image_resolution?

[LinasKo]

My bad, you're right. I thought it was automatic.

I'm very short on time this week, but would be happy to review a PR or implement it when I have a spare minute. Should be a tiny feature.

Would that be something you have time for, or shall I put it on my backlog?

[hardikdava]

@LinasKo I have opened up a PR to support this functionality.

#1708