How can I preprocess image for custom data.

[Hacetate]

THANKS for your creative Code for multi-view hair reconstruction. But I'd like to know How can I preprocess the RGB image to such the "orientation2d.exr"/"intensity.exr"/"confidence.exr". If convenient, please provide standard code for preprocessing data😉 because trying to align data is annoying😭.

[elerac]

Hello, @Hacetate !

Thank you for trying my code.

I want to upload the entire code of preprocessing but it is messy. But, I already provided a function to get orientation2d and confidence maps from an intensity image.

If you have already succeeded in compiling the cpp extension, you could use the generate_orientation_map function.

import cv2
import strandtools


img_intensity = cv2.imread("your_filename.exr", -1)  
# Here, you may need to convert BGR to Mono
height, width = img_intensity.shape  # the image is mono, not color

# Number of angles. higher is better 
num_angles = 36  # 180 ~ 360 is best but takes long time

# Parameter for GaborKernel (you may need to change it depends on the resolution of image)
# See opencv's getGaborKernel
# https://docs.opencv.org/4.x/d4/d86/group__imgproc__filter.html#gae84c92d248183bd92fa713ce51cc3599
sigma = 2.0
lambd = 6.0
gamma = 0.25 

img_orientation2d, img_confidence = strandtools.generate_orientation_map(img_intensity, num_angles, sigma, lambd, gamma)

Today, I attempted to check the code above, but unfortunately, my Python&CMake environment failed to compile the extension. As a result, I was unable to check it, but I believe it should work.
If you try the code, please let me know.

[SharmaVibhor]

dataprep.txt
Hi,

I tried to generate results for my own images using the above script. But the results for confidence.exr are blank.
I will attach the script I am using.

[elerac]

Hi @s58vshar

What is the exact meaning of the blank?
Does the blank mean that the image is not saved, or that the image values are filled with zero, or something else?

[SharmaVibhor]

The image is completely white. I have attached the rgb image, intensity, orientation as well as confidence image.
demo.zip

[elerac]

I checked the images and confirmed that the confidence image contains the values. I attached a screenshot of the image opened with tev. By clicking the "Normalize" button, tev visualizes it to the appropriate brightness level, and we can zoom in and check the values. As you can see, the typical values are higher than one, mapped to a white color in the standard image viewer. You can use this image by directly reading it as a float array in your code.

[SharmaVibhor]

Hi @elerac ,

Thank you for pointing that out. I will try to view the normalized images and then try to reproduce the results again.

[SharmaVibhor]

Hi, I tried to reproduce the results but they seem to filter out majority of points (99% to be exact). I have tried to view intermediate results which are the output of LPMVS and consistency maps and they seem really different from data provided in the paper.
I suspect the confidence.exr for sure because the confidence images provided in dataset are quite similar to a mask while my images look really different.
I also wanted to ask if you rendered all of the images out of blender or did you use the same algorithm to convert intensity images into confidence and orientation images?
consistency.zip

Some of my results are attached.