Recommendation

Situation Recommendation

Image Preprocessing

1. Get clothes in images using Mask R CNN.
2. Resize masked clothes for CNN.
3. Combine top(shirts) and bottom(pants) in one (64, 128) image.

Situation Classification: CNN Model

1. Image Classification using CNN.
2. Labeling images by categories.

for idx, cat in enumerate(categories):
    # labeling
    label = [0 for i in range(num_cat)]
    label[idx] = 1
    # image directory
    image_dir = lookbook + "/" + cat
    files = glob.glob(image_dir+"/*.jpg")
    print(len(files))
    for i, f in enumerate(files):
        # img -> rgb
        img = Image.open(f)
        img = img.convert("RGB")
        img = img.resize((image_w, image_h))

        # transform to numpy array
        data = np.asarray(img)
        X.append(data)
        Y.append(label)

3. Modeling CNN (Activation Function: relu, softmax / Overfit Prevention: dropout).
   using keras Library
   

from keras.models import Sequential
from keras.layers import MaxPooling2D
from keras.layers import Conv2D
from keras.layers import Activation, Dropout, Flatten, Dense

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=train_X.shape[1:], padding='same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

---

Item-based Recommendation

Data Flow

How to get item-based recommendation?

1. Get similarity between all set of items.
2. Sorting by similarity.
3. Return to application.

Average Color

Smart Coordinator uses Average Color when it computes similarity. Using masked image, In mask(not ROI), computes average color with each pixels.

def compute_average_image_color(img, r):
	count, r_total, g_total, b_total = 0, 0, 0, 0
	a = r[0]
	c = r[1]
	d = r[2]
	e = r[3]
	for x in range(a, d):
		for y in range(c, e):
			r, g, b = img[x,y]
			r_total += r
			g_total += g
			b_total += b
			count += 1
	return (r_total/count, g_total/count, b_total/count)

Similarity Algorithm

def similarity(category_dist, color_dist, image_dist):
	return (category_dist+color_dist+image_dist)/3

Evaluation

SSIM

SSIM is used for measuring the similarity between two images. The SSIM index is a full reference metric; in other words, the measurement or prediction of image quality is based on an initial uncompressed or distortion-free image as reference.

Evaluation Metric is getting image distance with various color jeans. If It return accuracy more 0.7 with jeans of different colors, FALSE, If not, TRUE.

Similarity

Evaluation Metric is getting similarity with various color jeans. If It return accuracy more 0.7 with jeans of different colors, FALSE, If not, TRUE.