OPRA Dataset: Online Product Reviews for Affordances

Kuan Fang*, Te-Lin Wu*, Daniel Yang, Silvio Savarese and Joseph J. Lim

For questions regarding OPRA Dataset please contact Kuan Fang and Te-Lin Wu.

Introduction

The OPRA Dataset was introduced in our Demo2Vec paper for reasoning object affordances from online demonstration videos. It contains 11,505 demonstration clips and 2,512 object images scraped from 6 popular YouTube product review channels along with the corresponding affordance annotations. More details can be found on our website.

Citation

@inproceedings{demo2vec2018cvpr,
author = {Fang, Kuan and Wu, Te-Lin and Yang, Daniel and Savarese, Silvio and Lim, Joseph J.},
title = {Demo2Vec: Reasoning Object Affordances From Online Videos},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}

Requirements

Install youtube-dl:

sudo -H pip install --upgrade youtube-dl

Install ffmpeg:

sudo add-apt-repository ppa:mc3man/trusty-media  
sudo apt-get update  
sudo apt-get install ffmpeg  

To visualize the dataset using Jupyter notebooks, these packages need to be installed: NumPy, OpenCV, Matplotlib, Jupyter.

Usage

One can follow the step-by-step instructions below or just run the bash script bash ./get_dataset.sh.

Download and unzip data/ (around 500MB). The folder contains playlists of YouTube product review videos (playlists/), product images (images/), and human annotations of the video segmentation and interactiion regions (annotations/).

wget ftp://cs.stanford.edu/cs/cvgl/OPRA/data.zip
unzip data.zip

Download the product review videos from YouTube (around 51GB). (Note that some of the url may be no longer valid when you run the script, because they have been deleted from the playlist or due to other technical issues.)

python download.py --playlist data/playlists --output data/raw_videos/

Segment the videos according to the annotations (around 13GB). After segmentation, raw videos will be no longer needed and can be removed.

python segment.py --annotations data/annotations/train.txt --raw data/raw_videos/ --output data/clips
python segment.py --annotations data/annotations/test.txt --raw data/raw_videos/ --output data/clips

Demo

We provide a simple demo for loading and visualizing the dataset. Please run the Jupyter notebook:

jupyter notebook

# Or run remotely. 
jupyter notebook --no-browser --port=8888

Open ./notebooks/visualize_annotations.ipynb in the browser (Chrome and FireFox are recommended) at this link. The notebook provide a basic demo of loading the dataset and visualizing the video clip, the target image, and the annotation.