Researchers in Amazon’s Seller Partner Services organization, together with colleagues at the University of California, Berkeley, have publicly released a massive dataset of product images and associated metadata to support research on product-related information management, information retrieval, and visual understanding.
The dataset could, for instance, help enable new, more-powerful AI models for image-based shopping or expansion of retailers’ product graphs.
“Computer vision is an empirically driven field, and its progress has been shaped by datasets,” says Jitendra Malik, the Arthur J. Chick Professor of Electrical Engineering and Computer Science at Berkeley, whose group helped develop the dataset. “In traditional image or video collections, we have very little information about a specific object — typically just a category label, like ‘chair’. For the objects in ABO, we have attributes, multiple views, and CAD models. This lets us infer much more from an image. We expect this will advance research in multiple areas of computer vision, particularly 3-D inference.”
Credit: Image source, ABO; animation by Gupta Media
Dubbed the Amazon Berkeley Objects Dataset, or ABO, the dataset includes images of 147,702 products, all annotated with metadata such as multilingual title, brand, model, year, product type, dimensions, and material. There are 398,212 static catalogue images; 8,200 images that provide 360° rotations in the plane at 5° intervals (for a total of 72 perspectives per product); and 7,900 fully 3-D product models that can be rotated along any axis and rendered in any 3-D environment under any lighting conditions.
The dataset is licensed under the Creative Commons Attribution-NonCommercial 4.0 International Public License (CC BY-NC 4.0), which prohibits commercial use of the dataset but is otherwise nonrestrictive.
Collectively — still images, thumbnails of the still images, 360° rotations, and 3-D models — the size of the dataset is almost 300 gigabytes. On the dataset website, researchers can download the entire dataset; browse product images filtered according to product name or type; or learn how to use a version of the dataset hosted by Amazon Web Services without having to download it.
“Data has become the most important component of AI and machine learning,” says Matthieu Guillaumin, a senior applied scientist at Amazon who helped develop the dataset. “During this project, we have been driven by the hope to spark major innovations in product understanding that have the potential to benefit shoppers worldwide.”
