How Rajeev Rastogi’s machine learning team in India develops innovations for customers worldwide

As vice president of machine learning at Amazon India, Rajeev Rastogi is helping his team drive innovations that have a profound impact not only on shoppers in India, but also on the company’s customers around the world. For example, models developed by Amazon’s scientists in India have been used globally to improve the quality of Amazon’s catalog by ensuring that for all products, images match with the title. In addition, including delivery speed as a feature in search ranking — a key factor that helps surface ‘faster’ offers to customers in search results — was first launched in Amazon India.

Rastogi began his career at Bell Labs. His early work involved developing clustering algorithms that could scale — a significant innovation in a field that was then dominated by statisticians working on relatively smaller data sets. Rastogi also served as the vice president of Yahoo Labs, where his team developed data-extraction algorithms to pull structured information from billions of webpages, and then present them to users in easily digestible ways.

Rastogi joined Amazon in 2012. His first Amazon project involved the development of algorithms to classify products into Amazon’s large and complex taxonomical structure — for example, to classify a Samsonite luggage set in ‘Carry-On Luggage,’ ‘Suitcases’ and ‘Luggage Sets.’ Since then, Rastogi has been involved in utilizing science to make an impact in a number of areas that have resulted in faster, more seamless and sustainable, shopping experiences.

In this interview, Rastogi spoke about the projects his teams have worked on to improve the shopping experience for Amazon’s customers, a recently developed statistical model that has helped Amazon reduce product-shipment damage in India, and innovations developed to help customers get what they need safely after the outbreak of the COVID-19 pandemic.

Q. What are some of the ways that science has helped improve the shopping experience for Amazon’s customers in India?

India is a unique market in several important ways. There are more than 600 million people online in the country. Many of them are relatively new to digital shopping. Over 85% of our traffic comes from a diverse range of mobile devices.  To complicate matters, mobile customers in India can experience fluctuating speeds due to congested towers and tower switching.

We’ve developed models to predict customers who are on a slow or spotty network based on criteria like device characteristics, cell tower information, and the latency of the last request. For such customers, we provide an adaptive experience and serve streamlined pages with a lower number of widgets that are easier to navigate.

With more than 22 languages and 19,500 dialects, India is also an incredibly diverse country with strong regional preferences. A customer searching for a sari in Gujarat may be interested in a “Bandhani,” which is popular in that state, while a customer in Karnataka searching for a sari may be looking for “Mysore Silk,” a popular variety in that region. To surface regionally popular and relevant products in search results, we have added regional sales for products as a feature in search.

A key problem in India and other emerging countries is that addresses are highly unstructured; they are also incomplete, with critical address fields such as street name missing from the address. For example, we have seen addresses on Amazon.in such as “Near Orion Mall, Malleswaram, Bangalore”, or “Near Bus Stand, Sambhaji Chowk, Nasik”. Our team has developed a machine-learning-based “Address Deliverability Score” to identify poor quality and incomplete addresses that are difficult to locate and deliver to, and intercept them at address creation time to improve address quality.  

You can also have issues related to catalog quality. For example, important attribute values such as the color of a product may be missing for a product. This means that a shoe might be red, and yet might not show up in the list of results for a customer searching for ‘red shoe.’

We use a variety of deep learning models to improve catalog quality by extracting attributes such as color from product titles and images, and backfilling missing product information. To give just one example, we use attention mechanisms to focus the attention of convolutional neural networks on parts of the image from where we want to extract the color of a product. 

We also utilize semi-supervised learning techniques to train neural networks extensively, which greatly reduces the need for large amounts of labeled data. What I love about this approach is that unlabeled data can be a treasure trove of information, particularly for understanding higher-level representations. For example, an algorithm can analyze text patterns around words to understand that ‘car’ and ‘automobile’ are similar without having to explicitly specify that they are synonyms.

India is a market unlike any other in the world, and I’m proud of how we are using science to solve some really difficult problems for our customers.

Q. How are you using science to make Amazon more sustainable?

Amazon has committed to reach net zero carbon by 2040, one decade ahead of the Paris Agreement. Science will play an extremely important role in enabling innovations that will make this happen.

Let me give you just one example. At this year’s European Conference on Machine Learning, members of my team presented a new model for determining the best way to package a given product. We’ve all seen customers not happy about damaged products and excessive product packaging. Incorrect packaging is not only wasteful and bad for the environment, but it also increases our packaging and concessions costs.

India is a market unlike any other in the world, and I’m proud of how we are using science to solve some really difficult problems for our customers.

Determining the optimal way to ship a product is complicated. Because one product is rarely shipped across all different package types, you run into situations where there’s a lack of ground truth data. In addition, we have the problem of enforcing ordinality into the process. We have to predict higher probabilities of damage for less expensive (less robust) packaging options, and lower probabilities of damage for more expensive (more robust) options. Enforcing ordinality is not something that standard machine learning techniques do naturally.

The solution developed by my team is as elegant as it is simple. Our scientists developed a linear model, with carefully designed constraints on the model parameters to impose ordinality. To further enforce ordinality, we used data augmentation. This means that for a product-package pair that resulted in product damage, we added examples of that product coupled with less robust packages, also labeled as resulting in damage. 

We’ve applied the model to hundreds of thousands of Amazon packages, reducing shipment damage very significantly while actually saving on shipping costs. This innovation is a testament to the incredible scientific talent at Amazon India. It also speaks volumes of our desire and our ability to take on the really big problems — those that have a significant impact on the lives of our customers and the world at large.

Q. What are some scientific innovations from your team to help customers get what they need safely during COVID-19?

As soon as the pandemic struck, I became interested in what we could do as scientists to keep people safe, and help them get what they need during these trying times. Could we use technology to generate an infection risk score for each individual? These scores could be leveraged by governments and organizations to prioritize testing and identify individuals to quarantine.

We all know that COVID-19 spreads through contacts. Many governments have developed contact tracing apps that use Bluetooth signals on mobile phones to track social contacts among individuals. However, it is challenging to use this fine-grained contact data of individuals to estimate an infection risk score for each individual. This is because the probability of infection transmission through a contact depends on the duration, distance, and location (indoors, outdoors) of the contact. Furthermore, individuals may have indirectly come in contact with a person who has tested positive for COVID-19. Or they may have come in contact with an infected person, but during the period when he or she was not contagious.

I worked with fellow scientists to develop a probabilistic graphical model called CRISP for COVID-19 infection spread through contacts between individuals. The model builds off the SEIR (Susceptible-Exposed-Infectious-Removed) approach that is commonly used to track the different epidemiological status of individuals. Our model captures the transitions between these different states, while also accounting for test outcomes. We developed a block-Gibbs sampling algorithm to draw samples of the latent infection status of each individual, given data about contacts and test results. These infection status samples are then used to compute infection risk scores for each individual. We also developed a Monte Carlo Expectation Maximization (EM) algorithm to infer the infection transmission probability for each contact taking into account factors such as contact duration, distance, and location.  

Also during the pandemic, our operations team built virtual pickup points to deliver packages to customers who live in quarantined apartment buildings. The problem: identifying customers who live in these buildings and educating them about the virtual pickup points. We used address segmentation machine learning models to extract apartment building names from delivery addresses input by customers. We then sent emails to these customers notifying them about the new features. Customers were really excited about this new feature — the email open rates announcing virtual pickup points were higher than 50%.

I’ve been at Amazon for eight years now. I joined Amazon because I was excited at the prospect of conducting scientific work that had the potential to have a real-world impact. And what was true back then remains true today — I come to work every day invigorated at the potential of making a difference in the lives of millions of people around the world.