Watch The Video
Zeyad Deeb

By: Zeyad Deeb on November 13th, 2018

Print/Save as PDF

In the (AI)sle™ - Part III Object Localization

(AI)sle  |  Artificial Intelligence  |  data science

Consumers’ minds are susceptible to subtle subconscious cues. It is estimated that up to 40% of consumers change their minds at the point of purchase because of something they see, learn, or do (Source: Huffington Post). For example, if a shopper goes to the store and gets a craving for chocolate, she will value options with that ingredient differently than she would on a normal day. But consumer choices are getting more complicated and more diverse over time. Brands and retailers who want to succeed in attracting and retaining customers will need to rely on data to make strong business decisions. That's where we come in!

Modern marketing is a sight to behold. Brands push creative boundaries with how they represent their products. There are no clear or defined rules on what attracts consumers, so companies mix and match attributes to find the magic combination of elements that will result in more purchases.

Let’s say you have two completely different products, with two different brands, but which both use the same flavoring ingredient. Looking at the two products below, it is not immediately obvious what the difference is because they both use Hershey’s chocolate to flavor the product. You must focus on the text and re-examine the images to be sure that one is cereal and one is cookie dough. At Label Insight, we use Machine Learning algorithms to recognize and organize information such as brand name, sub-brand, product title and flavor, among others.


(Source: Pillsbury)

AIs are like babies. All they do is eat and sleep. If you feed them a constant stream of quality data and maintain an environment of regular testing and evaluation, your AI can grow to be a mature and capable algorithm, able to recognize and predict a myriad of complex product data from hundreds of brands. Label Insight uses a number of different techniques in order to develop these powerful AIs, such as Object Localization, SANs and RCNNs.

Part III - Not all products are created equal: Utilizing Object Localization @ Label Insight

Object localization allows you to utilize machine learning to identify locations of objects on images. Because of Label Insight’s large and unique dataset, we can train machine learning models to help brands and retailers identify which brands appear on product packages, thus helping companies make informed decisions about how to organize their store aisles. Consider the example of a store which has low inventory turns, which is making management nervous for fear of spoilage. Partnering with Label Insight means they can use our data to analyze the store’s product lineup and identify what products are similar to each other in terms of ingredients, flavor, or brand name. This would allow the company to re-evaluate which products fill a need, and which are taking up valuable shelf space.


Stacked Attention Networks (SANs) & Region-based Convolutional Neural Networks (RCNNs)

Stacked Attention Networks, or SANs, take the words in a given text and look for them in an image. SANs use a multi-step approach, where the image is searched multiple times to gradually uncover more information from the image. SANs can be used to quickly identify package information, using either the product image or the words used on a product, as the search query. For example, we could feed the model some images that contained the word “Hersheys” and ask it to find other products with the same word on the package. Or, we could show the model some images of food products and ask it to tell us which ones contain “Hersheys.” The network architecture looks similar to the example below, where the model is attempting to match visual information with the text provided in the sentence “A cat is sitting in the bathroom sink.”


(Source: Microsoft AI Research)

This is essentially the same as the minimum classification error in speech recognition, the similarity between image I and sentence T using the log sum of exponential pooling (LSE):


where lambda is a factor that determines how much to magnify the importance of the most relevant pairs of image region feature v and attended sentence vector a 


How we use SANs with RCNNs: Detecting brand names, product names and other information on the package

Products are as diverse as the people who buy them. While this is a good thing for consumers, it presents challenges for brands and retailers attempting to find buyers for existing products, or more often, find new products that consumers will buy. Companies in the food business have to gauge consumer sentiment and quickly shift production to meet consumers’ rapidly changing tastes. Today it is quinoa, tomorrow it might be puffed lentils. Brands risk losing valuable time and resources on failed products. With our strategic alliance with Nielsen, using Nielsen Product Insider, we provide companies with access to our product insights in combination with their sales data. For example, knowing that snacks containing puffed lentils have become a larger share of the snack portfolio in the last year would be complementary to data about the snack sales.

Whatever the situation, Label Insight has the dataset and machine learning models to help companies make informed decisions about what products already exist, and which ones are prime to fill a market need.

Curious to learn more about using Label Insight’s data? We’re helping CPG brands and retailers tackle their toughest product data challenges. Let’s connect!

About Zeyad Deeb

Data Science at Label Insight