Introducing new products for recognition through
3D modeling and rendering
Since image recognition utilizes neural networks, there is always a preliminary system training stage. Typically, it works the following way: photographs of the target products are fed into the system that "learns" to identify them. Only then can it recognize the same product on a random new photograph.
?
How does it work?
When starting a new implementation project, all actual client products are trained, and the system is ready for operation. Later, when new products are developed (e.g., a different yogurt flavor), the system is additionally taught to recognize them. It is an ongoing process, a normal part of system operation. However, it has an obvious drawback – after the new product appears in retail, it takes some time to collect photographs and train the networks. During this period (usually several days), the system fails to recognize the new products; therefore, some data from the outlets are missing.
One workaround includes photographing the product before it gets to the shelves. But that requires additional offline work, multiple photographs, sending the products to and from the client, disposing of the remaining packages, etc. It works relatively fine for cookies and shampoos, but perishable products like dairy may be more of a problem.
One workaround includes photographing the product before it gets to the shelves. But that requires additional offline work, multiple photographs, sending the products to and from the client, disposing of the remaining packages, etc. It works relatively fine for cookies and shampoos, but perishable products like dairy may be more of a problem.
Rendering process
One of the alternative approaches that we applied with one of our clients is system training based on 3D models of products. Since the client possesses all the product information, like the blueprints of the packages, it is possible to model the new SKUs before they are even produced and get ready for their appearance on the shelves:
Step 1
Product models are developed using the client's data: 3D models are created, and labels texture is applied (as a rule, the package changes much less frequently than the label; therefore, the models can be reused multiple times)
Step 2
The models are "laid out" on virtual shelves the same way they may have looked in the outlet.
Step 3
Step 3: A series of renders are made, imitating photographs from the outlet. One of the essential advantages is that it is possible to variate the images, providing the system with extra product appearance options on the shelf:
1
putting the product on various shelves
2
various camera positions, imitating cases when a photograph is taken not at a straight angle
3
various product rotation angles, imitating cases when a product is not facing the customer
4
various lighting options
Summarizing
As the renders look almost the same as real photographs, the system trains successfully, allowing high recognition accuracy from the start. With this option at hand and rapid training of neural network capabilities our system supports current assortment recognition, which is a crucial matter in keeping product performance analytics up to date.