NOTAVIS / References / Smart refrigerator

Putting AI into a refrigerator

Smarter Tipp von Thomas Schweitzer

Embedded vision efficiently integrated

Fast, cost-effective & risk-free thanks to proven components
Smarter Tipp von Thomas Schweitzer
Proven components
Cost effective development

Flexible choice of processor board

Technology widely transferable

Prototype for a smart wine cooler
developed by NOTAVIS and DAB-Embedded

Smart technology simply integrated

 

Cameras and their data are the drivers for smart home applications, for staff-free supermarkets and for the analysis of user behavior. More and more applications are therefore being equipped with the intelligent sensors. The challenge here is the optimal integration of the complex technology. Using the development of a wine cooler as an example, NOTAVIS shows how companies can use embedded vision to achieve the ideal solution and thereby save time and costs on the way to market.

Which wine bottle was taken out of the refrigerator?

 

As part of a joint customer project between DAB-Embedded and NOTAVIS, a solution was to be developed to answer this question. In order to enable AI-based analyses of user behavior and to be able to identify bottles in any position without any doubt, cameras were to be used as sensors for this purpose. They provide the necessary data depth and resolution for the project.

 

 

The challenge:
Perfect integration of the cameras

The biggest challenge in designing the smart refrigerator was integrating the cameras. The prototype wine cooler is equipped with a pull-out drawer system made of metal rods. It offers the best compromise between space utilization and compact dimensions of the unit as well as a good overview of the assortment for the viewer. The metal rods make the bottles visible to the cameras from above and below. However, they must be positioned in such a way that they can capture all shelves and bottles perfectly.

From the first idea to the prototype

1. The optimal camera position

After several tests with different positioning of the cameras, a solution turned out to be optimal in which two cameras including lighting were integrated in an automatically extending arm in the lid and base of the wine cooler. This allows all bottles on the shelves to be displayed from above or below. This setup had to be implemented as uncomplicatedly as possible and based on standard components – both for the prototype and for the later standard version.

2. Data processing directly on the embedded system

Embedded vision with MIPI camera modules and a directly connected CPU board has emerged as the most suitable technology for integration into the wine cooler design: Image acquisition, data processing and analysis take place directly on the systems. There is no need for an additional, external computing unit, so the wine cooler can be used autonomously at any location. In addition, the systems dispense with all unnecessary functionalities and components, making them as compact and efficient as possible.

„The embedded vision systems from NOTAVIS' range convinced us because they are perfectly adapted to the application and offer the best compromise between performance and ultra-compact design. This has made integration extremely easy.“

Artisiom Staliarou, CEO DAB-Embedded

3. Standard components save development time and costs

MIPI camera modules from Vision Components, which NOTAVIS offers in numerous variants, are used for this purpose. They are hardly larger than the image sensors themselves, are available for long periods and, thanks to their industry-standard quality, are ideally suited for series production. They are also characterized by an optimum price/performance ratio. NOTAVIS supplies the corresponding LED illumination as an accessory. To ensure the sharpness of the images, the wine cooler drawers are equipped with sensors for position detection. The cameras, in turn, have electromagnetic adjustment of the optics to adjust the focus accordingly. In the implementation chosen for the prototype, the top two shelf pullouts are captured by the camera mounted on the bottom of the wine cooler, while the bottom three of the total of five pullouts are recorded from above.

Die Serienqualifizierung

1. MIPI modules allow flexible choice of processor platform

Another advantage of the VC MIPI camera modules: They can be connected to all common processor platforms via the MIPI-CSI-2 interface. As a subsidiary of VC, NOTAVIS supplies the corresponding drivers directly in the source code, so that the modules can be put into operation quickly and easily. MIPI cables and other accessories, for example for connecting an external trigger, are also available.

2. Image processing and analysis on the CPU board

Both cameras of the system are each connected to a CPU board, which thanks to its compact size can be optimally integrated into the lid or base of the wine cooler. The single-board computers control the optics and image acquisition, analyze the image data, and pass only the results to the refrigerator’s main electronics. Extensive AI algorithms can also be performed locally on the embedded systems without an Internet connection. The computing power of the main electronics remains free for the actual end application, whether it is the output of additional information on the detected wine bottles on a display, streaming of the data to a POS system, a cloud application or other conceivable applications for smart homes, intelligent store systems or the analysis of user behavior.

3. Rapid advancement to series production—and transferable to other applications

Already for the prototype, DAB-Embedded relied on industrial-grade VC MIPI camera modules and embedded CPU boards with components from the NOTAVIS store, which have proven their worth many times in series products. Optical quality and image processing results tunred out to be an ideal match. For the advancement to series production, the development service provider is developing an individual mainboard for the embedded system in the next step. All development work in the area of software and the available camera drivers can be taken over directly for this, so that the final solution can be finished quickly, easily and cost-effectively.

There are plans to adapt the integration of the camera solution for other markets and applications as well—whether for smart household appliances and home automation, for comprehensive store solutions or for use at the point of sale.

artisiom

“Together with NOTAVIS, we have decided on an embedded system from NVIDIA that fits our requirements perfectly. The platform is proven in use with VC’s MIPI modules and the delivery of all components was fast and from a single source via NOTAVIS. The support during the development additionally convinced us of the cooperation“

Art Staliarou
CEO DAB-Embedded

 

About DAB-Embedded

DAB-Embedded is an innovative Belgian engineering company, specialized in the design and development of electronic products including software. The focus is on electronics for industry, multimedia and consumer products as well as hardware for processing image and video data.

www.dab-embedded.com

Contact




    Phone

    +49 6331 25 99 793

    E-Mail

    thomas.schweitzer@NOTAVIS.com

    Adress

    Marie-Curie-Str. 14-16

    66943 Pirmasens

    Deutschland