OmniVision is giving digital imaging a front row seat
Digital imaging and machine vision, from a consumer perspective, is mostly about the visible: smartphone cameras, surveillance systems or pocket-sized DSLRs. But industrial applications of digital imaging deliver data from across many spectral bands, enabling Industry 4.0 and many new levels of manufacturing automation in addition to consumer products.
Silicon Semiconductor technical editor Mark Andrews recently spoke with OmniVision’s director of automotive marketing, Andy Hanvey, about the expanding market for machine vision and its future across multiple applications.
Digital imaging and machine vision, from a consumer perspective, is mostly about the visible: smartphone cameras, surveillance systems or pocket-sized DSLRs. But industrial applications of digital imaging deliver data from across many spectral bands, enabling Industry 4.0 and many new levels of manufacturing automation in addition to consumer products. Silicon Semiconductor technical editor Mark Andrews recently spoke with OmniVision’s director of automotive marketing, Andy Hanvey, about the expanding market for machine vision and its future across multiple applications.
Sensor technologies continue to expand their reach across commercial and consumer markets thanks to a combination of increasing quality, advancing technology and the introduction of AI or neuromorphic computing. Technology continues to transform how we collect, utilize and analyze sensory data. Chip quality has skyrocketed while prices have decreased across most applications, meaning that whether someone is driving with the aid of a heads-up display, avoiding injury thanks to the latest ADAS system or any other myriad applications, digital imaging and machine vision is found in so many applications it has become a growing market force.
OmniVision Technologies is a leading developer of advanced digital imaging solutions. Like the market itself that has grown from relatively simple sensors to advanced, miniaturized imaging devices, so has the OmniVision product portfolio, which embraces needs across mobile phones; security and surveillance; automotive; tablets, notebooks, webcams and entertainment devices, as well as medical, AR/VR, drone and robotics imaging systems.
As the sophistication of digital imaging has grown, so too have its applications. One market that is witnessing a transformation thanks to digital imaging is the automotive sector; each year, manufacturers add imaging sensors and digital cameras along with radar and other sensory systems to increase safety, add passenger benefits, and continually increase a vehicle’s ability to support a transition to greater levels of autonomy. Advanced imaging is also getting a boost from companies that share complementary technologies and partner to offer new services that deliver benchmark capabilities.
In July, OmniVision and Smart Eye AB, a global leader in interior sensing AI, teamed up to provide the automotive industry with what the companies believe is a ground-breaking, end-to-end interior sensing solution. By combining two leading technologies, the new solution enables complete in-cabin monitoring that can locate and identify each passenger along with detecting movement that could suggest inattentive driving or incapacity. The technology can also alert a driver if a passenger (such as an infant or child) is left unattended in the vehicle. The companies stated in their announcement that the new solution supports next generation, higher level vehicle autonomy, nighttime sensing, even in-vehicle video conferencing.
The two companies described their new product solution is the first integrated video processing chain combining innovative features based on the OmniVision OV2312 RGB-IR sensor with Smart Eye’s advanced interior sensing artificial intelligence.
“Interior sensing AI is crucial for the automotive industry. Not only is this technology improving automotive safety – saving human lives around the world – it is also enabling automakers to provide differentiated mobility experiences that enhance wellness, comfort and entertainment,” says Martin Krantz, Founder and CEO of Smart Eye. “By partnering with OmniVision, we are delivering on this vision: providing an end-to-end, highly advanced interior sensing system that meets the demands of automotive OEMs, at a price point that makes it viable for the mass market.”
“Empowered by our OmniPixel 3-GS pixel technology, the OV2312 is a 2.1MP, RGB-IR, global shutter image sensor that was designed specifically for interior applications, and it strikes a balance with MTF, NIR QE, and power consumption. We are proud to partner with Smart Eye to enable this accurate full interior sensing solution,” said Brian Pluckebaum, OmniVision’s automotive product marketing manager.
The ability to track driver and passenger movements in-vehicle can deliver immediate safety benefits, and it is believed that the ability to track in-vehicle motion, such as a driver’s movement to activate a vehicle system, will be an important key to achieving higher levels of vehicle autonomy. This latest achievement comes after years of noteworthy digital imaging advances. OmviVision’s automotive products marketing director, Andy Hanvey, shared company milestones that have led to some of today’s state of the art accomplishments.
“One of the most significant historically was development of the OV60A, the world’s first 0.6 μm pixel, 4K image sensor for mobile applications. The OV60A is the world’s first 0.61 μm pixel high resolution CMOS image sensor that revolutionizes the capabilities of next-generation mobile phone cameras,” he said. “The OV60A offers 60 megapixel resolution with the smallest pixel size in its class at .61 μm. The OV60A can fit into a 1/2.8 inch optical format, supporting either 3:4 or 16:9 aspect ratio configurations. To achieve this increase in resolution, along with a smaller pixel size and optical format, the OV60A is built on OmniVision’s PureCel Plus-S stacked die technology. It is 24 percent smaller in area than the previous generation 0.7 μm pixel solution, yet it can achieve higher quantum efficiency (QE) with better crosstalk and angular response than the 0.7 μm generation.”
Nyxel technology provides superior near infrared (NIR) quantum efficiency.
Hanvey said OmniVision also developed the OH0TA; smaller than the world’s smallest medical imager and CameraCubeChip, it is 0.55 mm x 0.55 mm with a 400 x 400 resolution. And it is the new generation of the company’s Guinness World Record holder ‘world’s smallest image sensor’ (OV6948), and wafer level module (OVM6948 with OV6948 embedded). The significance here is its record-setting size allows designers to add ultra-compact visualization to single-use and reusable endoscopes, as well as catheters and guidewires, with a small outer diameter of 1-2mm. The OH0TA’s increased resolution allows higher quality color images to be captured from within the body’s smallest organs, enabling medical devices to reach deeper into the body. Additionally, the sensor’s lower power consumption reduces ‘chip on tip’ camera heat for greater patient comfort and longer procedure durations, while also reducing noise for crisper images.
Another highlight according to OmniVision is the company’s OAX8000, the world’s 1st dedicated driver monitoring system (DMS) ASIC with an integrated AI neural processing unit, image signal processor and DDR3 memory. Hanvey said that most DMS processors on the market today are not dedicated to this application, requiring added circuitry to perform other system functions that consumes more power, occupies more board space and doesn’t allow room for on-chip SDRAM. By focusing the design of the OAX8000 ASIC on entry-level DMS, OmniVision was able to create the automotive industry’s most optimized solution, he explained.
To give an idea of how pervasive digital imaging has become across so many applications, Hanvey noted that another of OmniVision’s noteworthy accomplishments is recently reaching the milestone of 12 billion devices shipped since the company was founded in 1995. They believe the quantity of their products, and sizeable market pull for machine vision, demonstrates a trending growth pace and a bright future for companies that meet the exacting standards.
A final milestone Hanvey mentioned was the company’s launch of its Nyxel near infrared (NIR) imaging technology that OmniVision believes offers the world’s best NIR imaging quantum efficiency.
This is significant because competing mass-produced CMOS image sensors are still failing to achieve comparable NIR performance. With a 2.8 μm pixel pitch, Nyxel provides 60 percent QE in barely visible 850nm NIR light spectrum and 40 percent at the invisible 940 nm NIR wavelength, which are 5 and 3 times (respectively) more than the standard pixel structure.
Since advances in machine vision have accelerated to support manufacturer requirements and to enable future system level advances, it’s natural to wonder: how are changes in machine vision technology going to enable future automotive systems?
“This is a good question, many things could be impacted including enhanced nighttime safety, improved heads-up displays, ADAS-related improvements, imaging of vehicle surroundings and many others. The use of machine vision will be found in automotive use cases more and more, not limited to just ADAS. For example, a camera which would be used today for SVS, in the future could be used for SVS and machine vision. In addition, we have multiple cameras inside the car − these would be used today for DMS. This is an application that looks at the drivers face/eyes, and based on an algorithm, can interpret a level of distraction. In the early days of in-vehicle monitoring, this was known as ‘drowsy driver’ monitoring, but distracted driving can include many activities including eating, looking at a mobile phone, texting, or talking to passengers,” he explained.
The OmniVision Nyxel series of digital imaging technology adds near
infrared (NIR) capability that improves resolution and contrast compared
to conventional CMOS chips.
Looking ahead into the future of automotive design and the role that digital imaging will play across multiple applications, Hanvey noted that as systems multiply, power consumption as well as electromagnetic interference (EMI) becomes more of a challenge, so device designers need to stay ahead of requirements while anticipating future challenges.
“Within automotive applications, there is a trend that we see where the camera is required for more than one function; for example, human viewing and machine vision. At the image sensor level this could mean adding support for specific CFA (color filter array) patterns or having advanced features such as dual output. On the processing side, having an ISP that can process multiple CFAs also helps achieve dual-use functionality, which can be found in the OAX4000,” he said.
“OmniVision strives hard to future-proof designs and we can discuss a number of options. The first is providing a platform of sensors which are P2P (pin to pin) compatible. This means a design can have an upgrade path for a different feature set. Providing a wide range of functionality is exemplified by the OAX4000, so customers today can design their system with RGGB based cameras, but in 2-3 years, they may perform an upgrade to RCCB. This would be a change at the camera level (along with some software updates for the OAX4000 in the ECU), which is more efficient that starting from scratch.”
“As vehicle functionalities increase, OEMs are also concerned with EMI, system power consumption, functional safety and cybersecurity. OmniVision’s designs are based on system level thinking, evolving around problem solving and helping designers achieve optimal results. For instance, we have been continuously working on meeting the EMI requirement changes customers specify. In another approach, we are optimizing the process technology and pixel technology, which enables us to refine the image sensor QE, which in turn helps OEM designers reduce system power consumption.
“Currently, OmniVision’s automotive products are at least ASIL B qualified, with some of the latest products meeting ASIL C compliance. Observing the high demand trend for cybersecurity, OmniVision is addressing this in our product design; we are continuously looking for more ways to enhance cybersecurity in our products,” Hanvey said.
Commercial and consumer expectations for digital imaging/machine vision are constantly evolving, and Hanvey sees OmniVision’s products also evolving along two primary, parallel paths.
“For automobile and vehicle manufacturers, we expect the need for more high performance products to achieve OEM KPI, along with small size, broader light spectrum, unprecedented flexibility and low power consumption. This allows the OEMs to have faster time to market, smaller system design, more use case possibilities, and the ability to hide the camera(s) out of sight. For consumer applications, we see the need to provide the functionally for today and tomorrow with future-proof technologies that deliver smaller pixel size, more visual possibility for challenging lighting conditions that include the near infrared spectrum, higher resolution, better image quality, further object detection range and less power consumption. As a result, OmniVision enables smaller, lighter devices, broader imaging applications, higher image quality and better end-user safety.”
Andy Hanvey
Andy Hanvey joined OmniVision Technologies, Inc. in October 2016 and is currently Director of Automotive Marketing. Prior to OmniVision, he worked at Andor Technology, Aptina Imaging and most recently with Imagination Technologies. He has more than 25 years’ experience in the semiconductor industry in engineering, applications and marketing. Hanvey has an MSc degree in optoelectronics from Queens University, Belfast.