Key application areas
Adimec Advanced Image Systems BV / Caeleste CVBA / Delft University of Technology , Delft, the Netherlands / Grass Valley Nederland BV / intoPIX SA / ON Semiconductor Belgium
Project leader(s)Jochem Herrmann
Key project dates01 January 2018 - 31 December 2020
Developments in machine vision and ultra-HDTV broadcast will significantly improve resolution, colour gamut, dynamic range and image quality
Cameras with higher resolution, increased speed, and wider dynamic range and colour gamut are needed to fulfil the needs of next- generation machine-vision and ultra-HDTV broadcast systems. These are key challenges the SENSATION project addresses through and improvements to existing components, and new developments.
Image capture and transmission are generic technologies that are deployed in a multitude of business applications, including manufacturing, health-care, security, automotive, TV broadcast, digital entertainment (such as digital cinema and gaming) and, more recently, agriculture. In order to improve product quality and increase productivity, the resolution and speed of systems have to increase continually, under constraints of power consumption and thermal performance. In addition, vision-based professional applications will require higher spatial and temporal resolutions and improved image quality, especially in broadcast.
Help is at hand. Modern CMOS (complementary metal oxide semiconductor) image sensors have already started replacing the older, slower and power-hungry CCD (charged coupled device) ones. However, more needs to be done. Current machine-vision systems are mostly based on the industrial PC, where image processing takes place. In order to be able to process ever-increasing pixel rates, future machine vision will be based on embedded systems, which are not only smaller and more cost-effective than PCs; but also faster and consume less power thanks to highly optimised architectures. These are compelling benefits.
Developing next-generation machine vision and broadcast systems
The overall goal of the SENSATION project is to develop technologies and improve building blocks needed for the next generations of CMOS image sensors, video processing and transmission. Responding to the market trend of vision-based professional applications in image capture – that is, the move towards higher spatial and temporal resolutions, wider colour gamut, higher dynamic range and improved image quality – new and improved building elements will be developed to drive and address these trends in production technologies, as well as connectivity and digital networks.
This project investigates such innovations as image processing, image compression and transmission that are needed for future machine-vision systems. This is in line with Industry 4.0, the name given to the current trend of automation and data exchange in manufacturing technologies. Specialists in all these areas will work together creatively to develop the necessary elements of the value-chain. And, because standardisation is of paramount importance in the machine-vision market, there will also be collaboration with standards groups in Europe, USA, Japan and China.
On another front, the broadcast market is starting to migrate from HDTV to ultra-HDTV, a standard which supports 4Kx2K and 8Kx4K resolutions; 12 bits per pixel (versus 10 bits in HDTV); and a wider colour gamut and an increased dynamic range. Another aim of this project is to develop new and improved image processing (IP) in image sensors (moving from the 0.18μm/110nm to the 65nm technology node), and to bring UHDTV to a higher specification level.
It is worth noting that, though products associated with the high-end machine-vision and broadcast target markets are quite different, the technical challenges are very similar. It is therefore very beneficial to do R&D for both markets in the same project. This not only helps to share knowledge and designs between project partners, but it also helps develop and apply new standards that can be deployed in both markets.
Key project developments and deliverables are:
- Building blocks for CMOS image sensors: smaller global shutter pixels; increased dynamic range; increased data rates; auto-focus pixels; improved ultra-high-speed architectures; and high-speed serial interfaces;
- New solutions for camera transmission;
- Demonstration of results in cameras for machine vision and broadcast, and image-sensor evaluation set-ups;
- Standards for a high-speed serial interface for image sensors, image compression and camera interfaces. Here the broadcast development team will work with the SMPTE standardisation body.
Creative players needed
In order to meet future market requirements, innovations are required in products of all companies in the machine-vision and broadcast value-chains. A highly qualified and multi-disciplinary team is therefore required to develop the various aspects of the technologies, such as optics, image capture, PROJECT PROFILEvideo processing and transmission, and to optimise these technologies. This is why leading European companies in the imaging industry with proven track-records are part of the SENSATION project, jointly working on the required innovations. The project consortium consists of an R&D institute; fabless (without in-house chip production) design houses; semiconductor manufacturers and system integrators advanced in image sensor technologies, as well as design, video processing, transmission and camera integration.
Demonstrators will be developed, not only to verify functionality and performance, but also to demonstrate to potential customers the outcome of the project. This will also allow project partners to stay in the forefront of their markets and further improve their market position. Unusually, not only is there collaboration within the same value chain, but with outside ones as well. Partners collaborating in the SENSATION project face, for example, the same technological challenges found in different non-competitive markets. This not only drives information sharing (in experience, designs, best practices), but also joint developments and cooperation in standardisation.
Taking a look at the markets in which SENSATION will play a role, the main growth areas for machine-vision products (which focus on high resolution, high frame-rate, and area scan) are North America and EMEA (Europe, Middle East and Africa). However, significant growth is also expected in Asia and China. The market size in 2015 for machine-vision cameras was put at US$650m (excluding smart cameras and smart sensors). The compound annual growth rate (CAGR) for cameras is about 2%, several times that for industrial image-sensors which are also deployed outside the machine-vision-camera market.
In the next five years, the machine-vision market is expected to be driven by the following:
- Industry 4.0 in general, and regional initiatives like ‘made in China 2025’ will stimulate smart manufacturing where robotics and smart vision will play a key role;
- In the high-end market, the combination of ever-more demanding inspection tasks and the continual push for lower overall cost-of-ownership drive the need for small pixel, high resolution sensors that can perform high quality inspection at reasonable frame-rates, thereby reducing the overall inspection time (and cost);
- This trend sets new challenges throughout the vision system, including the smaller sensor-pixels which need to deliver improved optical performance;
- New camera-interface standards enable the move of sharply increased bit-rates from camera to centralised vision-systems. In parallel, there is also a trend towards on- camera ‘smart’ or ‘embedded’ vision, resulting in distributed image processing and a localised inspection decision process.
In 2015, the outside-broadcast market was US$58m and the studio market US$100m. The main growth areas are Asia-Pacific, Latin America and Russia. These markets will in the coming years be mainly driven by the following:
- Content creators are moving towards higher-resolution formats for sports broadcast television and cinema, to create premium-quality content in a market with intense competition from Web content, pay-TV, and video-on-demand. Japan is the first country to roll out UHDTV in 4K and 8K. The 2020 Olympics will be partly broadcast in 8K;
- Even though many TV programmes are being watched on-demand, consumption of live events is growing globally and will continue to increase, resulting in continual investment in cameras for live-event production and broadcasting;
- Slow-motion will continue to increase in popularity. Broadcasters will be using more cameras with slow-motion features in a wider range of applications (programmes). The trend is towards multi-purpose cameras instead of specialty cameras for each application;
- And equipment prices and operational cost will decrease giving content-creators faster return on investment.