Machine vision is a branch of computer vision. The visual control or vision application in the industrial automation field mainly refers to the extraction of relevant information from the collected digital images through the front-end optical camera, and after analysis and processing, it is used to analyze the production line process Or quality control to replace repetitive manual operations. At present, the practical applications of machine vision in the production process of the factory include: completeness quality inspection, identification, classification, part position and direction, size measurement, code reading recognition and traceability, surface/color difference detection, automated assembly line operation, etc. . It can be said that machine vision has installed a pair of "eyes" for the assembly line and automation equipment, which can provide all-weather, high-consistency, and high-reliability test results, adding to the intelligent level of factory production, and thus becoming a step forward. An important technology for intelligent manufacturing.
According to statistics from relevant organizations, in 2019, the global market size of machine vision technology used in the field of industrial automation reached about 5.37 billion U.S. dollars, and is expected to reach 12.29 billion U.S. dollars by 2023, with a compound annual growth rate of 21%. , The market potential is huge.
From the perspective of the overall supply chain, manufacturers participating in the development of machine vision systems can be roughly divided into upstream bottom developers, including core component and software developers; midstream equipment integration and software service providers engaged in secondary development; and downstream vendors. Right various machine vision application manufacturers. Among them, the core components and software suppliers mainly include light sources, lenses, industrial cameras, image capture cards, image processing software and other types of manufacturers, and in the entire cost structure, components and software development costs basically account for about 80% The ratio is the top priority. Light source suppliers include OPT Optoelectronics, Ruishi Optoelectronics, etc.; the main brands of industrial cameras include Basler, Baumer, DALSA, Point Grey, Images Source, etc.; the more well-known image software packages on the market include Halcon (MVtec), VisionPro ( Cognex), free open source libraries Opencv, MIL (Matrox), etc.; motion control card suppliers include Gogo, Leisai, Advantech, etc.
At present, most of the machine vision in the industrial control field belongs to the secondary development level. The form mainly includes system integration and assembly production automation special machines, etc. Developers are required to master the Know-How technology in different manufacturing environments and design targeted products Or integrated solutions, the back-end manufacturing process of electronic products such as visual dispensing control, visual screw machine control, visual positioning + flying shot, visual inspection, etc., are all development hotspots in the current market. Most manufacturers use the "software platform + vision development kit" development model. The development kit encapsulates various commonly used image processing algorithms based on the software platform. Software engineers can directly call the encapsulated algorithms to realize various complex images. Processing and analysis functions greatly reduce the difficulty and workload of secondary development.
Figure 1 The composition of the machine vision industry supply chain
Figure 2 Machine vision application analysis in the field of industrial automation (Source: Zhiyan Consulting)
Three levels of problems in vision applications
However, in terms of practical applications, in the current implementation of machine vision + motion control solutions, various "thresholds" are still high, and the problems mainly come from three aspects: First, for most technical development engineers, the machine Visual technology is a brand new field. It is difficult to learn a new programming language (such as C++), and it takes a lot of time and energy. In addition, visual projects are generally non-standard, and the developer’s experience is very important and ordinary It is no easy task for electrical engineers to quickly master the programming and debugging of machine vision applications.
Secondly, for equipment manufacturers, in order to improve the production efficiency of factories, there is an urgent need to upgrade the production line of the manufacturing industry. The application of machine vision has become one of the key points. However, the hardware investment cost of machine vision has been high, making them Be discouraged; in addition to hardware investment, it is often necessary to hire a high-paying visual engineer to develop and maintain the purchased visual system in the later period, which undoubtedly increases the labor cost of the factory. How to effectively control the investment cost is their main appeal, such as whether it is economical and convenient to add machine vision functions to the existing equipment system.
In addition, for end users, in the process of using machine vision technology, they will find that machine vision and automation control (process control, process control, etc.) are often divided into two different systems, and there may often be The instability of the entire system due to various factors such as communication (such as shutdowns, line outages, etc.). Especially in the current multi-variety, small-batch, and flexible production mode, frequent replacement of production lines increases the modification and debugging time of the machine vision system process, which greatly affects the production capacity of the factory. Therefore, how to make the equipment in production The medium is more stable, and the operation is more convenient. It can meet the needs of rapid adjustment and switching of production lines, which is a subject of machine vision technology development.
Seamless integration of machine vision and motion control
Generally speaking, most hardware architectures for machine vision + motion control applications use a controller, which is connected to the vision system and industrial smart cameras and light sources in various parts of the production line. The visual signals are transmitted to the In the controller. In the internal software framework, taking the common computer vision as an example, the vision program is generally located in the non-real-time user mode of the Windows system for code analysis and algorithm processing; while the PLC process control is mostly located in the real-time kernel mode. Since the task execution mechanism of the non-real-time user mode uses the priority level, when the CPU load is too high, the memory is too large, and the processing flow is too much, the phenomenon of lag may occur, making the machine in the non-real-time user mode The final time it takes for vision to complete the same inspection task is different, which slows down the production cycle and affects production efficiency.
In order to solve this problem, for machine vision applications, the TwinCAT Vision machine vision solution developed by Beckhoff integrates vision technology on its TwinCAT unified universal control technology platform, and the image processing function is added to integrate PLC, motion Among the universal control platform TwinCAT software of control, robotics, high-end measurement technology, Internet of Things and HMI, all the control tasks of the factory are completed through this unified software platform.
Compared with traditional vision applications, Beckhoff TwinCATV ision is also on the same controller. In addition to integrating motion control, I/O control, and PLC control, external industrial cameras are directly connected to the machine vision module in the controller. In the internal software architecture, TwinCAT Vision transfers the collected image signals to the PLC machine vision and process control system in the real-time kernel mode through the GigE camera drive, and performs precise operation in a “time slice” manner, and the image All processing-related control functions can be accurately synchronized in real-time in the runtime system, thereby eliminating delays and executing image processing algorithms in real time. The advantage of this solution is that one controller completes all control tasks, reduces hardware costs, and simplifies the system; the real-time system runs visual tasks to ensure that the response time of each execution is stable enough, speeds up production cycle, and improves factory productivity; task memory Sharing data between the two, multi-task coordination (such as vision + motion control, robotic arm/human, etc.) is more efficient and convenient; on the Beckhoff TwinCAT platform, only the IEC-based programming language (ST/LD, etc.) is required, and the original PLC programming skills and habits have shortened the development cycle and reduced labor costs. At the same time, the integrated visual function blocks of Beckhoff can be used for code development, which brings convenience to the technicians.
The underlying chip technology and algorithm library are the core
Industrial production requires high computing speed and accuracy of machine vision, which requires powerful underlying technology to support, such as the performance of CPU central processing unit chips, the characteristics of parallel processing multi-stage pipelines of FPGA devices, and professional vision algorithm libraries, etc. Etc., in order to improve the data throughput and data processing capabilities of the machine vision system.
In May of this year, Wuxi Xinje and Intel jointly released a variety of blockbuster products such as new Intel chip-based IoT industrial controllers and smart cameras. Based on Intel’s 14nm Apollo Lake&Kaby Lake platform, Xinje’s V200|V300 series IoT industrial controller has powerful computing performance and optimized image processing capabilities, and provides a variety of configuration options. S200 series smart cameras are based on Intel Apollo Lake platform, adopt IA+FPGA architecture, integrate image acquisition, image processing and external communication, and support 1.3 to 12 million pixels. Compared with traditional solutions, S200 series smart cameras have the highest surface when fully loaded. The temperature is lower than 50℃, which provides safety guarantee for intelligent manufacturing. In addition, the new X-SIGHT VISIONSTUDIO software platform has more than 2000 processing operators, including commonly used detection, positioning, calibration, communication, character reading, etc., without writing code, drag and drop components to complete development, and support multi-threading Task processing, and provide a wealth of HMI interactive modules. It is reported that in addition to the products and solutions introduced at this conference, Xinje and Intel will carry out all-round cooperation and exchanges in industrial manufacturing fields such as spray coating, polishing, artificial intelligence recognition, and sorting.
The development of machine vision based on FPGA devices can easily modify its internal logic functions through programming, thereby realizing high-speed hardware calculations and parallel operations. It is a more convenient solution for high-performance embedded vision systems. The consumption is also low, which is an important development direction of the computer vision system in the future.
In terms of vision software, Vision Pro is an excellent computer-based vision software developed by Cognex. It is characterized by simplicity and ease of use. Its Quick Build rapid prototyping environment combines the advancement, flexibility and ease of development of advanced programming. Combined, assist manufacturers to shorten the development cycle time. With VisionPro, users can access powerful pattern matching, spots, calipers, line positions, image filtering, OCR and OCV vision tool libraries, as well as one-dimensional bar code and two-dimensional code reading to perform various functions, such as detection, For identification and measurement, VisionPro software can be fully integrated with a wide range of .NET class libraries and user controls. In addition, through the API to connect VisionPro ViDi, a deep learning software designed for industrial image analysis, it can be further optimized for application scenarios such as complex inspection, component positioning, classification, and optical character recognition.
Integrated machine vision integration solution
Aiming at the "pain points" of vision applications in the existing production process, starting with a software and hardware integration solution that closely combines machine control, B&R's integrated vision system that integrates lenses, industrial cameras, and software is another Method to realize.
This kind of integrated integrated vision system of software and hardware solves the synchronization problem in vision inspection applications. The first is the synchronization of lighting and imaging. For vision, good lighting design constitutes 70% of the success weight of vision applications. Integration The visual solution is controlled by the same controller, so that the lighting LED intensity, strobe and the camera's focus, exposure, and imaging processes are synchronized at a very high speed, thereby ensuring the imaging quality; the second is the integration of the vision system and the automation system. Synchronization, the integrated vision solution enables seamless integration of vision applications with machine technology and control. At the same time, together with B&R's own control, motion control, and I/O modules, it achieves 100μS-level synchronization under the POWERLINK architecture, making the overall system Able to achieve higher accuracy and speed.
On the other hand, this kind of visual integration solution is very convenient for developers of on-site integration applications. He no longer needs to understand the knowledge of various lenses, cameras, controllers, lighting, and communication interfaces, and only needs to use one product to build Various vision applications shorten the project development cycle and greatly reduce the engineering time and cost. Using the ready-made software components provided by the B&R mapp technology framework, machine vision applications can be easily created with less programming, and the images captured by the smart camera can be integrated into the mapp View HMI application with just a few clicks. The camera and lighting parameters and trigger conditions can be changed at any time, making product changes and other runtime adjustments very easy to implement.
In fact, in various complex industrial production scenes, the lighting source is particularly important. It is directly related to the quality of the captured image, such as whether the image outline is clear, whether the imaging features are perfectly reflected, and so on. To this end, B&R’s integrated vision system also provides very flexible light source lighting management functions, including the modular design of the background light and the light strip, which can be rotated by -40° to +90°, and has a built-in flash controller and support A variety of colors, high-power fast response, these can be configured according to the field application, and the exposure time length, angle, color, etc. can also be modified in real time. These functions make machine vision applications more flexible and free.
The combination of machine vision and motion control has brought a new vitality to the upgrade of industrial automation production lines. It is also increasingly used in non-contact inspection and measurement scenarios to improve processing accuracy, find product defects, and perform Automatic analysis and decision making have become an important part of advanced manufacturing. At present, although the application of machine vision technology in the industrial automation industry involves many different disciplines such as optics, electronics, machinery, industrial software, etc., the technical difficulties are still relatively large, and the market entry barriers in certain specific fields are also high. ; But it is obvious that the combination of machine vision and motion control technology has promoted the optimization of factory production line automation level, product quality control, and labor cost structure, thereby enhancing the overall competitiveness of the company. In the future, machine vision systems that integrate AI technologies such as edge computing and deep learning will truly implant "eyes and brains" into machines and prepare for the industry 4.0 era.
According to statistics from relevant organizations, in 2019, the global market size of machine vision technology used in the field of industrial automation reached about 5.37 billion U.S. dollars, and is expected to reach 12.29 billion U.S. dollars by 2023, with a compound annual growth rate of 21%. , The market potential is huge.
From the perspective of the overall supply chain, manufacturers participating in the development of machine vision systems can be roughly divided into upstream bottom developers, including core component and software developers; midstream equipment integration and software service providers engaged in secondary development; and downstream vendors. Right various machine vision application manufacturers. Among them, the core components and software suppliers mainly include light sources, lenses, industrial cameras, image capture cards, image processing software and other types of manufacturers, and in the entire cost structure, components and software development costs basically account for about 80% The ratio is the top priority. Light source suppliers include OPT Optoelectronics, Ruishi Optoelectronics, etc.; the main brands of industrial cameras include Basler, Baumer, DALSA, Point Grey, Images Source, etc.; the more well-known image software packages on the market include Halcon (MVtec), VisionPro ( Cognex), free open source libraries Opencv, MIL (Matrox), etc.; motion control card suppliers include Gogo, Leisai, Advantech, etc.
At present, most of the machine vision in the industrial control field belongs to the secondary development level. The form mainly includes system integration and assembly production automation special machines, etc. Developers are required to master the Know-How technology in different manufacturing environments and design targeted products Or integrated solutions, the back-end manufacturing process of electronic products such as visual dispensing control, visual screw machine control, visual positioning + flying shot, visual inspection, etc., are all development hotspots in the current market. Most manufacturers use the "software platform + vision development kit" development model. The development kit encapsulates various commonly used image processing algorithms based on the software platform. Software engineers can directly call the encapsulated algorithms to realize various complex images. Processing and analysis functions greatly reduce the difficulty and workload of secondary development.
Figure 1 The composition of the machine vision industry supply chain
Figure 2 Machine vision application analysis in the field of industrial automation (Source: Zhiyan Consulting)
Three levels of problems in vision applications
However, in terms of practical applications, in the current implementation of machine vision + motion control solutions, various "thresholds" are still high, and the problems mainly come from three aspects: First, for most technical development engineers, the machine Visual technology is a brand new field. It is difficult to learn a new programming language (such as C++), and it takes a lot of time and energy. In addition, visual projects are generally non-standard, and the developer’s experience is very important and ordinary It is no easy task for electrical engineers to quickly master the programming and debugging of machine vision applications.
Secondly, for equipment manufacturers, in order to improve the production efficiency of factories, there is an urgent need to upgrade the production line of the manufacturing industry. The application of machine vision has become one of the key points. However, the hardware investment cost of machine vision has been high, making them Be discouraged; in addition to hardware investment, it is often necessary to hire a high-paying visual engineer to develop and maintain the purchased visual system in the later period, which undoubtedly increases the labor cost of the factory. How to effectively control the investment cost is their main appeal, such as whether it is economical and convenient to add machine vision functions to the existing equipment system.
In addition, for end users, in the process of using machine vision technology, they will find that machine vision and automation control (process control, process control, etc.) are often divided into two different systems, and there may often be The instability of the entire system due to various factors such as communication (such as shutdowns, line outages, etc.). Especially in the current multi-variety, small-batch, and flexible production mode, frequent replacement of production lines increases the modification and debugging time of the machine vision system process, which greatly affects the production capacity of the factory. Therefore, how to make the equipment in production The medium is more stable, and the operation is more convenient. It can meet the needs of rapid adjustment and switching of production lines, which is a subject of machine vision technology development.
Seamless integration of machine vision and motion control
Generally speaking, most hardware architectures for machine vision + motion control applications use a controller, which is connected to the vision system and industrial smart cameras and light sources in various parts of the production line. The visual signals are transmitted to the In the controller. In the internal software framework, taking the common computer vision as an example, the vision program is generally located in the non-real-time user mode of the Windows system for code analysis and algorithm processing; while the PLC process control is mostly located in the real-time kernel mode. Since the task execution mechanism of the non-real-time user mode uses the priority level, when the CPU load is too high, the memory is too large, and the processing flow is too much, the phenomenon of lag may occur, making the machine in the non-real-time user mode The final time it takes for vision to complete the same inspection task is different, which slows down the production cycle and affects production efficiency.
In order to solve this problem, for machine vision applications, the TwinCAT Vision machine vision solution developed by Beckhoff integrates vision technology on its TwinCAT unified universal control technology platform, and the image processing function is added to integrate PLC, motion Among the universal control platform TwinCAT software of control, robotics, high-end measurement technology, Internet of Things and HMI, all the control tasks of the factory are completed through this unified software platform.
Compared with traditional vision applications, Beckhoff TwinCATV ision is also on the same controller. In addition to integrating motion control, I/O control, and PLC control, external industrial cameras are directly connected to the machine vision module in the controller. In the internal software architecture, TwinCAT Vision transfers the collected image signals to the PLC machine vision and process control system in the real-time kernel mode through the GigE camera drive, and performs precise operation in a “time slice” manner, and the image All processing-related control functions can be accurately synchronized in real-time in the runtime system, thereby eliminating delays and executing image processing algorithms in real time. The advantage of this solution is that one controller completes all control tasks, reduces hardware costs, and simplifies the system; the real-time system runs visual tasks to ensure that the response time of each execution is stable enough, speeds up production cycle, and improves factory productivity; task memory Sharing data between the two, multi-task coordination (such as vision + motion control, robotic arm/human, etc.) is more efficient and convenient; on the Beckhoff TwinCAT platform, only the IEC-based programming language (ST/LD, etc.) is required, and the original PLC programming skills and habits have shortened the development cycle and reduced labor costs. At the same time, the integrated visual function blocks of Beckhoff can be used for code development, which brings convenience to the technicians.
The underlying chip technology and algorithm library are the core
Industrial production requires high computing speed and accuracy of machine vision, which requires powerful underlying technology to support, such as the performance of CPU central processing unit chips, the characteristics of parallel processing multi-stage pipelines of FPGA devices, and professional vision algorithm libraries, etc. Etc., in order to improve the data throughput and data processing capabilities of the machine vision system.
In May of this year, Wuxi Xinje and Intel jointly released a variety of blockbuster products such as new Intel chip-based IoT industrial controllers and smart cameras. Based on Intel’s 14nm Apollo Lake&Kaby Lake platform, Xinje’s V200|V300 series IoT industrial controller has powerful computing performance and optimized image processing capabilities, and provides a variety of configuration options. S200 series smart cameras are based on Intel Apollo Lake platform, adopt IA+FPGA architecture, integrate image acquisition, image processing and external communication, and support 1.3 to 12 million pixels. Compared with traditional solutions, S200 series smart cameras have the highest surface when fully loaded. The temperature is lower than 50℃, which provides safety guarantee for intelligent manufacturing. In addition, the new X-SIGHT VISIONSTUDIO software platform has more than 2000 processing operators, including commonly used detection, positioning, calibration, communication, character reading, etc., without writing code, drag and drop components to complete development, and support multi-threading Task processing, and provide a wealth of HMI interactive modules. It is reported that in addition to the products and solutions introduced at this conference, Xinje and Intel will carry out all-round cooperation and exchanges in industrial manufacturing fields such as spray coating, polishing, artificial intelligence recognition, and sorting.
The development of machine vision based on FPGA devices can easily modify its internal logic functions through programming, thereby realizing high-speed hardware calculations and parallel operations. It is a more convenient solution for high-performance embedded vision systems. The consumption is also low, which is an important development direction of the computer vision system in the future.
In terms of vision software, Vision Pro is an excellent computer-based vision software developed by Cognex. It is characterized by simplicity and ease of use. Its Quick Build rapid prototyping environment combines the advancement, flexibility and ease of development of advanced programming. Combined, assist manufacturers to shorten the development cycle time. With VisionPro, users can access powerful pattern matching, spots, calipers, line positions, image filtering, OCR and OCV vision tool libraries, as well as one-dimensional bar code and two-dimensional code reading to perform various functions, such as detection, For identification and measurement, VisionPro software can be fully integrated with a wide range of .NET class libraries and user controls. In addition, through the API to connect VisionPro ViDi, a deep learning software designed for industrial image analysis, it can be further optimized for application scenarios such as complex inspection, component positioning, classification, and optical character recognition.
Integrated machine vision integration solution
Aiming at the "pain points" of vision applications in the existing production process, starting with a software and hardware integration solution that closely combines machine control, B&R's integrated vision system that integrates lenses, industrial cameras, and software is another Method to realize.
This kind of integrated integrated vision system of software and hardware solves the synchronization problem in vision inspection applications. The first is the synchronization of lighting and imaging. For vision, good lighting design constitutes 70% of the success weight of vision applications. Integration The visual solution is controlled by the same controller, so that the lighting LED intensity, strobe and the camera's focus, exposure, and imaging processes are synchronized at a very high speed, thereby ensuring the imaging quality; the second is the integration of the vision system and the automation system. Synchronization, the integrated vision solution enables seamless integration of vision applications with machine technology and control. At the same time, together with B&R's own control, motion control, and I/O modules, it achieves 100μS-level synchronization under the POWERLINK architecture, making the overall system Able to achieve higher accuracy and speed.
On the other hand, this kind of visual integration solution is very convenient for developers of on-site integration applications. He no longer needs to understand the knowledge of various lenses, cameras, controllers, lighting, and communication interfaces, and only needs to use one product to build Various vision applications shorten the project development cycle and greatly reduce the engineering time and cost. Using the ready-made software components provided by the B&R mapp technology framework, machine vision applications can be easily created with less programming, and the images captured by the smart camera can be integrated into the mapp View HMI application with just a few clicks. The camera and lighting parameters and trigger conditions can be changed at any time, making product changes and other runtime adjustments very easy to implement.
In fact, in various complex industrial production scenes, the lighting source is particularly important. It is directly related to the quality of the captured image, such as whether the image outline is clear, whether the imaging features are perfectly reflected, and so on. To this end, B&R’s integrated vision system also provides very flexible light source lighting management functions, including the modular design of the background light and the light strip, which can be rotated by -40° to +90°, and has a built-in flash controller and support A variety of colors, high-power fast response, these can be configured according to the field application, and the exposure time length, angle, color, etc. can also be modified in real time. These functions make machine vision applications more flexible and free.
The combination of machine vision and motion control has brought a new vitality to the upgrade of industrial automation production lines. It is also increasingly used in non-contact inspection and measurement scenarios to improve processing accuracy, find product defects, and perform Automatic analysis and decision making have become an important part of advanced manufacturing. At present, although the application of machine vision technology in the industrial automation industry involves many different disciplines such as optics, electronics, machinery, industrial software, etc., the technical difficulties are still relatively large, and the market entry barriers in certain specific fields are also high. ; But it is obvious that the combination of machine vision and motion control technology has promoted the optimization of factory production line automation level, product quality control, and labor cost structure, thereby enhancing the overall competitiveness of the company. In the future, machine vision systems that integrate AI technologies such as edge computing and deep learning will truly implant "eyes and brains" into machines and prepare for the industry 4.0 era.
