How to produce flexibly using mobile robots, machine vision

Applying mobile robots, machine vision and artificial intelligence can help build a flexible production system.

Robotic and machine vision insights

• Mobile robots and machine vision can be used to help flexible manufacturing and intelligent manufacturing.
• Mobile robot use can be flexible, modular and scalable.
• Expand the radius of flexibility with AI, smart technologies and mobile robots, using model-based control and iterative learning.

Mobile robotics and machine vision are helping with the rapid development of technology and the continuous pursuit of efficiency, quality and sustainability in industry, as intelligent manufacturing has become the main direction for global manufacturing transformation and upgrades. In this digital and intelligent era, intelligent factories, as the physical carrier and practical platform of intelligent manufacturing, are redefining the development direction of production methods and manufacturing models. At the 2024 Intelligent Manufacturing Conference, Hikrobot built a highly automated, flexible and intelligent industrial application scenario through its mobile robot and machine vision technology.

“The core elements of a smart factory not only require a system that can optimize the production process on its own and improve efficiency, but also needs to have excellent adaptive capabilities to respond quickly to changes in the environment. More critically, it needs to learn about new environments in real time and evolve continuously,” said Jia Yonghua, CEO of Hikrobot, in his keynote speech at the conference. Yongua said a smart factory is more than just achieving digital and intelligent management of the production process, reducing production costs and improving product quality and production efficiency.

Mobile robot, machine vision: examples of use

Hikrobot built a highly automated, flexible and smart factory scenario through its mobile robot and machine vision technology. More importantly, intelligent factories can flexibly adjust production plans and production processes. This includes diversification of product categories, fluctuations in the number of orders, rapid deployments, rapid adjustment, quickly reconfiguring or expanding production lines to adapt to new product production and technological upgrades, which can greatly accelerate products’ time to market. Flexible manufacturing enables and gives greater value to the engine of intelligent manufacturing.

For example, in electric vehicle factories, the new car market is rapidly expanding, from the plant construction to vehicle production. In the photovoltaic industry, the speed from commissioning to volume startup is very short, which relies on the rapid deployment of highly flexible intelligent equipment and intelligent production lines, enabling factories to achieve such goals quickly.

Flexible manufacturing ignites the intelligent manufacturing engine

Intelligent factories are practical carriers of intelligence, which are closely coordinated and integrated at five levels: infrastructure, intelligent equipment, intelligent production lines, intelligent workshops and factory control. They achieve a highly collaborative, flexible and self-optimized manufacturing system.

“Intelligent equipment, intelligent production lines as well as intelligent workshops are the productivity of smart factories to achieve a high degree of automation and intelligence, while the foundation of flexible production lines lies in the intelligent equipment layer,” Yonghua said. The intelligent equipment layer includes automated machines and a variety of intelligent equipment, mainly machine vision and other perceptual equipment, as well as mobile robots and other conveyor equipment.

As the production process of the first-line staff, the flexibility of intelligent equipment determines the flexibility of the entire manufacturing process. In the intelligent production line layer, production process intelligence is achieved through the integration of various flexible intelligent equipment and execution systems. In the intelligent workshop layer, multiple flexible intelligent production lines are integrated with other systems to form intelligent production units, further optimizing efficiency and enhancing resource utilization.

Mobile robot use can be flexible, modular, scalable

At present, Hikrobot has subdivided the granularity of flexibility into the smallest units, so that the tentacles of flexibility can be injected into each part, component and algorithm. For example, for instance, visual sensors, actuators and various software components are split into the smallest functional modules that can work independently, and then through iBASE and heterogeneous platforms, many kinds of products and systems can be integrated to quickly build solutions to meet the needs of specific scenarios. Hikrobot said it delivers flexibility to every node in the production value chain, forming the intelligence of individual devices as well as the intelligence of composite devices, and gradually covering one production line, multiple production lines, and then workshops as well as the entire smart factory.

“Through the integration of layers of flexibility, the seemingly disordered and complex problems are realized through orderly modularization, achieving an orderly transformation from single, independent solutions and then to comprehensive solutions for the industry,” Yonghua added.

Hikvon Robotics provides many mobile robot types, such as fork picker robots, bin robots, latent robots and shift/heavy load robots and divides these into various functional sub-categories, such as stacking, handling, traction, lifting and other, with more than 1,800 product models. In machine vision, it has industrial cameras, intelligent cameras, 3D cameras, intelligent code readers, vision controllers, lenses and light sources. Machine vision subcategories create more than 1,200 product models.

“This design idea of the smallest modular unit and the concept of flexible design allow us to provide precise customized services, thus creating greater value for our customers,” Yonghua, adding that Hikrobot has cumulatively served more than 17,000 customers and more than 200 industry segments.

Expanding the radius of flexibility with AI, smart technologies

In recent years, the rapid development of AI technology has brought about unprecedented changes in the intelligence of the industrial sector. Especially in the field of machine vision and autonomous mobile robots, the role of AI will be more prominent. Hikrobot is addressing industrial development trends and market demand and is using AI and other intelligent technologies to improve products and further expand the flexible production radius of “machine vision + mobile robots.”

Yonghua said, “In these years, we focus on the characteristics of machine vision industry applications, especially on the research of AI small sample learning, unsupervised learning and industry large models and other technical difficulties, launching more than 10 AI algorithm tools, and actively promoting the penetration and application of ‘AI + machine vision’ in more industries. At the same time, we have combined the advantages of traditional classic algorithms and AI algorithms, effectively integrating the two according to application needs, and launched characteristic algorithms such as AI template matching, AI code reading and edge learning.”

Yonghua said Hikrobot constructed a panoramic 3D annotation map based on the features of the AI large model andn the future, whether an unmanned forklift or a composite robot, it will be positioned through improved 3D maps, positioning accuracy, point source identification and video logo detection.

With the support of AI machine vision and 3D technology, disorderly grasping can be achieved. This makes it more flexible, allowing for use of various materials and applications, including stacking and unpacking. Previously, it was only possible to stack the same variety, but now it is possible to achieve mixed stacking.

Mobile robot uses model-based control, iterative learning

For the application of AI on mobile robots, Yonghua said that at the control level, model-based optimal control and pre-planning will help eliminate deviation, improving future robot’s adaptive abilities. These capabilities will help robots gradually adapt to environments through iterative learning to improve the autonomy and robustness of control. At the level of single-unit decision-making, the AI-based path-planning scheme enables the network to have expert-level path planning capability by learning from a large amount of planning path data, while using parallel computing with network reasoning to achieve millisecond-level fast path planning at the board end. At the group decision-making level, using AI technology, a large amount of on-site scheduling data is used to predict the scheduling tasks of different scenarios based on a multi-task learning model, obtaining more accurate prediction results and improving the overall system scheduling efficiency.

Stone Shi is executive editor-in-chief, Control Engineering China; Edited by Mark T. Hoske, editor-in-chief, Control Engineering, WTWH Media, [email protected].