When critical parts roll off a production line, someone has to ensure that they’re built to specification. In some industries like healthcare, a poorly-engineered part that stops a medical device could cost clinic operators $3,000 per hour. In the automotive and aerospace sectors, poor engineering can put customers’ lives in danger.
Toronto-based Bluewrist, which won the Small Private category in the 2018 Ingenious Awards hosted by the Information Technology Association of Canada (ITAC), has spent the last 12 years building software that enables robots to fix minor issues on the production line before they become major problems, and it does it using an advanced form of machine vision.
“The goal was to provide intelligent systems that can be useful in manufacturing plants for inspection and guidance applications,” explained Bluewrist’s CEO Najah Ayadi, who founded the company in 2006. Ayadi, who had a masters of science in industrial engineering, got the idea for Bluewrist after working in robotics as an application engineer. “I was dealing with software that was obsolete and inefficient in terms of flexibility and accuracy, so I decided to come up with a solution that can make automation applications more flexible,” he said.
He created robotic guidance software that would allow customers to use different kinds of hardware, such as cameras and robotic systems, without having to rewrite it each time the customer needed a different robotic application.
“The focus has always been on 3D applications,” said Ayadi. “The two most challenging are tying inline inspections to quality requirements from the plant, and guidance applications where the customer wants to automatically pick up parts or use vision to guide robots to do certain tasks.”
Bluewrist’s systems automate the kinds of manual inspections that technicians would normally conduct with a coordinate measuring machine to verify the accuracy of an engineered part. This work requires temperature-controlled rooms, and can take hours for larger parts. The technicians that do it earn $70,000 per year on average.
More efficient inspections
By introducing machine vision techniques, Bluewrist has been able to automate these manual inspections. The company’s FlexiSight system uses measurement sensors that are either stationary or attached to robotic arms that can move into position when needed. They aren’t simple cameras, though; they’re 3D scanners, tailored to meet the rigorous demands of precision manufacturing environments. They produce a 3D ‘point cloud’ that scans all round the part to understand depth, alongside height and width.
Its product compares that point cloud data to CAD files from the manufacturer to see if the real-world object under inspection matches the precise tolerances needed. This includes measurements for features including holes, slots, surfaces, edges, studs, clips, gap and flushness. The manufacturer can select features from a library in the software, configuring it from a menu interface to suit its particular application and part.
Understanding how the point cloud maps to the CAD file involves a new approach to machine learning, said Ayadi. “Traditionally, machine learning is mostly in the 2D domain, where the algorithm tries to work out what it is looking at.” 3D scanning provides far more data, in a different format. “The complexity versus 2D images is exponential.”
When precision counts
Quality inspection can have meaningful results for manufacturers who can save a lot of money by catching defects during the manufacturing process rather than after the fact. In one application, Bluewrist worked with a car manufacturer that inserted up to 20 electrical connectors in a car seat before upholstering it.
The connectors are inserted by human operators, meaning that there is ample room for error. “If the insertion is incomplete or there is a gap, then once the vehicle is on the road, vibrations could cause them to dislodge,” said Ayadi. “That could result in a very expensive recall.”
Bluewrist had CAD files from the client that described exactly what a full connector insertion looked like. It used a robot to scan the connectors on the seat as it moved down the line, comparing it to the file. If the connectors were not properly inserted, it would flag the issue in SPCWorks, the company’s real-time quality control system that analyzes real-time dimensional data for all scanned parts and automatically reports on quality issues.
At Chinese auto manufacturer Geely, Bluewrist’s scanners helped to ensure that 10 automotive parts were placed properly on a tray before being taken to an automated adhesive station. The company used Intel-powered workstations to confirm part location and orientation in under five seconds, ensuring that the robot at the adhesive station would apply adhesive properly and avoid colliding with any of the parts.
A guiding eye
Even when robots are handling assembly, variations in the plant environment mean that they’re not 100 per cent accurate all the time. When you’re dealing with applications that require sub-millimeter precision, that can be a problem. For example, hardware that moves a component down the line to arrive under a robot arm may not always deliver that component at exactly the same position.
In some cases, temperature fluctuations at the plant can also affect component measurements. Bluewrist applies its 3D scanning technology to help here, too, measuring the environment exactly to ensure that robotic systems get things right by using a mixture of 3D scanners and high-precision cameras on its robot arm.
In one application, also with an auto-manufacturer, Bluewrist’s guidance application helped with the insertion of windshields into car frames. Traditionally, at least two technicians would have loaded a windshield in a labour-intensive process using an industrial-strength suction cup.
“More manufacturers now use flexible lines, meaning that they can produce multiple parts and products on the same line. For the software to know what part is being used, they don’t rely anymore on mechanical friction. They use machine vision to bridge the gap and ensure that the part gets located properly,” Ayadi says. “They have been using that for a few years now and relying on 2D vision, but 2D is not very reliable and it’s very sensitive to lighting and environmental changes. Our system provides the 3D information that is required.”
The company scans the windshield and aperture with 650-nanometer lasers from 3D smart sensors, feeding the data to its EzRG robot guidance software, which transforms it and feeds the results back to the robotic controller. The robot then inserts the windshield with an accuracy of 0.2mm.
BlueWrist also uses the same basic technologies for robotic picking solutions. It can use 3D scanning technology to survey a pile of randomly-placed parts, such as a box of washers, and use the point cloud data to locate individual parts in the collection, orient the robotic arm appropriately, and pick it up.
Bluewrist’s technology is already helping companies to serve customers in the US, China, Mexico and Europe. Now, the company is working on even better machine learning algorithms to make its image processing even faster and more accurate.
Intel, whose processors feature strongly in Bluewrist’s computing-intensive image analysis operations, sees the edge-based AI technology underpinning Bluewrist’s solution as a strong component in industrial IoT technology.
“The whole notion of edge computing is becoming more dominant and necessary,” concluded Intel Canada director Elaine Mah. Complex calculations supporting modern industrial processes require vast amounts of data and a lot of computing power. It makes sense to push the industrial applications closer to where they are needed to reduce latency and network traffic. “Vision computing will be a large part of that.”
