AI Picking: handle-in-the-box detects and loosens snagged components
At automatica, Marius Moosmann was introduced to me by Dr. Werner Kraus and Dr. Karin Röhricht. All three are employees of the Fraunhofer Institute for Manufacturing Engineering and Automation IPA. Marius Moosmann is the subject leader "Machine Learning" there. He presented an application that I had not seen before, which is why I asked him to contribute. He kindly and thankfully complied with this request.
Reinforcement Learning
In the future, artificial intelligence (AI) should be able to do what is still not going smoothly when it comes to reaching into the crate: Too often, the box is not completely empty. With the help of reinforcement learning, i.e. learning by trial and error, we at Fraunhofer IPA are working on an optimized and more autonomous application. My personal success story is the autonomous detection and release of jammed components. Even my team and I were surprised that the robot even learned to throw gripped components as far away as possible during training in the simulation.
Reaching into a crate is considered one of the supreme disciplines of robotics: a robot picks up components that are present in an unordered manner from a crate and deposits them in a defined manner for the next production step. This bulk material is a common occurrence in production, and getting to grips with it is not exactly a pleasant task for humans: on the contrary, it is a monotonous and strenuous task that is also costly for the company. A typical "4D task": dull, dirty, dangerous, dear. This burden on personnel and a payback period of typically less than two years in three-shift operation motivates both employees and management to use a robot-based grip-in-the-box.
Handle-in-the-box does not yet exploit potential
And yet the application has not arrived comprehensively in practice. What's more, it even lags far behind expectations. Every year, more than 200,000 new robots are installed worldwide for handling. Of these, only a fraction in the per mille range performs the reach-in-the-box. The majority of robots, on the other hand, grip blindly or, at most, use 2D image processing for semichaotic delivery, such as depalletizing.
At the same time, there are countless separation steps in production that would be suitable per se for the handle-in-the-box, not to mention potential applications in logistics. The user problem of the "4D tasks" mentioned at the beginning, which has existed for decades, is therefore still very large today and has not been solved, despite good economic efficiency.
Two challenges limit success
One reason for this: Handle-in-the-box cells are the first link in a chained production or assembly line. The clocking out of such a linked line is based on the fact that each station provides a guaranteed output. The "typical" reach-in-the-box brings uncertainties here. First, it is not guaranteed that a robot can remove all parts from the crate because it does not recognize them (sufficiently well). The last remaining parts must then be separated by hand. Secondly, as the crate is emptied, the cycle time also increases significantly. These fluctuations in the cycle time can be compensated either by worst-case design or buffers. The entire line therefore adapts to a possibly high cycle time or the grip-in-the-box starts earlier and works out a "head start" to prevent downtimes.
The empty box as a vision
These uncertainties currently prevent the widespread use of the grip-in-the-box in practice. At Fraunhofer IPA, our team is working on solutions that address these uncertainties. Our vision is a completely empty crate, without any human intervention. This "Vision Zero" is currently still a pipe dream. There may be various reasons for this: It may be due to image processing, unsuitable sensors or grippers, combined with objects that are difficult to grasp or other customer-specific challenges. This vision is also accompanied by efforts to automate setup efforts as much as possible. We refer to this as "automation of automation", which we would like to achieve in the long term. The best thing would be for a company to give the robot system a box full of components, allow it to parameterize itself automatically and thus be ready to grip the components after one or two days - without any manual setup at all and with a significantly increased robustness compared to previous applications.
But we are still a long way from that. In total, we have identified eight typical end-user problems that limit the use of the handle-in-the-box. We are developing solutions to these problems, in some cases using new technologies such as machine learning. This is because the market already offers many grip-in-the-box solutions for the model-based grasping discussed here. However, in my experience, the problems mentioned need to be addressed even better.
Therefore, I dedicate a lot of my research and project work to one of these eight problems, namely the detection and solution of jammed components. So far, there is no optimal solution for this. Some robot systems still try to loosen the entanglement by shaking movements, but this is not always successful and can damage the material. Therefore, I pursue the approach that the robot system already recognizes a snagging during the object position estimation. With this knowledge, it can implement the gripping and path planning in such a way that the snagging is elegantly solved by the movement of the robot system.
(English language video is below)
Reinforcement learning perfects the application in simulations
AI technologies have brought a real innovative leap here and made my solution possible in the first place. Briefly, AI is a very broad field of very different methods and processes for solving problems that previously required human intelligence. Machine learning (ML) is one of these methods and currently the most widely used. ML, in turn, is divided into three different subfields: Supervised learning, unsupervised learning, and reinforcement learning. I use the latter. It can best be compared to how a child learns, namely by "trial and error". Transferred to the robot, this means that it receives plus points for a correctly recognized and solved entanglement, and minus points in the negative case. It strives to achieve the maximum number of points and adapts its behavior accordingly or "learns".
However, there is one important difference: A child understands after only a few attempts that it has done something right or wrong. Until an artificial system has this knowledge, it needs hundreds, even thousands of attempts. This means that a robot system has to try out thousands of grips and learn from these successes or failures. Of course, this cannot be done on a real robot. The system would not be productive for far too long and would probably be worn out before it could even get started. Therefore, we train the robot system in a physical simulation. Here, it is relatively unimportant how many trials are needed until the system is ready for real use. With the help of the so-called "sim-to-real" transfer, what has been learned is then transferred to the real robot system. And the grip-in-the-box can get started.
Unhooking
In concrete terms, this means that the robot system takes a 3D image of the box. This enables it to locate around three to eight components based on depth information. A binary classifier network is used to check whether there are any entanglements. The gripping plan is then implemented heuristically, i.e. the robot system arrives at a suitable solution as quickly as possible within a few milliseconds. Here, the predictions of the snag detection are taken into account, but the free components are prioritized first. If there are no more free components, the unhooking process is used. To perform this, a maximum of three additional path points are added to the handle without hooking. The basis for this is a virtual hemisphere with a total of 17 possible path points. In test scenarios with connecting rods, we were able to achieve a reduction in the effective cycle time of ten percent with our development in the case of hooking.
My presented solution actually already works robustly and reliably and is the only one currently available in applied research. However, it also reaches its limits where there is no physical solution: for example, when a de-hooking movement would result in a collision with the edge of the box.
At Fraunhofer IPA, we have been offering our bin-picking software bp3 under license for many years. It is already being used successfully by over ten companies in three-shift operation. On the research side, we are continuously expanding it with new capabilities. One upcoming one will be improved recognition of very flat or reflective components. And my development of the dehooking detection and solution also expands our offering as of now.
Robot learned to throw long distance
But now I have to resolve what the robot from the introduction is all about, who suddenly surprised us with his talent for long throwing. In fact, what we call "rewardshaping" plays an important role in our training in simulations. So what exactly does the robot get its rewards for? Initially, we had programmed the application so that the robot would receive its reward if it separated the objects by the maximum distance - our definition of a successful unhooking. But the robot thought: If I hurl one of the detached objects away from each other, they will be even further apart and I will have done an optimal job. He was right. So we adjusted our reward strategy and now the application also works with correct discarding.
I would be very pleased to be able to demonstrate our current technical status to you personally and to give you an understanding of this optimized bin picking.
More information about the handle-in-the-box:
German: www.ipa.fraunhofer.de/binpicking
Author Information:
Marius Moosmann is subject leader "Machine Learning" at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA; marius.moosmann@ipa.fraunhofer.de;+49 711 9703813
Are we networking? LinkedIn
-> To the Cobot group on LinkedIn (Link)
In my own account/advertisement
The author of this blog is significantly involved in the AI/robotics project Opdra. He consults around robotics. More about him can be found here.
