AI enables new robot to manipulate soft, bendable objects

Say hello to the robot called Bifrost. With the help of AI technology, it uses its tactile capabilities to manipulate soft, bendable objects on command.

“Despite the impression many of us have in the media, robots still have a long way to go before they can competently manipulate so-called conformal objects,” explains Norwegian researcher Ekrem Misimi.

Misimi believes much more research is needed before this can be achieved.

“This is why we are proud to have developed a method that allows robots to learn to manipulate objects of this type. This is unprecedented research offering major innovation potential,” says -he.

“The technology currently under development could have many applications in a wide range of industries, but primarily in the food sector, where there is no shortage of compliant objects.”

An AI-based approach

The new technology emerged as part of a project, also called BIFROST, which in Norse mythology refers to a “bridge”. However, the inspiration for the method began in the kitchen, and specifically with the steps of preparing a humble cod fillet for dinner.

“We all know this process without really thinking about it,” Misimi says. “But if you think about it, you realize that many different operations are performed on the net. In the past, robots were not able to perform such tasks, because until recently , they were only trained to grasp objects that were rigid and non-compliant. Additionally, it is difficult to obtain agreement. robot to perform new tasks for which he has not been trained,” he says.

In the field of AI, robots struggle with something called generalization ability, an area in which humans are very expert. To achieve this, researchers have developed a new approach based on artificial intelligence. They train the robot by making it perform tasks similar to those it would perform in the real world. Naturally, these tasks don’t involve cod fillets to begin with.

Time-consuming tasks are the most difficult

On the table is an elongated cloth bag filled with rice, here simulating a cod fillet. It has the shape of an “I”, but in response to a command from the researchers, the robot manipulates it into a “C”.

“We thus succeeded in getting the robot to manipulate a three-dimensional object conforming to the command,” explains Misimi. “In itself, this may not seem very impressive. However, the robot has never been trained with this particular shape,” he says.

Misimi adds that it is easy for humans to perform tasks for long periods of time until they finish what they intend to do, but it is much more difficult for a robot.

“A combination of learning and perception is essential for a robot to perform such manipulation tasks.

“Plus one stain“, the longer it takes a robot to learn how to do it,” explains Misimi. “And the longer it takes, the more demanding it becomes on the robot.”

New opportunities

The research team recently published an article on the method within the framework of 2024 IEEE International Conference on Robotics and Automation (ICRA) procedure. The team explains that AI-based robotic manipulation offers new opportunities that were previously beyond our imagination.

“The great thing about this method is that the robot is trained exclusively by simulation,” explains Misimi.

The knowledge is then transferred to the real world, without any additional training. It’s completely new. The researchers’ main goal has been to train robots in skills inspired by what humans can do.

“In a way, we are making them better positioned to perform tasks that require human dexterity, tasks that today can only be accomplished by humans, both in terms of learning and perception,” explains Misimi.

The term perception is used here to describe an ability to be aware of what is happening in our environment, so that a task can be planned in advance and then executed. A combination of learning and perception is essential for a robot to perform dexterous manipulation tasks.

Successive manipulations are delicate

When a robot is tasked with manipulating a given object, it must combine its visual abilities and perception.

“In such situations, it may be necessary to work for a while to complete the task,” says Misimi. “To manipulate the object into a new shape, the robot has to perform many different actions and, until now, it was difficult to use robots for such tasks,” he explains.

He goes on to explain that a robot typically needs 20 to 60 seconds to manipulate an object into a new shape, depending on the complexity of the process.

“It’s a challenge to get a robot to focus on a task for an extended period of time, but we succeeded,” Misimi says.

Ideal for the food industry

The technology currently under development could have many applications across a wide range of industries.

“But first and foremost in the agri-food sectors, and particularly seafood, where there is no shortage of objects suitable for handling,” explains Misimi. “This is important because the use of robots can help enable Norway to maintain its food production industry and, in doing so, can also promote more sustainable and higher quality locally sourced food,” he says , then adding:

“At an even higher level, the knowledge gained from this project could help us address some of the fundamental challenges we face in modern robotics. This in turn will lead to the development of new technologies that will benefit both the industrial sector and society as a whole.

The research team also managed to refine the Bifrost robot so that it is now capable of performing pushing and grasping actions. This allows for even smoother handling of conforming objects and structures.