So how are we going to go about letting Jerry get a soda autonomously. Well again he has the capability already he just needs taught. There are several major motions that Jerry has to meet so let's break those down.
First, any robot that is going to get something from the fridge has to open its door. (We've already got the identification of the fridge, that is easy) Normally, a robot would be designed with a 7 DOF arm that is able to open a door without the robot actually moving. Since Jerry has such a limited arm that is not an option. In the video you can see that Jerry has to use his whole body to swing the door open. This motion will not be that difficult to achieve autonomously. Jerry is able to track the edge of the door very easily. He also has a general knowledge of the location of his arm with respect to that edge. This is where our mechanically adaptive design, or the slop, in the arm is useful. What we will likely do is have Jerry track the edge of the door and its motion and then have him move his arm along the projected path of that motion . But as the door moves to a new location he can update that projection. This is not a precise method. But it will be effective and his arm will stretch and bend a bit to allow for it to occur. So, so put it crudely, instead of grasping the handle precising he basically lassos it with the gripper and gives a jerk then continues jerking.
So we have the door open. Based on what he is retrieving he will have to do an identification. We plan to rely on a basic database of objects and the owners actually training Jerry when he arrives. He will know what a tupperwear bowl is and certainly what a can is. That is all proven and we have the code running for it already. So he identifies the can and again uses his entire body to move his gripper to the location. Again we don't rely on precision. Jerry is going to push right on past items that he is not after. However we don't want him to jerk out all the items in front when he pulls out a can from the back. This will be difficult. But in order to simplify it a bit we will likely have him blaze a trail on the way in to the item he wants. Then he can come back out. Honestly this is an equal action to a human. When we want a soda we are going to get to it. And we will mess up the organization of the frdige to get it.
As far as bringing the item to the person Jerry can just retrace his steps from where he began. Really the most difficult component of this task it to have Jerry open a door. Our solution is sloppiness. We are building a robot that take some abuse and adjust mechanically in order to eliminate processing. Unlike in an industrial setting where a chip has to be placed at great accuracy the human world is designed for uncoordinated humans, so our robots can be uncoordinated also.