Learning-based visual ball and plate control (Machine learning)
Haitao Yu
New way to control the world
We introduce and design a latent dynamic learning framework that without using touch sensor, only using camera as sensor to control to balance a rolling ball or control the ball to follow specific trajectories in 6 degree of freedom plate, mounted on a robot manipulator. The framework consists of three parts, vison, predict and controller. Respectively composed of Variational Autoencoder (VAE), MDN-RNN and Deep Deterministic Strategy Gradient (DDPG). The framework is trained on a dataset of simulation environment of ball and plate which is built on the CoppeliaSim. We show that the dynamic model learned based on the framework can balance the ball in the middle of the plate. Through subsequent migration training, the ball can be controlled to reach the designated location in the plate.
Page Views:
We introduce and design a latent dynamic learning framework that without using touch sensor, only using camera as sensor to control to balance a rolling ball or control the ball to follow specific trajectories in 6 degree of freedom plate, mounted on a robot manipulator. The framework consists of three parts, vison, predict and controller. Respectively composed of Variational Autoencoder (VAE), MDN-RNN and Deep Deterministic Strategy Gradient (DDPG). The framework is trained on a dataset of simulation environment of ball and plate which is built on the CoppeliaSim. We show that the dynamic model learned based on the framework can balance the ball in the middle of the plate. Through subsequent migration training, the ball can be controlled to reach the designated location in the plate.
