Machine Learning

Learning Inverse Dynamics with Gaussian Process Regression

Mechanics and Control of Liviving Systems Laboratory

My work at Cleveland State focused on improving the performance of functional electrical stimulation (FES) neuroprostheses and incorporating force sensors on a patient’s wrists so a caregiver can intuitively correct an action performed by the neuroprosthesis. FES stimulates paralyzed muscles via implanted electrodes to return arm functionality to people with tetraplegia [1]. My research involved using machine learning, specifically Gaussian Process Regression (GPR) [2], which could allow the system to learn the best way to control the neuroprosthesis and offer the ability to continuously improve its control based on caregiver feedback. In order to test the GPR algorithms, I designed a pendulum simulation in MATLAB able to be paused in order to provide feedback and restart the simulation. My algorithms related the torque given to the pendulum to its angular positions and velocities and used this information to learn the inverse dynamics of the system. Though in reality torque is a system input and position and velocity are system outputs, I defined position and velocity as inputs and torque as an output in my GPR model. This allows the user to query at desired positions and velocities in order for the system to return the required torques to achieve those positions and velocities. Hence, the GPR algorithms learn an approximation of the system model without actually being told any system properties. This learned model is used to predict the required torque to achieve a new position and velocity, as described.

After selecting an appropriate control for the pendulum, the user can pause the simulation and deliver feedback in the form of a virtual push on the pendulum rod to correct its motion. This push is interpreted as some addition or subtraction of the actual motor torque turning the pendulum. The model learned by the GPR uses this new data to recalculate the required torque and create a control that causes a motion closer to the one desired. The end goal will be to use a similar methodology to provide intuitive feedback to a neuroprosthesis for improved control.

Watch video on YouTube

MATLAB Simulation

GPR is used to learn how to control a virtual pendulum and feedback is given to change the motion of the pendulum.

In addition, I worked with implementing a method to test multiple control options in order to choose the one that promises the best results. To this end, I began implementing a technique using uncertain inputs for multi-step ahead predictions with GPR. In this case, the system input is the torque, position, and velocity at the current time step and the output is the position at the next time step. This allows the controller to estimate the outcomes of multiple input options by querying at different torque values to determine which will yield the most desirable results. This is very useful for improved control of FES and any robotics/control system as it gives the controller multiple options to choose from, which is almost always better than having only one rigid control option. I began implementing the techniques for uncertain inputs as described in [3].

In order to test these methods for robustness in a real system, I designed and built a robotic pendulum which I used as a testing platform for the algorithms mentioned thus far, to ensure that they can adequately learn the system model and use it to control the system. My system consisted of a brushless DC geared motor controlled by a servo drive, with an optical encoder for attaining position and velocity data. Resistor force sensors were attached to the pendulum rod to allow the user to deliver force feedback, which the system used to update its method of control, as done in the MATLAB simulation. While the goal is to implement these techniques for a neuroprosthesis, they can also prove valuable in applications such as industry robots working alongside humans, where the robot must learn from human feedback in order to operate safely.

Watch video on YouTube

Robotic Pendulum

Tactile Feedback is given to robotic pendulum to teach it a new motion to replicate.

I also worked on modifying a computer arm simulation which was developed by Dr. Antonie van den Bogert at Cleveland State, known as the Dynamic Arm Simultion (https://simtk.org/home/das) [4, 5], to allow the use of GPR for learning and controlling the muscles in the simulation. I worked with Dr. van den Bogert and a biomedical engineering masters student to modify the simulation software such that when the arm muscles are excited, the arm is held in place by a force acting with the same magnitude and in the opposite direction as the force produced on the arm by the excitations. This allowed us to record the forces caused on the arm by a given excitation. From such data obtained in real-world clinical trials, a GPR model relating arm force and muscle excitation can be learned. A caregiver will then be able to give feedback to the FES neuroprosthesis by guiding the paralyzed arm of a patient to a desired position using force sensors fitted on the patient’s wrists. The force/excitation GPR model will then be queried at the forces given by the caregiver to determine the new stimulation necessary to take the arm of the patient to the desired position.

Watch video on YouTube

Dynamic Arm Simulator

External force control of arm to achieve a four-target trajectory.

Click here to open poster

Click here to open thesis

The 9th Annual Cleveland State Interdisciplinary Research Conference

Awarded the Cleveland State Undergraduate Research Grant to continue my work during the Spring 2016 semester.

Conferences/Presentations

Lungu, Titus; Schearer, Eric M. Learning Inverse Dynamics with Gaussian Process Regression. The 9th Annual Cleveland State Interdisciplinary Research Conference. Cleveland State University. Cleveland, Ohio. Nov. 2015.

Lungu, Titus. Robots, Prosthetics, and Artificial Intelligence. High School Fall Visit Day. Cleveland State University. Cleveland, Ohio. Oct. 2015.

Lungu, Titus; Schearer, Eric M. Learning Inverse Dynamics with Gaussian Process Regression. Engineering Research Day. Cleveland State University. Cleveland, Ohio. Oct. 2015.

References

[1] Doucet, Barbara M., Amy Lam, and Lisa Griffin, Neuromuscular Electrical Stimulation for Skeletal Muscle Function, The Yale Journal of Biology and Medicine, vol 85, no 2, pp. 201-215, 2012.

[2] Rasmussen, Carl Edward., and Christopher K. I. Williams. Gaussian Processes for Machine Learning. Cambridge, MA: MIT, 2006. Print.

[3] Girard, Agathe, et al, Gaussian Process Priors with Uncertain Inputs- Application to MultipleStep Ahead Time Series Forecasting, 2003.

[4] Antonie J. van den Bogert, Dimitra Blana, Dieter Heinrich, Implicit Methods for Efficient Musculoskeletal Simulation and Optimal Control, Procedia IUTAM, Volume 2, 2011, Pages 297-316, ISSN 2210-9838, http://dx.doi.org/10.1016/j.piutam.2011.04.027.

[5] Chadwick, E., Blana, D., Kirsch, R., & Bogert, A. van den. (2014). Real-Time Simulation of Three-Dimensional Shoulder Girdle and Arm Dynamics. IEEE Transactions on Biomedical Engineering, 61(7), 1947–1956. doi:10.1109/TBME.2014.2309727

© 2015-2017 Titus Lungu. All rights reserved.

Photo credit: © iStockphotos