TinyRL: Teaching AI to Swing Up a Real Pendulum

Explore the application of reinforcement learning to teach an AI to swing up a real pendulum in this 22-minute video. Learn about hardware setup, Arduino communication, reinforcement learning concepts, reward functions, and deep neural networks. Follow the process of agent training, troubleshooting, and deployment to an ESP32. Discover how to modify the project scope, optimize hyperparameters, and implement a discrete action space for successful learning. Gain insights into practical challenges and solutions when applying AI to physical systems, and consider potential areas for further research in this fascinating intersection of machine learning and robotics.

Syllabus

- Introduction
- Hardware overview
- Modifying the pendulum tower
- Arduino communication interface
- Overview of reinforcement learning
- Reward function
- Agent actor-critic deep neural network
- Hyperparameter optimization overview
- Agent training with Python
- Troubleshooting an agent that does not learn
- Reduce scope to just swing up and use discrete action space
- Train simpler agent
- Deploy agent to ESP32
- Test agent on the pendulum
- Conclusion and further areas of research