PID and MPC control training

Sat, 01 Feb 2025 00:00:00 +0000

Principle of Model Predictive Control

Training delivered in February 2025 at Nehemis for its software teams. The goal: give developers who are not control engineers what they need to understand, tune and debug a control loop, then position Model Predictive Control relative to a PID: when it brings something extra, and at what cost.

The program below is the one that was delivered; it can be adapted to your system and the time available.

Part 1: Understanding a Dynamic System

Open-loop and closed-loop systems: what feedback changes.
Modeling a simple process, transfer functions, poles and stability.
Reading a step response: static gain, time constant, overshoot, settling time.
Experimental identification of a process from real measurements.

Part 2: The PID Controller in Practice

The role of each term (proportional, integral, derivative) and what each one costs.
Steady-state error, disturbance rejection, stability margin.
Tuning methods: empirical approach, Ziegler-Nichols, model-based tuning.
The implementation pitfalls that make a PID that is correct on paper fail in practice: integrator windup (anti-windup), actuator saturation, noise amplified by the derivative term and how to filter it.
Discretization: sampling period, recursive form of the controller, effects of computation time and delays.

Part 3: Introduction to Model Predictive Control

Principle: optimize a sequence of control inputs over a receding horizon rather than reacting to the instantaneous error.
Formulating the optimization problem: prediction model, cost function, prediction and control horizons.
What MPC can do that PID cannot: explicit constraints on inputs and states, multivariable (MIMO) systems, anticipating a setpoint known in advance.
The price to pay: a model is required, computational cost, sensitivity to modeling errors.
Selection criteria: when a well-tuned PID is still the right answer.

Part 4: Applications

Hands-on work on a system representative of the participants’ field.
PID vs. MPC comparison on the same process: performance, constraint satisfaction, robustness.
Discussion of internal use cases and possible next steps.

This training is offered in French or English, on site or remotely, and runs from one to three days depending on how deep you want to go into MPC.

Deep Reinforcement Learning training (5 days)

Thu, 03 Oct 2024 00:00:00 +0000

The agent-environment loop in reinforcement learning

A five-day training in deep reinforcement learning that alternates theory and practice with Stable-Baselines3 and OpenAI Gym: lectures in the morning, hands-on work in the afternoon, and a final project on an environment taken from your own field.

The program below is a template; it can be adapted to your needs and the time available.

Day 1: Reinforcement learning basics

Morning

What RL is, and how it differs from supervised and unsupervised learning
The building blocks: agent, environment, states, actions, rewards
Real applications: games, robotics, finance
The vocabulary: policy, state value, action value, reward, return; Markov decision processes (MDPs)

Afternoon

Tabular methods: Monte Carlo, temporal difference (TD), SARSA, Q-learning
Their strengths, limits and use cases
Hands-on: implement a simple Q-learning agent on FrozenLake (OpenAI Gym)

Day 2: OpenAI Gym environments

Morning

The role of the environment in training an agent
Standard environments: CartPole, MountainCar…
Working with an environment: env.reset(), env.step(), env.render()
Action and observation spaces

Afternoon

Building your own environment with the Gym API
Hands-on: a simple game with a custom reward logic

Day 3: Stable-Baselines3

Morning

Why SB3: reliable implementations of DQN, PPO, A2C…
Installation and setup
Plugging a Gym environment into SB3, a first PPO run on CartPole

Afternoon

Hands-on: tune a PPO agent on CartPole
Callbacks to steer training: early stopping, metric logging

Day 4: Advanced algorithms

Morning: Deep Q-Network (DQN)

From tabular methods to neural networks
Replay buffer, target network
Hands-on: DQN with SB3 on LunarLander

Afternoon: Actor-Critic (A2C, PPO)

The actor-critic principle
What PPO brings over DQN: sample efficiency, stability
Hands-on: PPO on a continuous environment, BipedalWalker

Day 5: Deployment and a project from your field

Morning

Saving and reloading a trained agent
Evaluating it on episodes it never saw in training
Integrating it into a real application: robots, simulations, games

Afternoon: final project

A complete project on an environment taken from your field: modelling, choice of algorithm, training, tuning and evaluation
Wrap-up: the difficulties met and where to go next

Companies | Etienne Chassaing

PID and MPC control training

Part 1: Understanding a Dynamic System

Part 2: The PID Controller in Practice

Part 3: Introduction to Model Predictive Control

Part 4: Applications

Deep Reinforcement Learning training (5 days)

Day 1: Reinforcement learning basics

Day 2: OpenAI Gym environments

Day 3: Stable-Baselines3

Day 4: Advanced algorithms

Day 5: Deployment and a project from your field