Installation¶

This is the only installation guide

Every RL Bootcamp edition points here. If you find another set of setup instructions — in a tutorial repository, a PDF, a notebook cell — it is out of date. Use this page.

Four commands and a smoke test, about 30 minutes, most of it downloading. Do it before the bootcamp: PyTorch and MuJoCo are several GB and venue Wi-Fi cannot serve a full room at once.

You need git, a Conda distribution and ~5 GB of free disk.

1. Install Conda¶

We recommend Miniforge — minimal and conda-forge-first. Miniconda and Anaconda also work. Skip this step if you already have one.

Linux macOS Windows

wget -O Miniforge3.sh "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3.sh -b -p "$HOME/miniforge3"
source "$HOME/miniforge3/etc/profile.d/conda.sh"
conda init bash          # or: conda init zsh

Reopen your terminal. Details: Linux.

brew install miniforge   # or use the installer below
conda init zsh

curl -L -o Miniforge3.sh "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3.sh -b -p "$HOME/miniforge3"
source "$HOME/miniforge3/etc/profile.d/conda.sh"
conda init zsh

Reopen Terminal. Details, including Apple Silicon: macOS.

Download the Miniforge3 Windows x86_64 installer from the releases page and run it with the defaults.
Open Miniforge Prompt from the Start menu. Run every later command there, not in plain cmd.

Details, including WSL2: Windows.

2. Create the environment¶

git clone https://github.com/SARL-PLUS/rl-bootcamp-setup.git
cd rl-bootcamp-setup
conda env create -f environment.yml
conda activate rlbootcamp

Expect 5–15 minutes. Every command from here on assumes rlbootcamp is active.

This is not the tutorial code

This repository is the environment, the smoke test and the examples. Your edition's exercises are published separately on the day — and that is the point: you can be completely ready before they exist.

What gets installed

Python 3.12 · NumPy · pandas · Matplotlib · PyYAML · ffmpeg · JupyterLab + Notebook · Gymnasium (+ MuJoCo and classic-control extras) · MuJoCo · Stable-Baselines3 · sb3-contrib · PyTorch · Hydra · TensorBoard · moviepy · pytest. The commented list is environment.yml at the repository root.

Alternative: pip only, no Conda

You must then install ffmpeg yourself — see troubleshooting.

python3.12 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install --upgrade pip
pip install "gymnasium[mujoco,classic_control]>=1.0" mujoco \
            "stable-baselines3>=2.3" sb3-contrib torch \
            tensorboard hydra-core moviepy tqdm pytest \
            numpy pandas matplotlib pyyaml jupyterlab notebook ipykernel

3. Smoke test¶

This is the step that matters. From the repository root:

python scripts/smoke_test.py

It checks your interpreter, every required package, ffmpeg, a classic-control environment, MuJoCo physics, a short PPO training run, rendering to screen and to .mp4, and that a notebook kernel starts on this environment's Python. Checks are independent, so one failure does not mask the rest.

You want every line to say ok and the last line to read Everything works. You are ready for the bootcamp. Version numbers will differ from the sample below — that is fine.

Expected output

1. Python interpreter
---------------------
       executable : /home/you/miniforge3/envs/rlbootcamp/bin/python
       version    : 3.12.12
       platform   : Linux x86_64
[  ok  ] Python version — 3.12.12
[  ok  ] Environment — running inside rlbootcamp

2. Packages
-----------
[  ok  ] NumPy                — 2.3.4  (arrays and maths)
[  ok  ] Matplotlib           — 3.10.6  (plots and rendering)
[  ok  ] PyTorch              — 2.12.0  (neural networks)
[  ok  ] Gymnasium            — 1.2.3  (the environment API)
[  ok  ] Stable-Baselines3    — 2.8.0  (RL algorithms)
[  ok  ] SB3-Contrib          — 2.8.0  (extra algorithms (masking, TRPO))
[  ok  ] MuJoCo               — 3.9.0  (physics simulation)
[  ok  ] TensorBoard          — 2.20.0  (training curves)
[  ok  ] JupyterLab           — 4.4.7  (running the notebook sessions)
[  ok  ] ipykernel            — 6.30.1  (the kernel notebooks actually run on)
[  ok  ] ffmpeg               — found on PATH  (saving .mp4 videos)

3. Stepping a classic-control environment
-----------------------------------------
[  ok  ] CartPole-v1 — obs shape (4,), reward 1.0

4. MuJoCo physics
-----------------
[  ok  ] Ant-v5 — obs shape (105,), action shape (8,)

5. Training (this takes ~15-30 seconds)
---------------------------------------
[  ok  ] PPO on CartPole — trained 5k steps, mean return 381

6. Rendering
------------
[  ok  ] classic-control frame — CartPole-v1 rendered 600x400 RGB
[  ok  ] video export — wrote smoke-episode-0.mp4 (3 KB)
[  ok  ] MuJoCo frame — Ant-v5 rendered 480x480 RGB

7. Notebooks
------------
[  ok  ] jupyter launcher — /home/you/miniforge3/envs/rlbootcamp/bin/jupyter
[  ok  ] notebook kernel — runs on this environment's Python

Summary
-------
Everything works. You are ready for the bootcamp.

Anything that says FAIL → Troubleshooting, with the whole output.

MuJoCo¶

Covered by checks 4 and 6 above; no separate step. MuJoCo 3.x is a self-contained pip wheel — there is no binary download and no mjkey.txt licence file any more. If something tells you otherwise, it is describing the old mujoco-py.

Rendering¶

Check 6 draws a frame, writes an .mp4, and renders MuJoCo off-screen. A warn on the MuJoCo frame means no display is attached (a server, a VM, plain SSH) rather than a broken install:

MUJOCO_GL=osmesa python scripts/smoke_test.py    # or: MUJOCO_GL=egl

4. Open a notebook¶

Some sessions are delivered entirely as Jupyter notebooks. The smoke test proves a kernel can start; do this once to prove you can drive one.

conda activate rlbootcamp
jupyter lab              # or: jupyter notebook

In a new Python notebook:

import sys, gymnasium as gym
print(sys.executable)
print(gym.make("CartPole-v1").reset(seed=0)[0])

Check which kernel the notebook is using

The most common notebook problem, and it does not look like a setup problem: the notebook opens fine, then every import fails.

The path printed by sys.executable must contain rlbootcamp. If not, register this environment as a named kernel and select it from the kernel menu:

python -m ipykernel install --user --name rlbootcamp \
                            --display-name "Python (rlbootcamp)"

5. Your editor¶

Optional — everything above works from a terminal. If you prefer an IDE, the interpreter and the notebook kernel are two separate settings and are easily confused:

6. Keep it fresh¶

Environments rot. Re-run the smoke test the night before the event:

conda activate rlbootcamp
git pull
conda env update -f environment.yml --prune
python scripts/smoke_test.py

Next: Working in the Conda environment →, or spend half an hour on the worked example — setup being correct is not the same as you being ready.