Troubleshooting¶

Fixes for problems people actually hit, grouped by symptom. Each entry says which OS it applies to. If yours is not here, run the smoke test and bring the entire output to an organiser.

First, the 90% fix

Most "it doesn't work" reports are one of three things:

The environment isn't active → conda activate rlbootcamp.
You're in the wrong directory → run from the repository root.
The environment is stale → conda env update -f environment.yml --prune.

Imports and environments¶

`ModuleNotFoundError` for `torch`, `gymnasium`, `stable_baselines3`…¶

All OSes. The environment isn't active, or wasn't fully created.

conda activate rlbootcamp
conda env update -f environment.yml --prune

Packages "missing" even though you definitely installed them¶

All OSes. You're using a different interpreter — system Python, the base environment, or another venv. Verify:

conda run -n rlbootcamp python -c "import sys; print(sys.executable)"

The path must contain envs/rlbootcamp. If you are in a notebook, check the kernel in the top-right corner instead — see the notebook kernel note.

Imports fail inside a notebook but work in the terminal¶

All OSes. The notebook is running on a different Python than the one you installed everything into. The notebook opens normally, so this rarely looks like a setup problem — but nothing will import.

Run this in a notebook cell (not the terminal):

import sys; print(sys.executable)

The path must contain rlbootcamp. If it does not, register this environment as a named kernel, then pick Python (rlbootcamp) from the kernel menu (top-right in JupyterLab):

conda activate rlbootcamp
python -m ipykernel install --user --name rlbootcamp \
                            --display-name "Python (rlbootcamp)"

Restart the kernel afterwards — changing the kernelspec does not affect an already-running kernel.

The usual cause

Launching jupyter lab from a terminal where rlbootcamp was not active. A jupyter from your base environment starts a base kernel. python scripts/smoke_test.py checks for exactly this.

`jupyter: command not found`¶

All OSes. Either the environment is not active, or it predates JupyterLab being added to environment.yml:

conda activate rlbootcamp
conda env update -f environment.yml --prune

`No module named 'sb3_contrib'`¶

All OSes. sb3-contrib provides extra algorithms (MaskablePPO, TRPO, QR-DQN). It is in environment.yml; if your environment predates it:

pip install sb3-contrib

`conda env create` hangs on "Solving environment"¶

All OSes. The classic solver can take a very long time. Either wait it out, or switch to the much faster libmamba solver:

conda install -n base conda-libmamba-solver
conda config --set solver libmamba

Miniforge uses a fast solver by default, which is one reason we recommend it.

Rendering and video¶

`DependencyNotInstalled: pygame is not installed`¶

All OSes. Classic-control environments (CartPole, Pendulum, MountainCar) need pygame to render — including when recording video, not just for on-screen windows. Training works without it, which is why this often surfaces late.

pip install "gymnasium[classic_control]"

Training works but `render()` or saving `.mp4` fails¶

All OSes. You're missing ffmpeg, the binary that writes video files. It is not a Python package, so pip list will not show the problem.

Linux macOS Windows

conda install -n rlbootcamp -c conda-forge ffmpeg
# or system-wide: sudo apt-get install -y ffmpeg

conda install -n rlbootcamp -c conda-forge ffmpeg   # or: brew install ffmpeg

conda install -n rlbootcamp -c conda-forge ffmpeg

Verify with conda run -n rlbootcamp ffmpeg -version.

`GLFWError` / `Failed to create GLFW window` / a black render window¶

This is OpenGL, and the fix differs per OS:

Linux macOS Windows

Install GL libraries, and use an off-screen renderer on headless machines (servers, CI, laptops without a display attached):

sudo apt-get install -y libgl1-mesa-glx libglew-dev libosmesa6-dev
export MUJOCO_GL=egl      # or: osmesa   (headless software rendering)

Add the export to your ~/.bashrc to make it stick.

Use the native backend:

export MUJOCO_GL=glfw

Interactive windows must run from a real Terminal session, not over plain SSH. On Apple Silicon also check you are on an arm64 Python — see Apple Silicon.

Native windowed rendering is finicky. Two reliable options:

Don't open a live window. Record to .mp4 instead — you only need ffmpeg, not a GL context.
Run everything under WSL2 and follow the Linux tab.

MuJoCo¶

`mujoco` import errors, or `Ant-v5` won't build¶

All OSes. Install the extras and the binding:

pip install "gymnasium[mujoco]" mujoco

Modern MuJoCo (3.x) ships as a self-contained pip wheel — there is no manual binary download and no mjkey.txt licence step any more. If a tutorial tells you to set LD_LIBRARY_PATH or MUJOCO_PY_*, it is describing the old mujoco-py and is out of date. Ignore it.

PyTorch¶

"You are trying to run PPO on the GPU, but…"¶

All OSes. This warning is correct, and harmless. Small MLP policies are slower on a GPU than on a CPU, because moving data to the device costs more than the matrix multiplies save. Pass device="cpu" to silence it:

model = PPO("MlpPolicy", env, device="cpu")

Torch is a huge download / I want a specific CUDA build¶

Linux & Windows with an NVIDIA GPU. The default wheel is fine for every bootcamp exercise. If you specifically want a particular CUDA build, use the selector at https://pytorch.org/get-started/locally/:

pip install torch --index-url https://download.pytorch.org/whl/cu124

A CPU-only machine is completely sufficient. You are not disadvantaged.

`OMP: Error #15: Initializing libiomp5...`¶

All OSes, most often macOS. Duplicate OpenMP runtimes. Workaround:

export KMP_DUPLICATE_LIB_OK=TRUE

Apple Silicon (M1–M4)¶

macOS arm64. Make sure the whole stack is native arm64, not Rosetta x86_64:

python -c "import platform; print(platform.machine())"   # want: arm64

If it prints x86_64 you installed an Intel Conda. Reinstall Miniforge (its installer picks the right architecture automatically) and recreate the environment. This is precisely why we recommend Miniforge.

Windows specifics¶

Long path errors during install¶

Windows. Enable long paths once, in an admin PowerShell:

New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" `
  -Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force

Reboot, then recreate the environment.

`conda` is not recognised¶

Windows. Use the Miniforge Prompt from the Start menu rather than plain cmd/PowerShell — or run conda init powershell once and reopen the shell.

Recommended: WSL2¶

Windows. The smoothest path for MuJoCo and ffmpeg. In an admin PowerShell:

wsl --install -d Ubuntu

Reboot, set your Ubuntu username and password, then inside Ubuntu follow the Linux installation tab from the top.

Networks and disks¶

Proxies and corporate networks¶

All OSes. Corporate proxies and TLS-inspecting firewalls break conda and pip with certificate errors such as SSLError, CERTIFICATE_VERIFY_FAILED or ProxyError.

# Tell conda and pip about the proxy
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080

If your organisation uses its own certificate authority, point the tools at its bundle rather than disabling verification:

conda config --set ssl_verify /path/to/corporate-ca-bundle.crt
pip config set global.cert /path/to/corporate-ca-bundle.crt

Don't just turn verification off

ssl_verify: false and pip --trusted-host are widely suggested online. They work by disabling the check that protects you from tampered packages. Use the CA bundle instead. If you cannot, install from a home network instead and bring the working environment with you.

Out of disk space mid-install¶

All OSes. A half-written environment is worse than none: it produces confusing partial-import errors. Clean up and start again:

conda clean --all          # reclaims cached packages, often several GB
conda env remove -n rlbootcamp
conda env create -f environment.yml

On a managed machine, check your home directory quota — Conda installs everything there by default.

Reproducibility¶

My results changed between runs¶

All OSes. Expected, and worth internalising early. RL is stochastic: network initialisation, action sampling, and environment resets are all random. Seeding makes a single run reproducible:

model = PPO("MlpPolicy", env, seed=0)
obs, info = env.reset(seed=0)

But a seeded run is one sample, not a result. Conclusions need several seeds. If two seeds disagree wildly, that is information about your setup, not a bug to suppress.

Note that exact bit-for-bit reproducibility across different machines, OSes or library versions is not guaranteed even with identical seeds.

Still stuck?¶

Re-run python scripts/smoke_test.py and copy the entire output.
Note your OS, whether you used Conda or venv, and the output of conda run -n rlbootcamp python -c "import sys; print(sys.executable)".
Open an issue on this repository, or bring it to an organiser.

Doing this before the event costs you five minutes. Doing it during the first session costs you a session.