NanoRL
NanoRL – RL training for LLMs in ~1,800 lines The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster (e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group](https://docs.skypilot.ai/en/latest/examples/job-groups.html) on k8s ). All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.