Perplexity and NVIDIA Researchers Introduce JOLT: a Joint Teacher-Student Reinforcement Learning Method

Researchers from Perplexity and NVIDIA introduced JOLT, short for Joint On-policy Learning and Teaching, a post-training method in which one model plays both teacher and student. The teacher sees privileged information, such as a verified solution; the student sees only the task. The twist is that the teacher is trained too, so its guidance stays within the student's reach.

On the MATH benchmark, the team reports JOLT matched the accuracy of GRPO, a widely used reinforcement learning method, using 6.5 times fewer processed training tokens. For any lab paying for reinforcement learning runs, that's a compute bill question.

We like research that attacks cost rather than just chasing a leaderboard, and this does. Standard outcome-reward training hands a model one score per attempt, which says nothing about which steps helped. On long agent tasks, where success is rare, learning crawls. Self-distillation fills in token-level feedback, and JOLT's contribution is noticing that a teacher holding the answer can take shortcuts the student can't follow, then training the teacher to stay close to the student.

The results are consistent across the five settings tested. The enhanced version, JOLT+, which adds the student's own reward, posted the top mean score in every one. The biggest gain was on AppWorld, an agentic benchmark, where it reached 72.09% against 62.20% for GRPO. On Terminal-Bench 2.1 it improved pass@1 from 8.25% to 10.96%.

Now the honest caveats. All scores come from the authors' own runs. The models tested are small to mid-sized, from Qwen3-0.6B up to a 30-billion-parameter mixture-of-experts model, and training hardware wasn't disclosed. Plain JOLT without the student reward actually trailed GRPO on AppWorld and Terminal-Bench. And the compute savings figure comes from fitted accuracy curves on one benchmark, with savings ranging from 4.3 to 7.4 times depending on the target.

Where does this lead if it scales? Cheaper post-training means more organizations can train capable agents on their own tasks, which we see as a good thing for competition. It also means capable agentic behavior gets cheaper to produce for anyone, and the safety evaluation work doesn't get cheaper at the same rate. That gap is the one I'm watching.

GRPO is a reinforcement learning method that scores whole attempts. On-Policy Self-Distillation, or OPSD, which JOLT builds on, lets the same model act as its own teacher when given extra context. JOLT was also compared with π-Distill. The paper is on arXiv alongside a Perplexity research blog post.