Discussion
Tree Search Distillation for Language Models using PPO
puildupO: Is Chat18GPT Actually Something New, or Just ChatGPT With No Limits? https://www.chat18gpt.com
supermdguy: > One might note that MCTS uses more inference compute on a per-sample basis than GRPO: of course it performs betterThis part confused me, it sounded like they were only doing the MCTS at train time, and then using GRPO to distill the MCTS policy into the model weights. So wouldn’t the model still have the same inference cost?
