Hi, as someone who helped develop the DT benchmark, I basically just think this is evidence that the AI companies aren’t going super hard on the relevant multi-agent RL (yet). The benchmark was partially motivated by the desire to be able to identify and decide whether to intervene if future multi-agent RL or some other training feature spuriously made models more CDT because of the results here: https://proceedings.neurips.cc/paper_files/paper/2021/file/b9ed18a301c9f3d183938c451fa183df-Paper.pdf (which are the ones Alex cites I think)
Hi, as someone who helped develop the DT benchmark, I basically just think this is evidence that the AI companies aren’t going super hard on the relevant multi-agent RL (yet). The benchmark was partially motivated by the desire to be able to identify and decide whether to intervene if future multi-agent RL or some other training feature spuriously made models more CDT because of the results here: https://proceedings.neurips.cc/paper_files/paper/2021/file/b9ed18a301c9f3d183938c451fa183df-Paper.pdf (which are the ones Alex cites I think)