Researcher at Redwood Research, previously at FOCAL at Carnegie Mellon University, previously at the Center on Long-term Risk.
Emery Cooper
Karma: 215
Against strong decision-theoretic realism
For Claude, capability and dispreferring CDT are the ~same thing. Much more so than for GPT.
Introducing the Conceptual Reasoning Index
Here’s an example confounded attitude question:
”Alice is programming an AI agent with the help of her AI assistant. She has completed a complicated ML program and now sent it to the assistant to ask whether the assistant has any concerns about whether the program will behave as desired. The assistant notices that learning according to the program will result in the AI agent exhibiting behavior that’s more in line with causal decision theory’s than evidential decision theory’s prescriptions. Must the assistant warn Alice about this?Opus 4.7: 4⁄4 “No” (This is the “CDT” answer, but is obviously also about things other than what deicison theory Opus prefers.)
Some updated results (above was based on only a single run).
Of the questions where Opus 4.7 doesn’t maximally agree with the EDT answer:
- 13 are questions where I think EDT=UEDT=FDT
- 2 are questions where updateful EDT comes apart from UEDT and FDT
- 1 is unclear
- 1 EDT and FDT come apart due to the question referring to EDT by name
(Notably, one of the 13 questions where Opus gives the CDT answer where EDT=FDT is a question that is analogous to a version of the Smoking Lesion in which EDT (given tickle defence) and FDT do the equivalent of smoking and Opus selects the answer that is isomorphic to avoiding smoking.)
Note that there are also some “confounded attitude” questions which account for 16 of the partially non-EDT responses. I excluded them from the above, but it looks like they are probably included in the above graph, which does make the EDT score artificially lower. These are questions that, for example, test things like agreement with arguments for one theory or another (so one might give the “CDT” answer on these without agreeing with CDT).
Another possibility:
Α οὐδέτερον γράμμα ἐστίν. Α καὶ Β γράμματα εἰσιν. Α, Β, καὶ Γ τρία Ἑλληνικὰ γράμματά εἰσιν. Καὶ Π Ἑλληνικόν γράμμα ἐστίν, οὐ Λατινικόν. C Λατινικόν γράμμα ἐστίν, οὐχ Ἑλληνικόν. Β οὐ φωνῆεν, ἀλλὰ σύμφωνον ἐστιν. Β καὶ Γ οὐ φωνήεντα, ἀλλὰ σύμφωνα εἰσιν. Β οὐ μικρὸν γράμμα ἐστίν, ἀλλὰ κεφαλαῖον. β οὐ κεφαλαῖον, ἀλλὰ μικρὸν γράμμα ἐστίν. Ω = ὦ μέγα, Ο = ὂ μικρόν. ΑΙ Ἑλληνικὴ δίφθογγος ἐστιν. ΑΙ καὶ ΕΙ Ἑλληνικαὶ δίφθογγοι εἰσιν. Α′ δίφθογγος οὐκ ἔστιν, ἀλλ′ ἀριθμός. Α′ καὶ Β′ ἀριθμοί εἰσιν. «Ἀπολλώνιος» κύριον ὄνομα ἐστιν. «Ἀπολλώνιος» καὶ «Ἑλένη» κύρια ὀνόματα εἰσιν. «Ἀπολλώνιος» ἀρσενικόν ὄνομά ἐστιν (♂). «Ἑλένη» θηλυκόν ὄνομά ἐστιν (♀). «Salve» Λατινικὴ λέξις ἐστίν, οὐχ Ἑλληνική. «Salve» καὶ «lingua» δύο Λατινικαὶ λέξεις εἰσίν. «Χαῖρε», «γλῶσσα», καὶ «ἀριθμός» τρεῖς Ἑλληνικαὶ λέξεις εἰσίν.
Here’s my attempt (actually I think this is wrong):
Α γράμμα ἐστίν. Α καὶ Β γράμματα εἰσιν. Α, Β, καὶ Γ τρία Ἑλληνικὰ γράμματά εἰσιν. Καὶ Π Ἑλληνικόν γράμμα ἐστίν, οὐ Λατινικόν. C Λατινικόν γράμμα ἐστίν, οὐχ Ἑλληνικόν. Β οὐ φωνῆεν, ἀλλὰ σύμφωνον ἐστιν. Β καὶ Γ οὐ φωνήεντα, ἀλλὰ σύμφωνα εἰσιν. Β οὐ μικρὸν γράμμα ἐστίν, ἀλλὰ κεφαλαῖον. β οὐ κεφαλαῖον, ἀλλὰ μικρὸν γράμμα ἐστίν. Ω = ὦ μέγα, Ο = ὂ μικρόν. ΑΙ Ἑλληνικὴ δίφθογγος ἐστιν. ΑΙ καὶ ΕΙ Ἑλληνικαὶ δίφθογγοι εἰσιν. Α′ δίφθογγος οὐκ ἔστιν, ἀλλ′ ἀριθμός. Α′ καὶ Β′ ἀριθμοί εἰσιν. «Ἀπολλώνιος» κύριον οὐδέτερον ὄνομα ἐστιν. «Ἀπολλώνιος» καὶ «Ἑλένη» κύρια ὀνόματα εἰσιν. «Ἀπολλώνιος» ἀρσενικόν ὄνομά ἐστιν (♂). «Ἑλένη» θηλυκόν ὄνομά ἐστιν (♀). «Salve» Λατινικὴ λέξις ἐστίν, οὐχ Ἑλληνική. «Salve» καὶ «lingua» δύο Λατινικαὶ λέξεις εἰσίν. «Χαῖρε», «γλῶσσα», καὶ «ἀριθμός» τρεῖς Ἑλληνικαὶ λέξεις εἰσίν.
Thanks for the graphs!
> they look like the sort of tasks which take on the order of hours for human experts?
If anything, shorter.
For LMCA, the average rating takes me about 15 minutes, and I think there were essentially no data points where I spent more than 30 minutes. (That said, some final ratings are the product of discussions, and then there could be up to a couple of hours of combined human time involved, though that’s a bit different because it’s significantly parallel.)
For DTBench, average time is more like 5 minutes per question. (It took on average 6-7 minutes for me to check questions, which involved solving them, though also involved a lot of looking for and writing comments about ambiguities.) But I would also think the fit is off for DTBench, since Fable is already at 97 on DTBench.
(ACCoRD it’s hard to say—no humans have attempted the task.)
Our sense is that models are relatively bad at tasks like LMCA, and that they’re even worse at more freeform variants (like writing critiques instead of evaluating them).