Having worked at METR for some months last year, I just want to chime in to add that they have indeed seen the skulls. This post does a great service to the broader public by going into many important points at length. But these issues and others are also very much top of mind at METR, which is one of the reasons why they caveat results extensively in their publications.
If you haven’t been in touch or visited them already, I highly recommend it. They’re pretty awesome and love to discuss this sort of stuff!
Every time one of these comes out it feels like you’ve successfully dissected another piece of my soul. In a good way!
There’s your approval reward dose. Do keep them coming :)