Lukas Petersson

• Lukas Petersson 21 Sep 2025 17:24 UTC

  3 points

  0

  in reply to: Matt Goldenberg’s comment on: You can’t eval GPT5 anymore

  We set it to some date in the future

• Lukas Petersson 19 Sep 2025 18:37 UTC

  2 points

  0

  in reply to: Ted Sanders’s comment on: You can’t eval GPT5 anymore

  Thanks! Vending-Bench v2 is going to be fire. Would love to include gpt5 <3

• Lukas Petersson 19 Sep 2025 16:25 UTC

  1 point

  0

  in reply to: vjprema’s comment on: You can’t eval GPT5 anymore

  This is a great point. I admit I have to better understand what each model provider does behind the scenes in the API. Sad if the days of access to the model is gone.

• Lukas Petersson 19 Sep 2025 13:59 UTC

  9 points

  0

  in reply to: Neel Nanda’s comment on: You can’t eval GPT5 anymore

  We thought about that, but then it’s not reproducible if we want to run it for new models later

• Lukas Petersson 19 Sep 2025 3:39 UTC

  17 points

  2

  in reply to: Ted Sanders’s comment on: You can’t eval GPT5 anymore

  Thanks, that would be great!

You can’t eval GPT5 anymore

AI mis­be­havi­our in the wild from An­don Labs’ Safety Report

• Lukas Petersson 3 Jul 2025 11:35 UTC

  4 points

  0

  on: Pro­ject Vend: Can Claude run a small shop?

  Thanks for highlighting our work!

The Same Heaven

Lin­guis­tic Im­pe­ri­al­ism in AI: En­forc­ing Hu­man-Read­able Chain-of-Thought

AI Safety as a YC Startup

• Lukas Petersson 16 Nov 2024 22:13 UTC

  7 points

  0

  on: Which evals re­sources would be good?

  This is probably not the first barrier to getting into evals, but I have an AI safety startup that designs evals. However, we don’t have the capacity to also do good elicitation. I think we lose a lot of signal from our evals because our agent is too weak to explore properly. We’re currently using Inspect’s basic_agent. Metr’s modular_public is better, but we prefer inspect over vivaria otherwise. I think open-sourcing a better agent would be positive for the evals community without contributing to capabilities.