Neuro-sama (the LLM-based AI VTuber) has beaten the game some time ago. As the code isn’t open it’s not possible to confirm whether the StS AI was done with LLMs though. Would definitiely be interesting to see how frontier LLMs perform!
In Claude’s first try, it played Ironclad on Ascension 1 and died to Hexaghost, the Act 1 boss. It wasn’t terrible but occasionally got the mechanics a little bit mixed up.
If anyone wants to make a proper harness in the future, I think probably the most interesting question here is if the LLM can learn from multiple playthroughs, unlocking harder difficulties, etc.
Interesting how much it’s relying on having information in training data and being able to look stuff up. I wonder how it would do with a “blind” play through of a game that didn’t previously exist.
This benchmark includes a Slay the Spire environment! When it was written, Gemini 2.5 did the best, getting roughly halfway through a non-Ascension run.
In Claude’s first try, it played Ironclad on Ascension 1 and died to Hexaghost, the Act 1 boss. It wasn’t terrible but occasionally got the mechanics a little bit mixed up.
I’m curious to see how well LLMs can play Slay the Spire. I could actually try that manually and see what happens.
Neuro-sama (the LLM-based AI VTuber) has beaten the game some time ago. As the code isn’t open it’s not possible to confirm whether the StS AI was done with LLMs though. Would definitiely be interesting to see how frontier LLMs perform!
Huh. Link for those interested.
Please do!
In Claude’s first try, it played Ironclad on Ascension 1 and died to Hexaghost, the Act 1 boss. It wasn’t terrible but occasionally got the mechanics a little bit mixed up.
Here’s the link to the chat history.
Thanks for following through.
If anyone wants to make a proper harness in the future, I think probably the most interesting question here is if the LLM can learn from multiple playthroughs, unlocking harder difficulties, etc.
Modern LLMs, maybe through notetaking?
Interesting how much it’s relying on having information in training data and being able to look stuff up. I wonder how it would do with a “blind” play through of a game that didn’t previously exist.
This benchmark includes a Slay the Spire environment! When it was written, Gemini 2.5 did the best, getting roughly halfway through a non-Ascension run.
I’d love to see the results of this :)
In Claude’s first try, it played Ironclad on Ascension 1 and died to Hexaghost, the Act 1 boss. It wasn’t terrible but occasionally got the mechanics a little bit mixed up.
Here’s the link to the chat history.