Our LLM agents can perform complex hacks like blind SQL union attacks.
SQL Union attacks are actually pretty simple and only work on poorly designed and typically old websites. Pretty much any website of the modern era sanitizes inputs to make such attacks impossible.
I have some doubts about the complex actions bit too. My impression so far is that LLMs are still pretty bad at doing long horizon tasks, that is they’re not reliable enough to use at all. SQL union attacks are the ones that seem to have 45 steps so I’m guessing those steps are mostly just guessing lots of different query structures, not really planning.
SQL Union attacks are actually pretty simple and only work on poorly designed and typically old websites. Pretty much any website of the modern era sanitizes inputs to make such attacks impossible.
I have some doubts about the complex actions bit too. My impression so far is that LLMs are still pretty bad at doing long horizon tasks, that is they’re not reliable enough to use at all. SQL union attacks are the ones that seem to have 45 steps so I’m guessing those steps are mostly just guessing lots of different query structures, not really planning.