What’s hilarious is this is just the same error that allows SQL injections. In this case, the “control layer” (messages from the LLM to the image generator) is getting hijacked by user input.
Were SQL a better language this wouldn’t be possible, all the command strings would separated somehow (such as putting them into a separate memory space) and the interpreter would be unable to execute a string not present at script load. (Arguments can be runtime but the command word can’t)
For LLMs you need some method to keep the channels separate. Dedicated attention heads for the system prompt?
Tokenize the system prompt into a different token space?
Were SQL a better language this wouldn’t be possible, all the command strings would separated somehow
SQL does support prepared statements which forbid injection. Maybe you’re thinking of something stronger than this? I’m not sure how long they’ve been around for, but wikipedia’s list of SQL injection examples only has two since 2015 which hints that SQL injection is much less common than it used to be.
(Pedantic clarification: dunno if this is in any SQL standard, but it looks like every SQL implementation I can think of supports them.)
This used to work. Point is this is a design error, the interpreter is treating the complete string as an input which includes runtime text.
I haven’t touched sql in a long time so I am sure theres a fix but SQL injections were an endemic issue for a long time, like buffer overflows for C. Same idea—design errors in the language itself (including the standard libraries) are what make them possible.
Yeah. It’s still possible to program in such a way that that works, and it’s always been possible to program in such a way that it doesn’t work. But prepared statements make it easier to program in such a way that it doesn’t work, by allowing the programmer to pass executable code (which is probably directly embedded as a literal in their application language) separately from the parameters (which may be user-supplied).
(I could imagine a SQL implementation forbidding all strings directly embedded in queries, and requiring them to be passed through prepared statements or a similar mechanism. That still wouldn’t make these attacks outright impossible, but it would be an added layer of security.)
What’s hilarious is this is just the same error that allows SQL injections. In this case, the “control layer” (messages from the LLM to the image generator) is getting hijacked by user input.
Were SQL a better language this wouldn’t be possible, all the command strings would separated somehow (such as putting them into a separate memory space) and the interpreter would be unable to execute a string not present at script load. (Arguments can be runtime but the command word can’t)
For LLMs you need some method to keep the channels separate. Dedicated attention heads for the system prompt?
Tokenize the system prompt into a different token space?
SQL does support prepared statements which forbid injection. Maybe you’re thinking of something stronger than this? I’m not sure how long they’ve been around for, but wikipedia’s list of SQL injection examples only has two since 2015 which hints that SQL injection is much less common than it used to be.
(Pedantic clarification: dunno if this is in any SQL standard, but it looks like every SQL implementation I can think of supports them.)
https://stackoverflow.com/questions/332365/how-does-the-sql-injection-from-the-bobby-tables-xkcd-comic-work
This used to work. Point is this is a design error, the interpreter is treating the complete string as an input which includes runtime text.
I haven’t touched sql in a long time so I am sure theres a fix but SQL injections were an endemic issue for a long time, like buffer overflows for C. Same idea—design errors in the language itself (including the standard libraries) are what make them possible.
Yeah. It’s still possible to program in such a way that that works, and it’s always been possible to program in such a way that it doesn’t work. But prepared statements make it easier to program in such a way that it doesn’t work, by allowing the programmer to pass executable code (which is probably directly embedded as a literal in their application language) separately from the parameters (which may be user-supplied).
(I could imagine a SQL implementation forbidding all strings directly embedded in queries, and requiring them to be passed through prepared statements or a similar mechanism. That still wouldn’t make these attacks outright impossible, but it would be an added layer of security.)