This is a really interesting post, thank you! I recently tried to explore the bottom of the capability curve (what can you one-shot?) but have been struggling to get a glimpse at the top (what can the most well orchestrated, well designed AI systems achieve?) and appreciate your detailed writeup as offering some clear insight into that latter question.
Coming from academia, my strong prior is that system designers kind of know what to look for ahead of time, and build tools to find outcomes they strongly suspect will exist. IE, often a fuzzer will seem perfectly designed to find just the specific bug(s) reported in the paper, and nothing else … almost as if the authors knew about those ahead of time, or strongly suspected they’d exist, and designed the system accordingly. I wonder if you could comment, along those lines, on what findings Aisle has produced autonomously or semi-autonomously which surprised you and fell outside the envelope of results you’d deliberately designed the system to prosecute?
This is a really interesting post, thank you! I recently tried to explore the bottom of the capability curve (what can you one-shot?) but have been struggling to get a glimpse at the top (what can the most well orchestrated, well designed AI systems achieve?) and appreciate your detailed writeup as offering some clear insight into that latter question.
Coming from academia, my strong prior is that system designers kind of know what to look for ahead of time, and build tools to find outcomes they strongly suspect will exist. IE, often a fuzzer will seem perfectly designed to find just the specific bug(s) reported in the paper, and nothing else … almost as if the authors knew about those ahead of time, or strongly suspected they’d exist, and designed the system accordingly. I wonder if you could comment, along those lines, on what findings Aisle has produced autonomously or semi-autonomously which surprised you and fell outside the envelope of results you’d deliberately designed the system to prosecute?