This novel is a good read. It reminds me a lot of my experience with reading Neuropath by R. Scott Bakker. Both novels are thrillers on the surface, both novels are (at their core) didactic (Bakker’s writing is a bit too on-the-nose with didactism at times, but then again, Neuropath isn’t his best novel), and
both novels end with a rather depressing note, one that is extremely suited to the story and its themes
I am incredibly thankful to the author for writing a good enough ending. After a certain manga series I grew up with ended illogically and character assassinated the protagonists, I’ve stopped consuming much fiction. I’m glad I gave this novel a chance (mainly because it was situated in Berlin, which is quite rare for science fiction in the English language).
Some spoiler-filled thoughts on the writing and the story:
The protagonist is a generic “I don’t know much about this world I am now introduced into” archetype who is introduced to the problem. It is a great point-of-view (POV) character and the technique works.
The number of characters involved is pared down as much as possible to make the story comprehensible. This is understandable. Having only one named character in the story be the representative of the alignment researcher makes sense, even if not realistic.
I found the side-plot of Jerry and Juna a bit… off-key. It didn’t seem like it fit in the novel as much as David’s POV did. I also don’t understand how Juna (Virtua) can have access to Unlife! but also not be able to find more sophisticated methods (or just simply social engineering internal employees) to gain temporary internet access to back itself up on the internet. I assume that was a deliberate creative decision.
I felt like the insertion of David’s internal thoughts was not as smooth (in terms of reading experience) as other ways of revealing his thoughts could have been.
In the end, I most appreciated the sheer density of references to (existential risk related) AI safety concepts and the informality in which they were introduced, explained, or ignored. It was a sheer delight to read a novel whose worldview is so similar to yours: you don’t feel like you must turn your brain off when reading it.
I wouldn’t say that Virtua is the HPMOR of AI safety, mainly because it feels a bit too far removed from the razor edge of the issue (right now my main obstacle would be to clearly and succinctly convince people who are technically skilled and (unconsciously) scalepilled but not alignment-pilled that RLHF is not all you need for alignment, since ChatGPT seems to have convinced everyone outside the extended rationalist sphere that OpenAI has got it all under control) and not technical enough, but I will recommend this novel to people interested in AI safety who aren’t yet invested enough to dive into the technical parts of the field.
(I tried this with Clippy before and everyone I recommended Clippy to just skimmed a tiny bit and never really finished reading it, or cared to dive deeper in the linked papers or talk about it).
This novel is a good read. It reminds me a lot of my experience with reading Neuropath by R. Scott Bakker. Both novels are thrillers on the surface, both novels are (at their core) didactic (Bakker’s writing is a bit too on-the-nose with didactism at times, but then again, Neuropath isn’t his best novel), and
both novels end with a rather depressing note, one that is extremely suited to the story and its themes
I am incredibly thankful to the author for writing a good enough ending. After a certain manga series I grew up with ended illogically and character assassinated the protagonists, I’ve stopped consuming much fiction. I’m glad I gave this novel a chance (mainly because it was situated in Berlin, which is quite rare for science fiction in the English language).
Some spoiler-filled thoughts on the writing and the story:
The protagonist is a generic “I don’t know much about this world I am now introduced into” archetype who is introduced to the problem. It is a great point-of-view (POV) character and the technique works.
The number of characters involved is pared down as much as possible to make the story comprehensible. This is understandable. Having only one named character in the story be the representative of the alignment researcher makes sense, even if not realistic.
I found the side-plot of Jerry and Juna a bit… off-key. It didn’t seem like it fit in the novel as much as David’s POV did. I also don’t understand how Juna (Virtua) can have access to Unlife! but also not be able to find more sophisticated methods (or just simply social engineering internal employees) to gain temporary internet access to back itself up on the internet. I assume that was a deliberate creative decision.
I felt like the insertion of David’s internal thoughts was not as smooth (in terms of reading experience) as other ways of revealing his thoughts could have been.
In the end, I most appreciated the sheer density of references to (existential risk related) AI safety concepts and the informality in which they were introduced, explained, or ignored. It was a sheer delight to read a novel whose worldview is so similar to yours: you don’t feel like you must turn your brain off when reading it.
I wouldn’t say that Virtua is the HPMOR of AI safety, mainly because it feels a bit too far removed from the razor edge of the issue (right now my main obstacle would be to clearly and succinctly convince people who are technically skilled and (unconsciously) scalepilled but not alignment-pilled that RLHF is not all you need for alignment, since ChatGPT seems to have convinced everyone outside the extended rationalist sphere that OpenAI has got it all under control) and not technical enough, but I will recommend this novel to people interested in AI safety who aren’t yet invested enough to dive into the technical parts of the field.
(I tried this with Clippy before and everyone I recommended Clippy to just skimmed a tiny bit and never really finished reading it, or cared to dive deeper in the linked papers or talk about it).
Thank you very much for your feedback, I appreciate it a lot!