At the end of the day, I hope this will have been a cowpox situation and lead people to be better informed at avoiding actual dangerous information hazard situations in the future.
I seem to remember reading a FAQ for “what to do if you think you have an idea that may be dangerous” in the past. If you know what I’m talking about, maybe link it at the end of the article?
I think genuinely dangerous ideas are hard to come by though. They have to be original enough that few people have considered them before, and at the same time have powerful consequences. Ideas like that usually don’t pop into the heads of random, uninformed strangers.
Depends on your definition of “Dangerous.” I’ve come across quite a few ideas that tend to do -severe- damage to the happiness of at least a subset of those aware of them. Some of them are about the universe; things like entropy. Others are social ideas, which I won’t give an example of.
I think genuinely dangerous ideas are hard to come by though.
Daniel Dennett wrote a book called “Darwin’s Dangerous Idea”, and when people aren’t trying to play down the basilisk (i.e. almost everywhere), people often pride themselves on thinking dangerous thoughts. It’s a staple theme of the NRxers and the manosphere. Claiming to be dangerous provides a comfortable universal argument against opponents.
I think there are, in fact, a good many dangerous ideas, not merely ideas claimed to be so by posturers. Off the top of my head:
There are some things which could be highly dangerous which are protected almost purely by thick layers of tedium.
Want to make nerve gas? well if you can wade through a thick pile of biochemistry textbooks the information isn’t kept all that secret.
Want to create horribly deadly viruses? ditto.
The more I learned about physics, chemistry and biology the more I’ve become certain that the main reason that major cities have living populations is that most of the people with really deep understanding don’t actually want to watch the world burn.
You often find that extremely knowledgeable people don’t exactly hide knowledge but do put it on page 425 of volume 3 of their textbook, written in language which you need to have read the rest to understand. Which protects it effectively from 99.99% of the people who might use it to intentionally harm others.
Argument against: back when cities were more flamable, people didn’t set them on fire for the hell of it.
On the other hand, it’s a lot easier to use a timer and survive these days, should you happen to not be suicidal.
“I want to see the world burn” is a great line of dialogue, but I’m not convinced it’s a real human motivation. Um, except that when I was a kid, I remember wishing that this world was a dream, and I’d wake up. Does that count?
Second thought—when I was a kid, I didn’t have a method in mind. What if I do serious work with lucid dreaming techniques when I’m awake? I don’t think the odds of waking up into being a greater intelligence are terribly good, nor is there a guarantee that my live would be better. On the other hand, would you hallucinations be interested in begging me to not try it?
Based on personal experience, if you’re dreaming I don’t recommend trying to wake yourself up. Instead, enjoy your dream until you’re ready to wake up naturally. That way you’ll have far better sleep.
Based on personal experience, I would have agreed with you, right up until last year, when I found myself in the rather terrifying position of being mentally aroused by a huge crash in my house, but unable to wake up all the way for several seconds afterward, during which my sleeping mind refused to reject the “something just blew a hole in the building we’re under attack!” hypothesis.
(It was an overfilled bag falling off the wall.)
But absent actual difficulty waking for potential emergencies, sure; hang out in Tel’aran’rhiod until you get bored.
Sorry, should have defined dangerous ideas better—I only meant information that would cause a rational person to drastically alter their behavior, and which would be much worse for society as a whole when everyone is told at once about it.
I hope they’re as hard to come by as you think they are.
Alternatively, Roko could be part of the 1% of people who think of a dangerous idea (assuming his basilisk is dangerous) and spread it on the internet without second guessing themselves. Are there 99 other people who thought of dangerous ideas and chose not to spread them for our 1 Roko?
At the end of the day, I hope this will have been a cowpox situation and lead people to be better informed at avoiding actual dangerous information hazard situations in the future.
I seem to remember reading a FAQ for “what to do if you think you have an idea that may be dangerous” in the past. If you know what I’m talking about, maybe link it at the end of the article?
Perhaps the article you read was Yvain’s The Virtue of Silence?
I think genuinely dangerous ideas are hard to come by though. They have to be original enough that few people have considered them before, and at the same time have powerful consequences. Ideas like that usually don’t pop into the heads of random, uninformed strangers.
Depends on your definition of “Dangerous.” I’ve come across quite a few ideas that tend to do -severe- damage to the happiness of at least a subset of those aware of them. Some of them are about the universe; things like entropy. Others are social ideas, which I won’t give an example of.
Daniel Dennett wrote a book called “Darwin’s Dangerous Idea”, and when people aren’t trying to play down the basilisk (i.e. almost everywhere), people often pride themselves on thinking dangerous thoughts. It’s a staple theme of the NRxers and the manosphere. Claiming to be dangerous provides a comfortable universal argument against opponents.
I think there are, in fact, a good many dangerous ideas, not merely ideas claimed to be so by posturers. Off the top of my head:
Islamic fundamentalism (see IS/ISIS/ISIL).
The mental is physical.
God.
There is no supernatural.
Utilitarianism.
Superintelligent AI.
How to make nuclear weapons.
Atoms.
They do, all the time, by contagion from the few who come up with them, especially in the Internet age.
There are some things which could be highly dangerous which are protected almost purely by thick layers of tedium.
Want to make nerve gas? well if you can wade through a thick pile of biochemistry textbooks the information isn’t kept all that secret.
Want to create horribly deadly viruses? ditto.
The more I learned about physics, chemistry and biology the more I’ve become certain that the main reason that major cities have living populations is that most of the people with really deep understanding don’t actually want to watch the world burn.
You often find that extremely knowledgeable people don’t exactly hide knowledge but do put it on page 425 of volume 3 of their textbook, written in language which you need to have read the rest to understand. Which protects it effectively from 99.99% of the people who might use it to intentionally harm others.
Argument against: back when cities were more flamable, people didn’t set them on fire for the hell of it.
On the other hand, it’s a lot easier to use a timer and survive these days, should you happen to not be suicidal.
“I want to see the world burn” is a great line of dialogue, but I’m not convinced it’s a real human motivation. Um, except that when I was a kid, I remember wishing that this world was a dream, and I’d wake up. Does that count?
Second thought—when I was a kid, I didn’t have a method in mind. What if I do serious work with lucid dreaming techniques when I’m awake? I don’t think the odds of waking up into being a greater intelligence are terribly good, nor is there a guarantee that my live would be better. On the other hand, would you hallucinations be interested in begging me to not try it?
Based on personal experience, if you’re dreaming I don’t recommend trying to wake yourself up. Instead, enjoy your dream until you’re ready to wake up naturally. That way you’ll have far better sleep.
Based on personal experience, I would have agreed with you, right up until last year, when I found myself in the rather terrifying position of being mentally aroused by a huge crash in my house, but unable to wake up all the way for several seconds afterward, during which my sleeping mind refused to reject the “something just blew a hole in the building we’re under attack!” hypothesis.
(It was an overfilled bag falling off the wall.)
But absent actual difficulty waking for potential emergencies, sure; hang out in Tel’aran’rhiod until you get bored.
Sorry, should have defined dangerous ideas better—I only meant information that would cause a rational person to drastically alter their behavior, and which would be much worse for society as a whole when everyone is told at once about it.
I hope they’re as hard to come by as you think they are.
Alternatively, Roko could be part of the 1% of people who think of a dangerous idea (assuming his basilisk is dangerous) and spread it on the internet without second guessing themselves. Are there 99 other people who thought of dangerous ideas and chose not to spread them for our 1 Roko?