We should encourage thinking about the intent (incoming) and expected effect (outgoing) of truisms, rather than their literal meaning. If either of the above injunctions actually doesn’t apply to you, you’ll know it.
Yes, this is my concern too. However, anyone who posts to a newsgroup saying “I’m about to write my own cipher, any advice” should not do it. The post indicated someone who planned to actually start writing code; that’s a definite sign that they shouldn’t do it.
Even though it’s unlikely to work, it is still the approach which minimizes risk; even a small reduction in their probability of going ahead will likely be a bigger effect than any other safety advice you can give, and any other advice will act against its efficacy.
Entertainingly, he’s entering the field from mathematical cryptography; so “don’t design your own cipher” is precisely the wrong analogy to use here :-)
It used to be the domain of the linguists… But you’re correct; nowadays, I’m using mathematical cryptography as a short hand for “y’know, like, real cryptography, not just messing around with symbols to impress you friends”.
If you think you have an AI that might improve itself and act on the real world, don’t run it.
Strike “and act on the real world”—all AIs act on the real world.
I mean, act on the real world in a way more significant than your typical chess-playing program.
This rules out FAI.
Sure, this is advice along the lines of “don’t design your own cipher”.
Only more so.
In general wise, but in this case we need a cipher, don’t have any, and will probably be handed a bad one in the future.
Our truisms need to be advice we would want everyone to follow.
We should encourage thinking about the intent (incoming) and expected effect (outgoing) of truisms, rather than their literal meaning. If either of the above injunctions actually doesn’t apply to you, you’ll know it.
My concern is you’ll also ‘know’ it doesn’t apply to you when it does. People write ciphers all the time.
Yes, this is my concern too. However, anyone who posts to a newsgroup saying “I’m about to write my own cipher, any advice” should not do it. The post indicated someone who planned to actually start writing code; that’s a definite sign that they shouldn’t do it.
See the addendum above; “don’t do it” isn’t likely to work.
Even though it’s unlikely to work, it is still the approach which minimizes risk; even a small reduction in their probability of going ahead will likely be a bigger effect than any other safety advice you can give, and any other advice will act against its efficacy.
“Then they are fools and nothing can be done about it.” In any case, this seems to be the opposite of the concern you were citing before.
If we use truisms that everyone knows have to be ignored by someone, It becomes easier to think they can be ignored by oneself.
I reread the thread, leaning towards your position now.
Entertainingly, he’s entering the field from mathematical cryptography; so “don’t design your own cipher” is precisely the wrong analogy to use here :-)
“mathematical cryptography”? What other sort of cryptography is there?
It used to be the domain of the linguists… But you’re correct; nowadays, I’m using mathematical cryptography as a short hand for “y’know, like, real cryptography, not just messing around with symbols to impress you friends”.
Ah, OK!
It’s possible in that case that I may actually know your friend, if they happened to touch on some of the same parts of the field as me.
No extra clues :-)