When I thought about it, I realized this seemed very similar to a standard hack used on people that we already rely on computers to defend us against. To be specific, it follows an incredibly similar framework to one of those Lottery/Nigerian 419 Scam emails.
Opening Narrative: Attempt to establish some level of trust and believability. Things with details tend to be more believable than things without details, although the conjunction fallacy can be tricky here. Present the target with two choices: (Hope they don’t realize it’s a false dichotomy)
Choice A: Send in a small amount of utility. (If Choice A is selected, repeat False dichotomy)
Choice B: Allow a (fake) opportunity to acquire a large amount of (fake) utility to slip away.
Background: Make a MASSIVE number of attempts. even if the first 10,000 attempts fail, the cost of trying to hack is minimal compared to the rewards.
So to reduce it to a simpler problem, the first question seems to be, how do we create the best known spam filter we have right now?
And then the second question seems to be “How can we make a spam filter MUCH better than that?” to protect our Lovable Senile Billionaire Grandpa who has Nuclear Weapons and a tendency to believe everything, and who relies on emails for critical world altering decisions so a single false positive or false negative means terrible costs levels of epic spam filtering.
So to attempt to help that, I’ll try to list all of the Antispam tactics I can find, to at least help with the first part.
Training Phase: The Training Phase for a Bayesian Spam Filter which needs to go live in a super hostile environment should be as long and thorough as it possibly can. You know how some places use validation and some use bounty testing? We should use both.
Sysadmin: Multiple someones need to begin by reviewing everything. Then, they need to continue by reviewing everything. There’s a built in Human tendency to ignore risks after you’ve been dealing with them for a while and nothing has happened. I don’t know what it’s called exactly, but presumably there are countermeasures in place at top secret type facilities that need extremely vigilant security guards at all times. We need to begin by doing those, and then again, validate and offer bounties to hackers while the system isn’t live.
Secrecy: Many explicit, open list of countermeasures can generally be planned around by a determined hacker. Hackers can’t plan for security measures they aren’t aware of. The secrecy ALSO needs to be validated and bounty tested.
At first, this does sound a bit contradictory (How do you do a open source test of “Secrecy”? But you’d want to do that first, before say, having the FAI develop it’s own spam filtering that the public shouldn’t know. Google has this problem sometimes where they battle with Search Engine Optimizers who are trying to fake having genuine good content when they are, in fact, irrelevant and trying to sell you on lies to make money (Much like our Muggers, really). We need to find out how additional ways to fight spamdexing as well. This has another good Wikipedia page: http://en.wikipedia.org/wiki/Spamdexing
The Bounty system is important because we want to take advantage of temporal discounting. People will frequently take small amounts of utility now over large amounts of utility later even to irrational levels, so we need to offer bribes so that the kinds of people who might try to trick the FAI later come to trick it during development while we would still be actively fixing problems and it didn’t have massive responsibilities.
From my personal experience coding, another good way to make sure your code is developed well enough to withstand all sorts of attacks and problems is to have an incredibly robust set of test data. Problems that can’t be seen with a single version and 50 records and 2 users often pop up across multiple versions and 1 million records and 70 users. So that as well, but more so.
A lot of this may be common knowledge already, but I thought listing everything I knew would be a good starting point for additional security measures.
When I thought about it, I realized this seemed very similar to a standard hack used on people that we already rely on computers to defend us against. To be specific, it follows an incredibly similar framework to one of those Lottery/Nigerian 419 Scam emails.
Opening Narrative: Attempt to establish some level of trust and believability. Things with details tend to be more believable than things without details, although the conjunction fallacy can be tricky here. Present the target with two choices: (Hope they don’t realize it’s a false dichotomy)
Choice A: Send in a small amount of utility. (If Choice A is selected, repeat False dichotomy) Choice B: Allow a (fake) opportunity to acquire a large amount of (fake) utility to slip away.
Background: Make a MASSIVE number of attempts. even if the first 10,000 attempts fail, the cost of trying to hack is minimal compared to the rewards.
So to reduce it to a simpler problem, the first question seems to be, how do we create the best known spam filter we have right now?
And then the second question seems to be “How can we make a spam filter MUCH better than that?” to protect our Lovable Senile Billionaire Grandpa who has Nuclear Weapons and a tendency to believe everything, and who relies on emails for critical world altering decisions so a single false positive or false negative means terrible costs levels of epic spam filtering.
So to attempt to help that, I’ll try to list all of the Antispam tactics I can find, to at least help with the first part.
Bayesian Spam filtering: I was going to try to summarize this, but honestly, the Wikipedia article does a better job then I can: http://en.wikipedia.org/wiki/Bayesian_spam_filtering
Training Phase: The Training Phase for a Bayesian Spam Filter which needs to go live in a super hostile environment should be as long and thorough as it possibly can. You know how some places use validation and some use bounty testing? We should use both.
Sysadmin: Multiple someones need to begin by reviewing everything. Then, they need to continue by reviewing everything. There’s a built in Human tendency to ignore risks after you’ve been dealing with them for a while and nothing has happened. I don’t know what it’s called exactly, but presumably there are countermeasures in place at top secret type facilities that need extremely vigilant security guards at all times. We need to begin by doing those, and then again, validate and offer bounties to hackers while the system isn’t live.
Secrecy: Many explicit, open list of countermeasures can generally be planned around by a determined hacker. Hackers can’t plan for security measures they aren’t aware of. The secrecy ALSO needs to be validated and bounty tested.
At first, this does sound a bit contradictory (How do you do a open source test of “Secrecy”? But you’d want to do that first, before say, having the FAI develop it’s own spam filtering that the public shouldn’t know. Google has this problem sometimes where they battle with Search Engine Optimizers who are trying to fake having genuine good content when they are, in fact, irrelevant and trying to sell you on lies to make money (Much like our Muggers, really). We need to find out how additional ways to fight spamdexing as well. This has another good Wikipedia page: http://en.wikipedia.org/wiki/Spamdexing
The Bounty system is important because we want to take advantage of temporal discounting. People will frequently take small amounts of utility now over large amounts of utility later even to irrational levels, so we need to offer bribes so that the kinds of people who might try to trick the FAI later come to trick it during development while we would still be actively fixing problems and it didn’t have massive responsibilities.
From my personal experience coding, another good way to make sure your code is developed well enough to withstand all sorts of attacks and problems is to have an incredibly robust set of test data. Problems that can’t be seen with a single version and 50 records and 2 users often pop up across multiple versions and 1 million records and 70 users. So that as well, but more so.
A lot of this may be common knowledge already, but I thought listing everything I knew would be a good starting point for additional security measures.