It doesn’t matter what IQ they have or how rational they were in 2005
This is a reference to Eliezer, right? I really don’t understand why he’s on Twitter so much. I find it quite sad to see one of my heroes slipping into the ragebait Twitter attractor.
Only inasmuch he’s a proof-by-example. By that I mean he’s one of the most earnest/truthseeking users I found when I was still using the platform, and yet he still manages to retweet things outside his domain of expertise that are either extraordinarily misleading or literally, factually incorrect—and I think if you sat him down and prompted him to think about the individual cases he would probably notice why, he just doesn’t because the platform isn’t conducive to that kind of deliberate thought.
I recall a rationalist I know chiding Eliezer for his bad tweeting, and then Eliezer asked him to show him an example of a recent tweet that was bad, and then the rationalist failed to find anything especially bad.
Perhaps this has changed in the 2-3 years since that event. But I’d be interested in an example of a tweet you (lc) thought was bad.
It’s not the tweets, it’s the retweets. People’s tweets on Twitter are usually not that bad. Their retweets, and, for slightly crazier people, their quote tweets are what contain the bizarre mischaracterizations, because they’re the pulls from the top of the attention-seeking crab bucket.
I run a company that sells security software to large enterprises. I remember seeing this (since deleted) post Eliezer retweeted last year during the Crowdstrike blue screen incident, and thinking: “Am I crazy? What on earth is this guy talking about?”
The audit requirements Mark is talking about don’t exist. He just completely made them up. ChatGPT’s explanation here is correct; even if you’re selling to the federal government[1], there’s no “fast track” for big names like Crowdstrike. At absolute maximum your auditor is going to ask for evidence that you use some IDS solution, and you’ll have to gather the same evidence no matter what solution you’re using.
Now, Yudkowsky is not a mendacious person, and he isn’t going to pump misinfo into the ether himself. But naturally if anybody goes on Twitter long enough they’re gonna see stuff like this, and it will just feel plausible to you. It will pass whatever cogsec antimalware blacklists & heuristics you’ve developed for assessing the credibility of things on the internet.
Probably because, like, if you overheard this kind of thing at a party, it would be credible! It’s only on this platform, where people are literally stepping over one another to concoct absurd lies for attention, where people are additionally incentivized to present as having personal expertise in the lie to go slightly more viral, and then an algorithm is selectively boosting the people that do that well enough and effectively enough to the top of your feed, that you encounter this nonsense. And then it goes into your world model, and the next time you see someone claim some crazy thing about how the food industry is in cahoots with Big Chicken you’re more likely to believe it, etc. etc.
And the vast majority of software companies, to be clear, don’t have to do anything like FedRAMP. The largest and most ubiquitous compliance frameworks, like SOC2 or ISO 27001, are self-imposed standards maintained by nonprofits like the AICPA and have nothing to do with the government.
Appreciate the example. I remember reading that retweet!
At the time it sounded plausible to me, and I assumed it was accurate about certain industries.
I’m interested in understanding a bit more what’s going on here. Are we sure you’re talking about the same kinds of companies? I’d guess you’re dealing with companies in the range of 2k-20k employees, and I think Crowdstrike was substantially affecting companies in the range of 20k-200k employees (or at least that’s what I thought of when I saw this tweet), where I imagine auditors have to use much more broad-brush tools to do auditing.
The sorts of companies I imagine as having this kind of broad-strokes audit are extremely broad service industries – airlines, trains, grocery stores, banks, hospitals – where my impression is they often use very old software and buggy hardware due to their overwhelming size and sloth, and where I suspect that a lot of decisions get made by the minimum possible thing required to meet some formal requirements.
The purpose of these audits is not generally to verify with certainty that you’re doing everything you say. That would be very hard, maybe impossible. Mostly you fill out a form saying you’re doing X. If it turns out later after a breach you weren’t doing the stuff you claimed to do during the audit, you’re sued.
We’re doing a PoC right now for a company with >400,000 employees. I am not their security team, and we haven’t sold yet, but on our end everything we’ve run into is normal procurement BS. The main thing that happens as you start to sell to larger customers is that you have to fill out a lot of forms saying your product does not use slave labor and such.
The “you’re sued” part is part of what ensures that the forms get filled out honestly and comprehensively.
Depending on the kind of audit you do, the actual deliverable you give your auditor may just be a spreadsheet with a bunch of Y/N answers to hundreds of questions like “Do all workstations have endpoint protection software”, “Do all servers have intrusion detection software”, etc. with screenshots of dashboards as supporting evidence for some of them.
But regardless of how much evidence an external auditor asks for, at large companies doing important audits, every single thing you say to the auditor will be backed internally with supporting evidence and justification for each answer you give.
At a bank you might have an “internal audit” department that has lots of meetings and back-and-forth with your IT department; at an airline it might be a consulting firm that you bring in to modernize your IT and help you handle the audit, or, depending on your relationship with your auditor and the nature of the audit, it might be someone from the audit firm itself that is advising you. In each case, their purpose is to make sure that every machine across your firm really does have correctly configured EDR, fully up to date security patches, firewalled, etc. before you claim that officially to an auditor.
Maybe you have some random box used to show news headlines on TVs in the hallways—turns out these are technically in-scope for having EDR and all sorts of other endpoint controls, but they’re not compatible with or not correctly configured to run Microsoft Defender, or something. Your IT department will say that there are various compensating / mitigating controls or justifications for why they’re out of scope, e.g. the firewall blocks all network access except the one website they need to show the news, the hardware itself is in a locked IT closet, they don’t even have a mouse / keyboard plugged in, etc. These justifications will usually be accepted unless you get a real stickler (or have an obstinate “internal auditor”). But it’s a lot easier to just say “they all run CrowdStrike” than it is to keep track of all these rationales and compensating controls, and indeed ease-of-deployment is literally the first bullet in CrowdStrike’s marketing vs. Microsoft Defender:
CrowdStrike: Deploy instantly with a single, lightweight agent — no OS prerequisites, complex configuration, or fine tuning required. Microsoft: Complicated deployment hinders security. All endpoints require the premium edition of the latest version of Windows, requiring upfront OS and hardware upgrades for full security functionality.
You wrote in a sibling reply:
Further, the larger implication of the above tweet is that companies use Crowdstrike because of regulatory failure, and this is also simply untrue. There are lots of reasons people sort of unthinkingly go with the name brand option in security, but that’s a normal enterprise software thing and not anything specific to compliance.
I agree that this has little to do with “regulatory failure” and don’t know / don’t have an opinion on whether that’s what the original tweet author was actually trying to communicate. But my point is that firms absolutely do make purchasing decisions about security software for compliance reasons, and a selling point of CrowdStrike (and Carbon Black, and SentinelOne) is that they make 100% compliance easier to achieve and demonstrate vs. alternative solutions. That’s not a regulatory failure or even necessarily problematic, but it does result in somewhat different outcomes compared to a decision process of “unthinkingly going with the name brand option” or “carefully evaluate and consider only which solutions provide the best actual security vs. which are theater”.
The audit requirements Mark is talking about don’t exist. He just completely made them up.
The screenshotted tweet says that you’re required to install something like Crowdstrike, which is correct and also seems consistent with the ChatGPT dialogue you linked?
There are long lists of computer security practices and procedures needed to pass an audit for compliance with a standard like ISO27001, PCI DSS, SOC 2, etc. that many firms large and small are subject to (sometimes but not necessarily by law—e.g. companies often need to pass an SOC 2 audit because their customers ask for it).
As you say, none of these standards name specific software or vendors that you have to use in order to satisfy an auditor, but it’s often much less of a headache to use a “best in class” off-the-shelf product (like CrowdStrike) that is marketed specifically as satisfying specific requirements in these standards, vs. trying to cobble together a complete compliance posture using tools or products that were not designed specifically to satisfy those requirements.
A big part of the marketing for a product like CrowdStrike is that it has specific features which precisely and unambiguously satisfy more items in various auditor checklists than competitors.
So “opens up an expensive new chapter of his book” is colorful and somewhat exaggerated, but I wouldn’t describe it as “misinformation”—it’s definitely pointing at something real, which is that a lot of enterprise security software is sold and bought as an exercise in checking off specific checklist items in various kinds of audits, and how easy / convenient / comprehensive a solution makes box-checking is often a bigger selling point than how much actual security it provides, or what the end user experience is actually like.
There is no such thing as directionally correct. The tweet says “If you use Crowdstrike, your auditor checks a single line and moves on. If you use anything else, your auditor opens up an expensive new Chapter of his book.” This is literally and unambiguously false. No security/compliance standard—public or private—requires additional labor or verification procedures for non-Crowdstrike EDR alternatives. The steps to pass an audit are the same no matter what solution you’re using for EDR. Further, as a vendor, there is a very low ceiling for supporting most of the evidence collection required for any of the standards you cite, even at large scale. Allowing your users to collect such evidence (often, just screenshotting a stats page) is not nearly the largest barrier to entry for new incumbents in Crowdstrike’s space.
Further, the larger implication of the above tweet is that companies use Crowdstrike because of regulatory failure, and this is also simply untrue. There are lots of reasons people sort of unthinkingly go with the name brand option in security, but that’s a normal enterprise software thing and not anything specific to compliance.
This is a reference to Eliezer, right? I really don’t understand why he’s on Twitter so much. I find it quite sad to see one of my heroes slipping into the ragebait Twitter attractor.
Only inasmuch he’s a proof-by-example. By that I mean he’s one of the most earnest/truthseeking users I found when I was still using the platform, and yet he still manages to retweet things outside his domain of expertise that are either extraordinarily misleading or literally, factually incorrect—and I think if you sat him down and prompted him to think about the individual cases he would probably notice why, he just doesn’t because the platform isn’t conducive to that kind of deliberate thought.
I recall a rationalist I know chiding Eliezer for his bad tweeting, and then Eliezer asked him to show him an example of a recent tweet that was bad, and then the rationalist failed to find anything especially bad.
Perhaps this has changed in the 2-3 years since that event. But I’d be interested in an example of a tweet you (lc) thought was bad.
It’s not the tweets, it’s the retweets. People’s tweets on Twitter are usually not that bad. Their retweets, and, for slightly crazier people, their quote tweets are what contain the bizarre mischaracterizations, because they’re the pulls from the top of the attention-seeking crab bucket.
I run a company that sells security software to large enterprises. I remember seeing this (since deleted) post Eliezer retweeted last year during the Crowdstrike blue screen incident, and thinking: “Am I crazy? What on earth is this guy talking about?”
The audit requirements Mark is talking about don’t exist. He just completely made them up. ChatGPT’s explanation here is correct; even if you’re selling to the federal government[1], there’s no “fast track” for big names like Crowdstrike. At absolute maximum your auditor is going to ask for evidence that you use some IDS solution, and you’ll have to gather the same evidence no matter what solution you’re using.
Now, Yudkowsky is not a mendacious person, and he isn’t going to pump misinfo into the ether himself. But naturally if anybody goes on Twitter long enough they’re gonna see stuff like this, and it will just feel plausible to you. It will pass whatever cogsec antimalware blacklists & heuristics you’ve developed for assessing the credibility of things on the internet.
Probably because, like, if you overheard this kind of thing at a party, it would be credible! It’s only on this platform, where people are literally stepping over one another to concoct absurd lies for attention, where people are additionally incentivized to present as having personal expertise in the lie to go slightly more viral, and then an algorithm is selectively boosting the people that do that well enough and effectively enough to the top of your feed, that you encounter this nonsense. And then it goes into your world model, and the next time you see someone claim some crazy thing about how the food industry is in cahoots with Big Chicken you’re more likely to believe it, etc. etc.
And the vast majority of software companies, to be clear, don’t have to do anything like FedRAMP. The largest and most ubiquitous compliance frameworks, like SOC2 or ISO 27001, are self-imposed standards maintained by nonprofits like the AICPA and have nothing to do with the government.
Appreciate the example. I remember reading that retweet!
At the time it sounded plausible to me, and I assumed it was accurate about certain industries.
I’m interested in understanding a bit more what’s going on here. Are we sure you’re talking about the same kinds of companies? I’d guess you’re dealing with companies in the range of 2k-20k employees, and I think Crowdstrike was substantially affecting companies in the range of 20k-200k employees (or at least that’s what I thought of when I saw this tweet), where I imagine auditors have to use much more broad-brush tools to do auditing.
The sorts of companies I imagine as having this kind of broad-strokes audit are extremely broad service industries – airlines, trains, grocery stores, banks, hospitals – where my impression is they often use very old software and buggy hardware due to their overwhelming size and sloth, and where I suspect that a lot of decisions get made by the minimum possible thing required to meet some formal requirements.
The purpose of these audits is not generally to verify with certainty that you’re doing everything you say. That would be very hard, maybe impossible. Mostly you fill out a form saying you’re doing X. If it turns out later after a breach you weren’t doing the stuff you claimed to do during the audit, you’re sued.
We’re doing a PoC right now for a company with >400,000 employees. I am not their security team, and we haven’t sold yet, but on our end everything we’ve run into is normal procurement BS. The main thing that happens as you start to sell to larger customers is that you have to fill out a lot of forms saying your product does not use slave labor and such.
The “you’re sued” part is part of what ensures that the forms get filled out honestly and comprehensively.
Depending on the kind of audit you do, the actual deliverable you give your auditor may just be a spreadsheet with a bunch of Y/N answers to hundreds of questions like “Do all workstations have endpoint protection software”, “Do all servers have intrusion detection software”, etc. with screenshots of dashboards as supporting evidence for some of them.
But regardless of how much evidence an external auditor asks for, at large companies doing important audits, every single thing you say to the auditor will be backed internally with supporting evidence and justification for each answer you give.
At a bank you might have an “internal audit” department that has lots of meetings and back-and-forth with your IT department; at an airline it might be a consulting firm that you bring in to modernize your IT and help you handle the audit, or, depending on your relationship with your auditor and the nature of the audit, it might be someone from the audit firm itself that is advising you. In each case, their purpose is to make sure that every machine across your firm really does have correctly configured EDR, fully up to date security patches, firewalled, etc. before you claim that officially to an auditor.
Maybe you have some random box used to show news headlines on TVs in the hallways—turns out these are technically in-scope for having EDR and all sorts of other endpoint controls, but they’re not compatible with or not correctly configured to run Microsoft Defender, or something. Your IT department will say that there are various compensating / mitigating controls or justifications for why they’re out of scope, e.g. the firewall blocks all network access except the one website they need to show the news, the hardware itself is in a locked IT closet, they don’t even have a mouse / keyboard plugged in, etc. These justifications will usually be accepted unless you get a real stickler (or have an obstinate “internal auditor”). But it’s a lot easier to just say “they all run CrowdStrike” than it is to keep track of all these rationales and compensating controls, and indeed ease-of-deployment is literally the first bullet in CrowdStrike’s marketing vs. Microsoft Defender:
You wrote in a sibling reply:
I agree that this has little to do with “regulatory failure” and don’t know / don’t have an opinion on whether that’s what the original tweet author was actually trying to communicate. But my point is that firms absolutely do make purchasing decisions about security software for compliance reasons, and a selling point of CrowdStrike (and Carbon Black, and SentinelOne) is that they make 100% compliance easier to achieve and demonstrate vs. alternative solutions. That’s not a regulatory failure or even necessarily problematic, but it does result in somewhat different outcomes compared to a decision process of “unthinkingly going with the name brand option” or “carefully evaluate and consider only which solutions provide the best actual security vs. which are theater”.
The screenshotted tweet says that you’re required to install something like Crowdstrike, which is correct and also seems consistent with the ChatGPT dialogue you linked?
There are long lists of computer security practices and procedures needed to pass an audit for compliance with a standard like ISO27001, PCI DSS, SOC 2, etc. that many firms large and small are subject to (sometimes but not necessarily by law—e.g. companies often need to pass an SOC 2 audit because their customers ask for it).
As you say, none of these standards name specific software or vendors that you have to use in order to satisfy an auditor, but it’s often much less of a headache to use a “best in class” off-the-shelf product (like CrowdStrike) that is marketed specifically as satisfying specific requirements in these standards, vs. trying to cobble together a complete compliance posture using tools or products that were not designed specifically to satisfy those requirements.
A big part of the marketing for a product like CrowdStrike is that it has specific features which precisely and unambiguously satisfy more items in various auditor checklists than competitors.
So “opens up an expensive new chapter of his book” is colorful and somewhat exaggerated, but I wouldn’t describe it as “misinformation”—it’s definitely pointing at something real, which is that a lot of enterprise security software is sold and bought as an exercise in checking off specific checklist items in various kinds of audits, and how easy / convenient / comprehensive a solution makes box-checking is often a bigger selling point than how much actual security it provides, or what the end user experience is actually like.
There is no such thing as directionally correct. The tweet says “If you use Crowdstrike, your auditor checks a single line and moves on. If you use anything else, your auditor opens up an expensive new Chapter of his book.” This is literally and unambiguously false. No security/compliance standard—public or private—requires additional labor or verification procedures for non-Crowdstrike EDR alternatives. The steps to pass an audit are the same no matter what solution you’re using for EDR. Further, as a vendor, there is a very low ceiling for supporting most of the evidence collection required for any of the standards you cite, even at large scale. Allowing your users to collect such evidence (often, just screenshotting a stats page) is not nearly the largest barrier to entry for new incumbents in Crowdstrike’s space.
Further, the larger implication of the above tweet is that companies use Crowdstrike because of regulatory failure, and this is also simply untrue. There are lots of reasons people sort of unthinkingly go with the name brand option in security, but that’s a normal enterprise software thing and not anything specific to compliance.