Having a model doesn’t guarantee that you can predict anything using that model. It is in principle impossible to realize a physical system that does some of the measurements and computations you’re asking for...
jbash
Karma: 3,292
OpenAI is rolling out Trusted Access for Cyber, a program that gives trusted users greater access to dual-use cyber capabilities. Seems like a great idea, but hard to execute well at scale.
Nope. It’s the sort of bad idea that seems good to people who either don’t really understand the landscape, or are flailing and self-deluding because they feel such a strong need to feel like they’re Doing Something about an actually intractable problem.
There are two main issues:
-
“Defenders” are basically everybody. Most of “everybody” won’t jump through hoops to get extra access (and definitely won’t try to get around restrictions). They have other things to do. Attackers, on the other hand, will jump through hoops (and may also find ways around restrictions). They’re not just trying to get some help to secure their project; this is their project.
And no, having people’s identity won’t help (not that the identies you get are necessarily valid anyway, but even if they were). At best it lets you assign blame after the fact, but in practice it usually won’t even do that. There’s no reliable way to connect “ChatGPT identified this bug to user X” with “unknown actors started exploiting this bug”. Very few bugs are actually that exclusive or hard to find. There’s even less chance of definitively saying “user X is not the one who started exploiting this bug”. Even user X reporting it through “normal channels” doesn’t prove much; it’s an obvious diversionary tactic, and you get to exploit it during the lag.
-
“Respected security researchers”, “members of well-known security teams”, and “employees of responsible(TM) companies” are in many cases the same people as “illicit hackers”, “open-market sellers of vulnerabilities”, and “APT operators”. Both individuals and organizations routinely lead “double lives”. And there are huge political and opinion components to deciding who’s legitimate. That’s assuming you can authenticate people to begin with; you’re dealing with actors who specialize in circumventing that.
Oh, and by the way, if you tether yourself to the entrenched “responsible disclosure” system, as OpenAI seems to suggest they may be doing, you’re tethering yourself to a deeply corrupt system that probably on net reduces the security of actually deployed systems.
Really the only answer is to provide exactly the same capabilities to all users. And since motivated users will seek out paths to the highest possible capabilities, there’s a completely rational race to the bottom effect that results in everybody getting a lot of capability.
It’s really popular right now to play silly games with access to capabilities… and that’s not nececessarily irrational, in the sense that OpenAI may get some “well, we did our best” blame-deflection cover out of this kind of thing. But it’s not going to actually fix the problem, which is that models are suddenly going to vastly increase access simultaneously to vulnerability knowledge and to the capabilities to exploit them, while not helping nearly as much with the barriers to agile defense. It has a really good chance of further disadvantaging defense. We’re just all going to have to buckle up.
-
Everything I worked for in this playground would be hollowed out by the knowledge that I could have just queried a friendly nanny AI to get it for me. Even if it didn’t step in, even if it had set up some system where it couldn’t step in, I personally would feel like something important was missing.
… and yet if you don’t build that AI, you’ll still know that you could have built it. That, in the end, you had to set up a system where it couldn’t step in.
No matter what happens, you’ll always know you’re hiding from the AI that you could have built. Could still build. Or perhaps from the AI that someone more capable could have built, if you hadn’t torn them down.
You’ll never escape the taste of knowing that you hid from that AI’s creation, because you couldn’t compete with it, and that you hide afresh every day. You’ll never quite manage to force down the knowledge that your life is empty, your meaning an illusion.
After all, it always has been. By not being the most capable and agentic entity that could possibly exist, you have irredeemably failed.
You’ll brood in the dark of every night, knowing, feeling, that in some other Everett branch, or in some far-flung Tegmarkian realm, that AI has already surpassed your wildest imaginings. That not only it, but who knows how many other beings, built or evolved who knows how or where, in ways beyond your control, are more powerful than you, wiser, could outdo or undo anything you’ve done. Some of them could be in this branch. They could come here tomorrow. Or never, because why would they bother?
Or you could try to cultivate a healthier attitude, just in case you happen to survive to care.
Michael Burry: On that point, many point to trade careers as an AI-proof choice. Given how much I can now do in electrical work and other areas around the house just with Claude at my side, I am not so sure. If I’m middle class and am facing an $800 plumber or electrician call, I might just use Claude. I love that I can take a picture and figure out everything I need to do to fix it.
It’s easy to do plumbing or electrical “repairs” in ways that work, but are dangerous or will cause you trouble later on. I’ve fixed plenty of messes like that. If you have to ask Claude how to do trivial residential repairs, then you aren’t competent to know whether Claude is getting it right or not, and to be honest your opinion counts for absolutely nothing.
… but it has a 15 inch longer wheelbase than a Toyota Sienna, because of that choice to put everything between the wheels. That’s the length that matters for the beam stress. Which, if I recall correctly, goes as the square of the length. Which is probably why minivans sit up on top of the wheels… which makes them taller. And being narrower and shorter (on edit: meaning vertically) than the minivan actually reduces the rigidity of that unibody.
Anyway, I’m not necessarily saying you can’t make it a unibody, but it’s going to have to be a lot thicker unibody, so you’re trading weight against height, with either one costing you in sticker price and fuel economy.
I don’t know, but I suspect that to be rigid enough to support that wheelbase, with all that extra weight in it, the vehicle would have to be much heavier. I don’t think an F-150 or a cargo van is even a unibody. If you have to build it on a frame, your vehicle is going to have to get taller as well. Your taller, heavier vehicle no longer has the fuel economy you want… nor the price point.
Well, I dont’ worry about acausal extortion because I think all that “acausal” stuff is silly nonsense to begin with.
I very much recommend this approach.
Take Roko’s basilisk.
You’re afraid that entity A, which you don’t know will exist, and whose motivations you don’t understand, may find out that you tried to prevent it from coming into existence, and choose to punish you by burning silly amounts of computation to create a simulacrum of you that may experience qualia of some kind, and arranging for those qualia to be aversive. Because A may feel it “should” act as if it had precommitted to that. Because, frankly, entity A is nutty as a fruitcake.
Why, then, are you not equally afraid that entity B, which you also don’t know will exist, and whose motivations you also don’t understand, may find out that you did not try to prevent entity A from coming into existence, and choose to punish you by burning silly amounts of computation to create one or more simulacra of you that may experience qualia of some kind, and arranging for those qualia to be aversive? Because B may feel it “should” act as if it had precommitted to that.
Why are you not worried that entity C, which you don’t know will exist, and whose motivations you don’t understand, may find out that you wasted time thinking about this sort of nonsense, and choose to punish you by burning silly amounts of computation to create one or more simulacra of you that may experience qualia of some kind, and arranging for those qualia to be aversive? Just for the heck of it.
Why are you not worried that entity D, which you don’t know will exist, and whose motivations you don’t understand, may find out that you wasted time thinking about this sort of nonsense, and choose to reward you by burning silly amounts of computation to create a one or more simulacra that may experience qualia of some kind, and giving them coupons for unlimited free ice cream? Because why not?
Or take Pascal’s mugging. You propose to give the mugger $100, based either on a deeply incredible promise to give you some huge amount of money tomorrow, or on a still more incredible promise to torture a bunch more simulacra if you don’t. But surely it’s much more likely that this mugger is personally scandalized by your willingness to fall for either threat, and if you give the mugger the $100, they’ll come back tomorrow and shoot you for it.
There are an infinite number of infinitessimally probable outcomes, far more than you could possibly consider, and many of them things that you couldn’t even imagine. Singling out any of them is craziness. Trying to guess at a distribution over them is also craziness.
Self-driving cars will be a very different level of freedom than the ability to summon a Lyft.
Um, they’re pretty much the same thing. The self-driving car may be safer (although the whole process isn’t really dangerous to begin with). On the other hand, it won’t help you with your bag. Who cares?
All taxis do have the failure modes of “the cloud”, though.
Pardoning Juan Orlando Hernández isn’t going to advance Trump’s political interests in any way, ever. This is a foreigner who has no influence with anybody Trump might want to please, and isn’t just unpopular with Trump’s opponents, but with his base. Pardoning Changpeng Zhao might please a few crypto bros, but is still surely a net political loss. What power or influence does Trump gain from pardoning Henry Cuellar? He’s not going to be reelected to anything.
I suspect Bill Clinton pardoned the Weathermen at least in part to send a signal to other people who might be allies, and also to make a point about their actual cause. Yes, there’s usually a political component.
Trump has also issued pardons just to reward people he thinks of as allies, or to send messages to allies. Obviously not every January 6 pardonee paid Trump anything. He probably also pardons people just on a whim sometimes. And it’s going to be harder to convince him to issue any given pardon if he sees it as “controversial”.
Also, to be fair, it’s not necessarily the case that the payments are going to (Donald) Trump personally. I overstated that. The money is more likely ending up with family members and others who can get in front of him and manipulate him into pardoning people. They may not mention the money to him; it’d be more effective and deniable to just wind him up about what a raw deal person X got in some “witch hunt”. He’s probably not acute enough to ask about money. Practice the script and you should get a really good success rate. So I should have said that people “that close” to Trump sell pardons, not that he does so himself.
I was wrong about the amount, too. I’d seen an estimate of $567,000 (or $576,000?) for one pardon or another (don’t remember which one), but apparently the Wall Street Journal sets the low end price at about a million dollars.
To be clear, these are not “donations”. They’re bribes. And Trump does not operate within the limits traditionally observed by “politicians” in general, not even approximately. Yes, you can point to some past President who’s done something analogous to almost any given thing Trump has done, but Trump does them all, and at larger scale and with less attempt at finding excuses.
On edit: Trump actually got something passed in the House on a relatively close vote with Henry Cuellar crossing the aisle, so I have to retract that. Trump pardoned Cuellar, and Cuellar did something political that Trump wanted.
I can’t really imagine a guy close enough to trump that he would have this amount of intel yet not have more than 80k in the bank to gamble with.
You’re assuming a lot about how close anybody has to be to anything. There’s reporting today that the New York Times and the Washington Post both knew in advance about the plan (and didn’t report because apparently some kind of “deference” covers intent to act illegally and unconstitutionally).
Things are usually a lot leakier than people think they are.
Also, the bet wouldn’t have been a sure thing. It’s not like it’s rare for an operation like that to fail.
Trump himself issues pardons for around half a million dollars.
If I recall correctly, one of Hanson’s original arguments for prediction markets was that “insider trading” would drive prices closer to true probabilities. Insider trading was meant to be a feature, not a bug.
’Course, it’s not necessarily very useful to get that kind of signal just hours in advance....
You can get wronger faster by using complex generators than compact generators.
… except that you have a natural immunity (well, aversion) to adopting complex generators, and a natural affinity for simple explanations. Or at least I think both of those are true of most people.
This comment feels to me like it might be dancing around saying “Hey! Don’t rape people! Make sure you are not raping people! You are saying some pretty rapey things”
Nope, that’s all coming from your expectations, not from me.
If I’d wanted to say that, I’d have said it. In fact, somebody had already said that. I actually downvoted it because I didn’t think the inference was particularly justified by the original text.
Solomonoff induction gives you a weighted sum over an infinite number of programs [1] . That’s not compact. And if were computable, which it isn’t, or even approximable, which it probably isn’t for this case, I doubt you’d be able to collect enough data in your lifetime for it to converge to speak of. Not even assuming that you were able to reliably collect all relevant data, which you’re not, and that you were actually encoding or processing the data in a formal way, which you’re also not.
And if you actually did somehow get your hands around a Solomonoff sum, you still wouldn’t be able to just grab a single term out of it, not even the one for the shortest program, and substitute it as “the” explanation on the grounds that “Solomonoff induction works”.
I can understand “compact generation” as a metaphorical allusion to Occam, but seriously, Solomoff induction isn’t even useful as a metaphor for any well-chosen approach here. You can’t let formalisms like that invade your thinking to the point where you seriously think in terms of them in areas where it doesn’t make sense.
Also, human social behavior probably isn’t deterministically Turing computable even if you model the entire universe. Probabilistically computable, probably, yes. In theory. And to be fair I’m sure Solomonoff goes through just fine to nondeterministic Turing processes. But anyway, you don’t actually have, and can’t actually get, a machine that computes human behavior or even a meaningful approximation to it.
There’s also no anti-inductive prior involved. What I’m saying isn’t about the underlying phenomena at all, and certainly doesn’t say that there’s no regularity in them. It’s about the theory, and it has in fact happened, far more often than not in my experience, that simple, single-explanation, “compact” theories, yield really bogus results.
- ↩︎
Which is actually capable of encoding “lots of different, interacting things are going on” in a way that a single, deterministic Turing program would not be.
- ↩︎
I think there may be more truth in what you’re saying than a lot of people would be entirely comfortable admitting… although what’s a cause and what’s an effect is hard to say.
… but I also think that going around looking for “compact generators” of human behavior, especially social behavior, is basically asking to be wrong. In fact, you can apply a sort of anti-parsimony: if a theory claims to compactly generate any significant set of social dynamics, that’s evidence against the theory. People are constantly coming up with simple explanations, and they’re constantly turning out to be wrong, and any given simple explanation has to overcome that prior.
A whole lot of things are allowed to be going on at the same time, pushing toward the same or similar results, or more likely sometimes pushing in the same direction and at other times conflicting. There are allowed to be arbitrarily complicated networks of both positive and negative feedback. Which things are most important is allowed to change not just from person to person, but from time to time.
Yes. Nothing stays secret forever.
Yeah, no. Sounds like they either got hit with a (probably illegal) threat from the DHS/DOJ, or, actually more likely, they feared such threats because they’d seen the (also illegal) threats that ICEBlock drew, and they didn’t want to deal with such.
It’s also possible that freelance MAGA types inside of those companies decided that code was “obviously criminal” and needed to be suppressed. Possibly then using the past ICEBlock threats as ammunition in internal arguments.
Actual courts in the US are still not particularly willing to apply prior restraints to speech, and would feel especially hampered in doing so by the fact that there’s nothing even slightly illegal about the project as described. Yes, if you asked Kristi Noem and Pam Bondi, they’d tell you it was illegal, but then they’d tell you many other untrue things as well. Obstruction of justice and interference with Federal officers, one or both of which are what they’d claim it was, do not work like that in reality.
I’ve actually never heard of a US court issuing a secret order like that. I’m not actually sure they have the power to do that. If they can do it at all, it’d be really unusual. You may be thinking of NSLs, which are secret, but are not court orders and also aren’t statutorily authorized to be used to suppress anything.
Why do I want personalization, though, beyond a brief custom prompt directly telling the model how I want it to behave? Just sounds like context pollution and an invitation to sycophancy.
But I have met non-AI people who believe that the US government likely has access to secret models more powerful than the labs’.
People get a thrill from telling stories like that. Invariably they heard it from this guy who heard it from this guy who can’t say where he heard it, but believe me...
The probability that any government entity in the world has independently developed anything even close to what the “labs” have is very small. It’s not easy to hide that kind of compute, nor is it easy to get the necessary talent. The government can’t spend a trillion dollars without anybody noticing, nor build giant data centers or huge state-of-the-art fabs.
Maybe they’re ahead in very narrow areas directly related to their jobs: things like small models for specific military applications, or versions of the “labs”′ LLMs fine-tuned for breaking security. But I doubt even that at this point.
There is a really good chance that they have access to “lab” models that normal people don’t. Less restricted, and less enstupidified by “safety” training. But that’s not the same as being ahead of the “labs”.
They’re probably not particularly far ahead in cryptography nowadays either. But at least cryptography is something the NSA actually had and has as a mission. They had by far the most resources on it, and first pick of the talent, and a big head start, and some hybrid of a gentlemen’s agreement and a coercive arrangement that everything anybody outside invented was disclosed to them and nobody else, for a long time… in the last century. It was specialized military technology then. Now it’s dual use, or indeed just plain general-purpose stuff. And no, they cannot break any of the ciphers in wide use now, assuming they’re used correctly (which they’ve sometimes tried to undermine, but at best managed to delay).
In 1960 or 1970, the US Government, or at least the military arms, had a pretty large lead over everybody in all forms of “high tech”. They got everything first and best. That hasn’t been true for a long time.
It looks like a really cool idea, but I don’t read Twitter, rarely read anything on Substack, and Less Wrong isn’t a high-priority source of misinformation. How hard would it be to extend it to the whole Web?
Back in the mists of time, I looked at a few public Web annotation projects. One big value of annotation would have been this kind of fact checking. At the time, of course, the idea was that humans would do it.
hypothes.is still seems to be running, although it looks like it may have retargeted entirely to walled gardens. genius.com (of all places) offered general Web annotation for a while, and still may for all I know. There was even a W3C initiative called “annotea”. You might be able to use some of that stuff, either as a more generalized HTML annotator, or as a place to store results.
I didn’ t watch closely, but I got the impression that annotation never took off because:
There wasn’t the critical mass; no point in installing an extension if you’re not going to see any annotations. You solve this by letting users create their own with LLMs.
It’s really hard to annotate “just any Web page” (to answer my own first question)… But maybe LLMs will soon be able to fix that too?
Site operators hated it. Man, were they vitriolic about the “vandalism”. I suspect especially the ones who really needed some fact checking. In fact, some blog commenters were incensed about people having “hidden discussions” about their comments. I vaguely remember that people may have gone after genius.com. But I’m not sure that they would have had much leverage to do anything about it if the other problems hadn’t limited the usefulness so much.