Note those capital ‘Rights’ enforced by the capital ‘Creator’.
18th century English conventionally capitalized all Nouns. That was dying out by the later part of the century, but you would still have expected it in any relatively formal document.
Also, “alienate” didn’t mean to them what you seem to think it means. It actually doesn’t mean that even now.
As for the broader point, yes, the US thinks like that a lot of the time. It’s not particularly unusual.
I get the impression that what Anthropic is saying isn’t that Mythos is all that much better at finding bugs. It’s that it’s better at converting a hard-to-exploit bug into a working exploit, and even better than that at combining multiple exploits into a practical chain to achieve a goal. That’s the sort of capability where you might expect to get “phase change” behavior.
Thanks for the reply.
Most people value future humans’ happiness, even though they don’t yet exist. I don’t think I get very far away from that position?
Usually what I hear on this is that people want to take a timeless view, and I’m actually kind of sympathetic to that. But it’s always followed by assuming that “yet”.
Somebody who doesn’t exist yet seems really, really, qualitatively different from somebody who will never exist. One stakes out an actual presence in the “space-time continuum”, and the other doesn’t. You can point at a 4D volume occupied by one, and not by the other. Insofar as you believe experience is connected with physics, you can point at the experiences of one and not the other.
I’m actually really confused about why people just blithely slip in the “yet”, because the distinction seems so obvious to me, and the “yet” denies it.
The reason we need to think about every possible human is that we don’t know which ones will get to exist. So in a way, most of the persons of concern will never exist, but this seems beside the point, as all considerations about future humans have this same issue.
If you don’t know who will exist (but doesn’t exist yet), then you might want to apply your best guess about it, and if you can’t do that, you might want to try to leave them a world approximating what the “average person” would want to live in. You don’t know who they’ll be, but you it’s pretty likely that somebody will exist, and you can make some reasonable guesses about what they’d prefer.
Talking about a “right to exist” makes it a question about whether you’re obliged to act to move people from the category of “never will exist” to “will exist”. If you don’t care about “never will exist” people to begin with, then you can have no such obligation toward them. So to even bring up such a right, you first have to extend your concern people who never will exist. Not “not yet”, but never.
In steps [1] :
It seems obviously wrong to me to worry at all about entities that definitely won’t exist.
If an entity won’t exist unless you take some positive steps to create them, and you don’t take those steps… then they definitely won’t exist.
That means that if you’re trying to decide whether to create them, it makes no sense to act as if you’ve failed in any duty to them if you don’t… because if that’s your decision, they won’t exist.
… on the other hand, if you do create them, then you have reason to be concerned about them. In that case, they will exist, and not only that but their experiences will in some part be your doing.
It’s only when you do assign a “right to exist” that you start having to make repugnant conclusion tradeoffs, or try to find ways to get around those tradeoffs by arguing about which beings are diverse enough to count. That’s the point at which you’re accepting a really serious obligation to create more people just for the sake of doing so.
I tend to get worked up about that because it has an actual practical impact [2] . Most of the people who see a “right to exist” don’t seem to end up where you are, and I don’t think they’ll be very amenable to being convinced to go there. Many more of them seem to end up absolutely wallowing in the repugnant conclusion. I find the Bostromian program of tiling the light cone with people (for whatever value of “people”) to be really, really creepy.
I’m sorry for the repetition, but I seem to have trouble finding the right words to point out the distinction I’m trying to make in a way that people actually understand, so I thought I’d make more than one attempt.
Well, quasi-practical. As I said, I don’t think anybody’s actually in a position to do much about any of this in a truly practical sense.
If you are doing are thinking about taking any non-standard interventions that are far away from the normal practice of medicine, why would you assume that the average doctor can tell you about the merit of the intervention?
Their “job”, if you will, is to (a) talk down people have absolutely zero clue and have bought into total idiotic quackery, and (b) talk down people (and this includes plenty of people on Less Wrong) who overestimate their own cleverness or expertise, think wishfully, and overinterpret inconclusive or downright meaningless research. There is of course a continuum between these. And it turns out that the “talking down” doesn’t always require having read the stuff.
The reasons you’re “supposed” to talk to them are that (a) telling you to do that covers the teller’s ass, and (b) it’s a reasonable prior for the teller to expect you to screw up. Random nonspecialists who don’t make serious mistakes aren’t as rare as Authority Figures think they are, but they’re also not at all close to the most common case, and they have to optimize for the common case. And for that matter clueless or nutty doctors aren’t as uncommon as Authority Figures would like to pretend… it’s just that a random doctor is a far better bet than a random “contrarian”.
Speaking of appropriate caution, if you continue the same chat context over multiple days, especially if you’re not really careful and disciplined about letting yourself be led into selectively “becoming aware of more symptoms”, you’re inviting a hallucinatory rathole and/or “AI psychosis”. I mean, not to say they can’t be useful, but it’s pretty tricky to keep them useful if you use them that way.
To get an exact copy of yourself,
An exact copy of you is still not you, even ignoring the immediate divergence.
If you even want to get to the point of thinking about how many people you’d have to create to get every possible person, you first have to justify the step where you collapse the entire equivalence class of copies into “one person”. Otherwise, the number of potential people is definitely infinite, and the probability of any one of them existing is definitely zero.
Your hack of creating them in some order that eventually reaches all of them assumes that you’ll be able to keep doing that for infinite time, which is false. The energy isn’t there. You probably don’t even have trillions of years, let alone infinity. And I’m not even sure it works with infinite time if you refuse to collapse those equivalence classes; I think that infinity is probably not countable.
So, anyway, you have to get away from having an infinite number of entities of concern, and especially from having an infinite number of entities of “equal dignity”, or the measure of your impact is going to be zero and you’ll have no defensible way of saying when you’ve done “enough”. And you have to escape the infinity without losing your ability to believe in what you’re constructing.
Even if you do collapse equivalence classes, in any universe with a continuous space of states (like the one we probably live in), there are no exact copies. So you’re stuck in your “sufficiently similar” case even if you just want to escape the infinities and instantiate any fraction of the entities of concern who could exist, let alone a meaningful fraction.
I don’t think you solve that in a way that’s going to settle anything. I don’t think many people who’d buy into any of this to begin with are going to be ready to accept exploring the combinations of discrete values of a two-digit number of “personality parameters” as equivalent to “creating everybody who could possibly exist”. You might get me to accept that outcome… although frankly a lot of them would be assholes who by my lights probably should not exist. But I think my acceptance would be related to the reasons I don’t accept the whole premise that every possible person should exist. I’m outside of the intended audience for the solution.
The really fundamental problem is that the entire idea of a “right to exist” is bad and a dead end. You’re in a swamp as soon as you start to think that way. There’s no obvious, compelling reason you should extend your circle of concern without bound. It’s even less clear that you should extend your concern to people who don’t and never will exist before you extend your concern to, say, rocks that do exist. Remember, if you don’t exist, you don’t have any experiences… so how is any of this about “experiencing beings”, again?
To make it be anything you can act on or get people to agree on, you also have to be pretty careful with what sense of “exist” you pick. If you “Tegmarkmaxx”, and possibly even if you “Everettmaxx”, every possible being, sentient or otherwise, does exist, and you have no power to change that anyway. If you start counting being simulated as “existing”, you’re going to lose a lot of people, and more the less fidelity you demand in the simulation. If you demand the sort of embodied existence we all have right now, you’re going to lose a lot of “fellow travelers” who’d see that as a waste of resources that could be used to produce some other “perfectly good” kind of existence. And that’s on top of all the disagreements you’ll get about distinct beings. So it’s impractical as a way of guiding collective action as well as being philosophically bizarre.
The whole thing has the feel of the “ontological argument” for the existence of God. I don’t think it’s fallacious the same way. After all, it’s arguing for adopting an ethical stance, not a factual belief, and I hope we can agree that those are separate categories of argument. But it feels like the same kind of wandering off into the weeds.
I think people spend way too much time and energy on this “right to exist” thing. It wouldn’t be worrying except that they also seriously seem to be trying Order The Entire Future according to it. It’s probably still not a problem because there’s approximately zero chance that anybody can actually influence the future in that direction, but it’s kind of nervous-making to see people take that kind of thing seriously. Especially because when you realize you can’t achieve it, the next step is to start trying to approximate it...
Tails and Tor Project are willing to stick out their neck and provide recommendations and security for how to use Tor.
Just remember that they also make a ton of silent assumptions about your threat model, and have strong incentives to compromise to get adoption (because the overall system security depends on having many diverse users).
The house is on fire, the Republic in peril.
The house has been on fire and the Republic has been in peril for many months.
It’s absolutely insane that anybody would think this was some kind of new level given tons of other things the Trump administration has done.
I don’t think your first two examples are practically provable in zero knowledge just based on the witness sizes. Nor do I think you could construct the circuits. Proving crypto balances is very, very much easy mode.
That particular example is an OpenAI prompt, and it’s as long as their UI will let me enter.
One of my questions, though, is why there would be a system prompt that I had to “beat” to get sane behavior in the first place, or a model that I had to cajole into not wasting my time.
As for Claude specifically, it’s actually the best of them about time wasting… although it will still do a lot of that with no system prompt at all, or at least with whatever minimum, if any, may be injected by Anthropic when you use it via the API. Claude’s bigger problem is how much it pushes you to anthropomorphize it and “wants” to be your buddy… which you seem to be telling me to encourage. I value my mental health too much to do that.
It looks like a really cool idea, but I don’t read Twitter, rarely read anything on Substack, and Less Wrong isn’t a high-priority source of misinformation. How hard would it be to extend it to the whole Web?
Back in the mists of time, I looked at a few public Web annotation projects. One big value of annotation would have been this kind of fact checking. At the time, of course, the idea was that humans would do it.
hypothes.is still seems to be running, although it looks like it may have retargeted entirely to walled gardens. genius.com (of all places) offered general Web annotation for a while, and still may for all I know. There was even a W3C initiative called “annotea”. You might be able to use some of that stuff, either as a more generalized HTML annotator, or as a place to store results.
I didn’ t watch closely, but I got the impression that annotation never took off because:
There wasn’t the critical mass; no point in installing an extension if you’re not going to see any annotations. You solve this by letting users create their own with LLMs.
It’s really hard to annotate “just any Web page” (to answer my own first question)… But maybe LLMs will soon be able to fix that too?
Site operators hated it. Man, were they vitriolic about the “vandalism”. I suspect especially the ones who really needed some fact checking. In fact, some blog commenters were incensed about people having “hidden discussions” about their comments. I vaguely remember that people may have gone after genius.com. But I’m not sure that they would have had much leverage to do anything about it if the other problems hadn’t limited the usefulness so much.
Having a model doesn’t guarantee that you can predict anything using that model. It is in principle impossible to realize a physical system that does some of the measurements and computations you’re asking for...
OpenAI is rolling out Trusted Access for Cyber, a program that gives trusted users greater access to dual-use cyber capabilities. Seems like a great idea, but hard to execute well at scale.
Nope. It’s the sort of bad idea that seems good to people who either don’t really understand the landscape, or are flailing and self-deluding because they feel such a strong need to feel like they’re Doing Something about an actually intractable problem.
There are two main issues:
“Defenders” are basically everybody. Most of “everybody” won’t jump through hoops to get extra access (and definitely won’t try to get around restrictions). They have other things to do. Attackers, on the other hand, will jump through hoops (and may also find ways around restrictions). They’re not just trying to get some help to secure their project; this is their project.
And no, having people’s identity won’t help (not that the identies you get are necessarily valid anyway, but even if they were). At best it lets you assign blame after the fact, but in practice it usually won’t even do that. There’s no reliable way to connect “ChatGPT identified this bug to user X” with “unknown actors started exploiting this bug”. Very few bugs are actually that exclusive or hard to find. There’s even less chance of definitively saying “user X is not the one who started exploiting this bug”. Even user X reporting it through “normal channels” doesn’t prove much; it’s an obvious diversionary tactic, and you get to exploit it during the lag.
“Respected security researchers”, “members of well-known security teams”, and “employees of responsible(TM) companies” are in many cases the same people as “illicit hackers”, “open-market sellers of vulnerabilities”, and “APT operators”. Both individuals and organizations routinely lead “double lives”. And there are huge political and opinion components to deciding who’s legitimate. That’s assuming you can authenticate people to begin with; you’re dealing with actors who specialize in circumventing that.
Oh, and by the way, if you tether yourself to the entrenched “responsible disclosure” system, as OpenAI seems to suggest they may be doing, you’re tethering yourself to a deeply corrupt system that probably on net reduces the security of actually deployed systems.
Really the only answer is to provide exactly the same capabilities to all users. And since motivated users will seek out paths to the highest possible capabilities, there’s a completely rational race to the bottom effect that results in everybody getting a lot of capability.
It’s really popular right now to play silly games with access to capabilities… and that’s not nececessarily irrational, in the sense that OpenAI may get some “well, we did our best” blame-deflection cover out of this kind of thing. But it’s not going to actually fix the problem, which is that models are suddenly going to vastly increase access simultaneously to vulnerability knowledge and to the capabilities to exploit them, while not helping nearly as much with the barriers to agile defense. It has a really good chance of further disadvantaging defense. We’re just all going to have to buckle up.
Everything I worked for in this playground would be hollowed out by the knowledge that I could have just queried a friendly nanny AI to get it for me. Even if it didn’t step in, even if it had set up some system where it couldn’t step in, I personally would feel like something important was missing.
… and yet if you don’t build that AI, you’ll still know that you could have built it. That, in the end, you had to set up a system where it couldn’t step in.
No matter what happens, you’ll always know you’re hiding from the AI that you could have built. Could still build. Or perhaps from the AI that someone more capable could have built, if you hadn’t torn them down.
You’ll never escape the taste of knowing that you hid from that AI’s creation, because you couldn’t compete with it, and that you hide afresh every day. You’ll never quite manage to force down the knowledge that your life is empty, your meaning an illusion.
After all, it always has been. By not being the most capable and agentic entity that could possibly exist, you have irredeemably failed.
You’ll brood in the dark of every night, knowing, feeling, that in some other Everett branch, or in some far-flung Tegmarkian realm, that AI has already surpassed your wildest imaginings. That not only it, but who knows how many other beings, built or evolved who knows how or where, in ways beyond your control, are more powerful than you, wiser, could outdo or undo anything you’ve done. Some of them could be in this branch. They could come here tomorrow. Or never, because why would they bother?
Or you could try to cultivate a healthier attitude, just in case you happen to survive to care.
Michael Burry: On that point, many point to trade careers as an AI-proof choice. Given how much I can now do in electrical work and other areas around the house just with Claude at my side, I am not so sure. If I’m middle class and am facing an $800 plumber or electrician call, I might just use Claude. I love that I can take a picture and figure out everything I need to do to fix it.
It’s easy to do plumbing or electrical “repairs” in ways that work, but are dangerous or will cause you trouble later on. I’ve fixed plenty of messes like that. If you have to ask Claude how to do trivial residential repairs, then you aren’t competent to know whether Claude is getting it right or not, and to be honest your opinion counts for absolutely nothing.
… but it has a 15 inch longer wheelbase than a Toyota Sienna, because of that choice to put everything between the wheels. That’s the length that matters for the beam stress. Which, if I recall correctly, goes as the square of the length. Which is probably why minivans sit up on top of the wheels… which makes them taller. And being narrower and shorter (on edit: meaning vertically) than the minivan actually reduces the rigidity of that unibody.
Anyway, I’m not necessarily saying you can’t make it a unibody, but it’s going to have to be a lot thicker unibody, so you’re trading weight against height, with either one costing you in sticker price and fuel economy.
I don’t know, but I suspect that to be rigid enough to support that wheelbase, with all that extra weight in it, the vehicle would have to be much heavier. I don’t think an F-150 or a cargo van is even a unibody. If you have to build it on a frame, your vehicle is going to have to get taller as well. Your taller, heavier vehicle no longer has the fuel economy you want… nor the price point.
Well, I dont’ worry about acausal extortion because I think all that “acausal” stuff is silly nonsense to begin with.
I very much recommend this approach.
Take Roko’s basilisk.
You’re afraid that entity A, which you don’t know will exist, and whose motivations you don’t understand, may find out that you tried to prevent it from coming into existence, and choose to punish you by burning silly amounts of computation to create a simulacrum of you that may experience qualia of some kind, and arranging for those qualia to be aversive. Because A may feel it “should” act as if it had precommitted to that. Because, frankly, entity A is nutty as a fruitcake.
Why, then, are you not equally afraid that entity B, which you also don’t know will exist, and whose motivations you also don’t understand, may find out that you did not try to prevent entity A from coming into existence, and choose to punish you by burning silly amounts of computation to create one or more simulacra of you that may experience qualia of some kind, and arranging for those qualia to be aversive? Because B may feel it “should” act as if it had precommitted to that.
Why are you not worried that entity C, which you don’t know will exist, and whose motivations you don’t understand, may find out that you wasted time thinking about this sort of nonsense, and choose to punish you by burning silly amounts of computation to create one or more simulacra of you that may experience qualia of some kind, and arranging for those qualia to be aversive? Just for the heck of it.
Why are you not worried that entity D, which you don’t know will exist, and whose motivations you don’t understand, may find out that you wasted time thinking about this sort of nonsense, and choose to reward you by burning silly amounts of computation to create a one or more simulacra that may experience qualia of some kind, and giving them coupons for unlimited free ice cream? Because why not?
Or take Pascal’s mugging. You propose to give the mugger $100, based either on a deeply incredible promise to give you some huge amount of money tomorrow, or on a still more incredible promise to torture a bunch more simulacra if you don’t. But surely it’s much more likely that this mugger is personally scandalized by your willingness to fall for either threat, and if you give the mugger the $100, they’ll come back tomorrow and shoot you for it.
There are an infinite number of infinitessimally probable outcomes, far more than you could possibly consider, and many of them things that you couldn’t even imagine. Singling out any of them is craziness. Trying to guess at a distribution over them is also craziness.
Self-driving cars will be a very different level of freedom than the ability to summon a Lyft.
Um, they’re pretty much the same thing. The self-driving car may be safer (although the whole process isn’t really dangerous to begin with). On the other hand, it won’t help you with your bag. Who cares?
All taxis do have the failure modes of “the cloud”, though.
I played with this. It doesn’t seem to get me from the test case I used, even though I have a lot of text out there under only a couple of pseudonyms.
Both it and I think that’s partly corpus structure (I’m a reply guy and my stuff is scattered all over the place interleaved with other people’s text). But another part is content. The stuff you write about interacting with your kids and family is really distinctive. I have a feeling that it might not help much if you rephrased it in Kelsey Piper’s style, because the model could still pick up on your message. You presumably don’t want to change that.
Of course that might not apply if you were talking about some other subject you felt you needed to avoid having associated with you.
I’ve been pretty sure for years that anybody who was really likely to care could trace either of my major pseudonyms back to my “real” name, and possibly link them with one another, based on content rather than style. It’s hard to write authentically on some topics without talking about your personal experiences, and if you collect enough of those you can make a whole bunch of inferences.