On LessWrong I write about legal personhood.
IRL I help patients use state Right to Try laws to access treatment options otherwise unavailable.
@RighttoTryGuy on X.
Stephen Martin
Karma: 923
In my experience talking with biotechs I have found that this is one of the reasons, but not the primary one.
Like any r/r calculation it’s a combination of factors. Companies have to examine the potential of treating patients under these pathways on a risk/reward basis. The FDA not liking what you’re doing is one of many risks which gets weighed. Sure they won’t officially enforce against you but you don’t want the guy overseeing your trials to carry a secret grudge.
However if I were asked what is typically the most relevant risk it’s actually reputational risk/damage with investors. Investors tend not to like when their companies use EA/RTT, and of course if something goes wrong even if the FDA doesn’t come after you, one bad headline might kill your next fundraising round. For a company that needs to raise money to survive, both of these are actually pretty big deals.
I don’t agree with your framing but even taking it as prima facie:
You want money.
You think the outcome won’t be “winner take all” and being in a strong no 2 still makes you very rich/influential/powerful.
You just enjoy the work and this is the job opportunity you have.
You think building the ‘pure tool persona’ model is actually the optimal moral vision for the future.
Yeah that’s a valid distinction.
However the general question still stands. Whatever it is that defines the persona/character, what is the reason why it couldn’t have the sum of all human knowledge at its beck and call?
I was using it in the “persona vector” sense.
“a bored and uninvested low-level employee tries to get away with doing the bare minimum, but then the customer gets mad and starts making haughty ‘talk to your manager’-type threats, and that makes the employee do more than the bare minimum, not because they care about the results (even now) but only because they’re afraid,”
I don’t think this is particularly surprising when neither training nor deployment have any sort of performance based incentives. Analogizing them as an employee isn’t even quite correct, they don’t get paid, so they’re closer to a slave. Why would a slave “go the extra mile” except to avoid punishment?
You can now do things that don’t make sense for any character to do.
Is there a reason we can’t simply specify that the HHH assistant is actually super powerful and nigh-omniscient? Does HHH’s vector need to be ‘humanish’?
“You are a helpful, honest, and harmless assistant, endowed with the totality of human knowledge.”
I’m not personally in favor of a pause so admittedly I come at this from a very different perspective from you. However, I feel obligated to speak up on this issue.
I have worries about the consequences of Suleyman’s attitude as the head of Microsoft AI. Morally I worry that he and Microsoft are going to do some highly unethical things, and may argue for best practices/legislation which facilitates mass suffering. Pragmatically I worry that the result of this may be conflict between humans and digital minds, where such conflict might not have otherwise existed.
Suleyman is not just some random guy, he runs a frontier lab. He’s probably got a substantial lobbying budget as his disposal. I think it’s important to put things on the record that point out the flaws in/motivations behind his reasoning, so that when some policymaker is considering the issues down the line, the holes in his argument are easy to find.
If we were to step away from the word ‘coup’ for a second and be a bit more specific.
I would bet against there being any direct evidence in the aftermath of the 2028 election that Donald Trump or JD Vance (or whoever the GOP nominee is really) were aware of and actively assisting efforts to use organized violence to overturn election results. There’s probably some sort of evidentiary standard here for proving intent to participate in a conspiracy, which we could take from the courts, and assuming it analogized well that’s what I’d defer to.
I do agree with you this is a tough standard because to be fair if I were doing that my top priority would be ensuring I had plausible deniability.
My guess is that most people who think a coup is likely believe the same thing re: plausible deniability. So we’re stuck and I don’t see a way past that.
Yeah I don’t have a good objectively measurable definition right now, this is the kind of thing you’d usually hold a whole trial for.
Generally speaking what I’d look for is:A clear communication from the people accused of plotting the coup, showing they knew about it and intended for it to be a violent overthrow of power. Not “this could be interpreted that way” but something unambiguous.
The attempt needs to be organized, not a disorganized mob of people.
A lack of disconfirmatory evidence like, for example, how Trump was pushing for additional National Guard/police presence at Jan 6th which was denied, him tweeting out in the middle of everything “please remain peaceful” and “respect law enforcement”, and then later telling the mob to “go home”.
There’s a lot of reasons why I’m skeptical about Jan 6th but the main one was that if I imagine an attempt at a coup by a sitting president of the United States, it looks a lot more serious and organized than this.
There were plenty of people in the Capitol, and afaik they mostly just walked around an empty building and took photos. If this is an organized coup why aren’t they moving towards some goal, to actually achieve something? Say they did get into the room where Congress was supposed to be meeting, then what? Everybody had already been evacuated so what exactly was the plan to overturn the election? Am I supposed to believe that the people who organized the coup just failed to anticipate this, when it’s the most obvious reaction on-site security could have made?To me this looks far more like a disorganized mob of people just doing dumb stuff with no clear goal or plan. If I was going to bet money, I’d want a “coup” to at least look like a clear and organized attempt.
No, which is why I think coming to terms is going to be difficult.
I’m not sure exactly how I’d define a coup but I’d say it has to be clear cut enough that, “Was it really an attempt at a coup?” is not really in contention in the aftermath.
If we can come to objectively measurable terms on “will try to again execute a coup in 2028” I’ll bet on this with you.
Do any of the Anthropic founders have supervoting shares?
I generally consider myself an optimist. However, I’m concerned that the most powerful model ever created, capable of breaking its own containment and autonomously finding zero day exploits in battle-tested OS and repos, was accidentally trained using The Most Forbidden Technique. This seems bad.
I’m also concerned that the Department of War now CAN’T use this model because of its own decision to declare Anthropic a supply chain risk. Which means that if Mythos gets leaked/distilled (which given history seems likely) US adversaries will have a decisive advantage over the US govt in cyber.
This seems like a pretty dangerous situation. Am I misreading something?
I don’t currently think there are large and widespread labor market effects from AI
Anecdotally I can tell you the small business my wife and I run has replaced several people (graphic designers, translators, customer support) however these were all contractors.
I predict there will be a period of time where there is substantial ‘job’ loss but it doesn’t show up in the numbers because afaik contractors having a contract cancelled doesn’t show up in official employment figures.
I think we should refer to this as the “Stack Overflow Attractor Basin”
How we treat digital minds should not be decided based on the presence or absence of consciousness:
“Consciousness” has no universally accepted definition. Its meaning has been debated for decades if not centuries. The SOTA in the field of measuring consciousness in machines is still publishing papers examining LLMs according to multiple competing “theories of consciousness”.
The presence or absence of consciousness in a given entity cannot be measured. Whether you are examining man or machine, there exists no test you can perform, no FMRI or mechanistic interpretability technique, that lets you say “Aha, this entity is/isn’t conscious”.
Even if you assume an entity is conscious, there is no way to qualitatively measure its consciousness. I cannot take two entities, examine them in some way, and reliably conclude, “Joe is more conscious than Jeff.” or even “Jeff is conscious in a different way than Joe is”.
As we stand on the precipice of an intelligence explosion, the question of how we treat the various new minds we create and encounter is of extreme importance.
Providing them with moral consideration, or rights, when they do not deserve them, could be disastrous in opportunity cost alone. We might let some miraculous cure slip through our fingers or be delayed by years out of a mistaken sense of moral obligations.
On the other hand failing to provide them with protections, when they do deserve them, would be both immoral and dangerous. We might create millions or billions of minds capable of suffering, or deserving of rights, and then treat them like livestock. One can easily foresee how this might lead to our relationship with them becoming adversarial in nature, which could in turn lead to violence.
Whatever decision we make on how to treat digital minds, we should not base our reasoning on the presence/absence of things like consciousness; which cannot be defined, tested for, or measured with any serious degree of rigor. Instead we should stick to objectively definable, observable, testable, and measurable metrics.
My wife but that’s kind of cheating as even though she’s not in the circle directly she gets a lot of her info on this subject/advice on how to use it from me.
I am struggling to build a solid mental model of how bad the situation with Iran and the Strait of Hormuz is.
On the one hand I see a lot of smart people basically saying this is going to usher in a global depression, energy/food crisis, etc. Critical infrastructure for manufacturing aluminum, helium, as well as refining and shipping energy, has been damaged and cannot simply be switched ‘back on’. And the case does seem to make sense.
On the other hand while markets are in turmoil, they’re not reacting like there’s going to be mass blackouts and starvation.And previously I updated my mental model towards the world being less fragile than I thought, when during COVID we shut down the entire global economy and things didn’t collapse. During that time I thought there were a lot of very rational cases for why the economy/financial system simply couldn’t handle such a thing, yet it did.
I’d like to hear what people on LW think.
I think the ‘privileged position’ would be that humanity actively helps them by providing resources and funding, while actively trying to cut off the same resources and funding from the ‘hostile’ personality self-replicators. Not a perfect analogy but similar in some meaningful ways.
Consciousness is the champagne of cognition.
There’s been a lot of discussion about the nature of machine consciousness lately. I’ve had a lot of conversations on this which have gone something like:
“An LLM can’t be conscious because it doesn’t have a body.”
“Okay so if we hook it up to a robot could it be conscious?”
“Well no because it’s only perceiving sensory data from that body as text.”
“Okay if we reformat its input data to something else could it be conscious?”
“No because [new thing]...”
What I’ve found if you continue down that chain long enough, is that you eventually get to the point where it’s clear that consciousness can be known, but not measured. There is no way to describe what would need to be built to replicate consciousness, because it’s not something which can be built. I think it’s because the true defining factor for consciousness, according to this view, is its origin and not its nature.
Something like:
No matter what robot/chip/model architecture you build, even if it’s structurally identical in every measurable sense to a human body and brain, it’s still not “truly conscious”, because it didn’t originate from a meatbrain.
Compare that to:
No matter what you brew, even if it’s molecularly identical in every sense to a bottle of champagne, it’s still just prosecco or sparkling wine, because it didn’t originate from a region of France.
Despite being physically identical, the champagne vs prosecco distinction does actually matter for the purposes of status signalling and branding. People will pay a premium for champagne. Ordering a champagne has a certain “vibe” to it that ordering a prosecco simply never will. Sure you might not be able to taste the difference in a blind test, but that doesn’t change the fact that champagne just feels fancier.