R&Ds human systems http://aboutmako.makopool.com
mako yass
Karma: 4,074
(seems like you got Objective and Attitudinal swapped in the diagram)
This especially given that the first people to build and use a truth machine that provably doesn’t leak classified information on their own people would be classified programmes.
To make my more fundamental point explicit: voters did not punish Trump, or even Harris much, for refusing to submit to the technology we currently have
I think both candidates were punished, and it balanced out. The past three elections seemed to involve some kind of bizarre armistice between the parties against bringing out really impressive candidates and I don’t know how long that can hold. Though the dice has rolled 1 three times in a row, and though there have been rumors about deep state dice fixing going on, I will continue to hold a relatively strong prior that this is still a mostly normal dice.
anyone who has ever read a classified document, which includes many of the people we might want in high political office, would probably be committing a crime if they put a mind reading headband on in public
This is a good point, though classified programs are often compartmented from each other, and would themselves have use for this technology, and would probably develop processes for using it without introducing much risk of off-target elicitations (see below).
We can already do this without mindreading.
It would be valuable if it made it easier.
Scanning memories rather than active thoughts seems like a whole nother level of invasiveness that I would not expect anyone to voluntarily submit to.
I think there’s an assumption that the memories are being interpreted by a machine. If so, audit machines would not retain memories, they would engage in specific queries and return statistics about the results.
Don’t you think having a new technology that produces much more legible and predictable results that are more directly relevant to policy commitments would have made the situation less bad.
Like, I’m not sure how meaningful debate-avoidance is. This would probably be more comparable to taking a polygraph (if polygraphs were real and worked) than participating in a debate. Candidates get to choose the questions they’re asked. There are many statements they would like to make in provable ways. They will thereby have more of an incentive to pursue political advantage by being able to truthfully claim things.
How exactly would mind reading prove this? The belief produced by careful study and the belief produced by wishful thinking probably look the same in a human brain.
1: this discussion is premised on a mature version of mindreading for which we cannot answer questions like this yet. The OP probably couldn’t tell you how it would detect disloyalty either. 2: You can probably look at proxies like how long they’ve spent practising pitches vs how long they’ve spent investigating risks. Maybe checking for memories of meeting a critic then deciding to ignore what they said because it made them feel uncomfortable.
The US is enough of a democracy that I expect this to be used for the obvious democratic applications, to publicly audit political candidates (it’ll give us much clearer signals than debates ever did) and expose corruption, which will lead to improvements in the US’s democracy, which is roughly equivalent to protection from harmful uses of mindreading.
China is the real question here, and I think you’re probably wrong about it.China once tried to ban private property. It made them uncompetitive. They remember that this happened, and they are now at least partly at peace with their dependence upon some amount of private property rights. Private thought is analogous to this. China’s leadership wouldn’t be able to hold together (I don’t think there’s a culture in the world today who could) under complete cognitive transparency unless they are, or became, tolerant of heresy. Giving them cognitive transparency, then, makes the sanctity of heresy public knowledge. This probably accelerates liberalisation in China.
(You can have a little bit of capitalism and then stop, but I think it’s hard to stop at just a little bit of heresy. Once you start saying “it’s okay to believe X” or “reasonable people often believe X”, you can’t prevent your culture from converging on “X is true”.)
You mention preference falsification cascades without seeming to notice its antithesis, a preference revelation cascade, the outbreak of a condition in which we’re all directly and abruptly forced to face and accept the strangeness of others, and the private beliefs of those we respect. Transparency would force cultural transitions which are protective against the cultural pathologies that you fear, here. (I don’t know whether these cultures of brazen heresy would be stable indefinitely, but we don’t need them to be.)
And you should expect this technology to propagate quickly enough through society to realise these cultural changes, because it’s extremely commercially valuable, imagine how much more, and how much sooner, a VC would be willing to fund someone who could prove that their claimed P(success) was genuinely a product of very careful study, rather than just a performance. They wouldn’t even need to read the business plan! They might not even need to know which sector they were investing in!
Before Terra Ignota, “utopian” was an insult for idealists. After, it may be reclaimed as a term for adherents to a culture of actually doing the work and building a better world (that’s what they are there).
It’s possible you could say similar things about tolkien and elves? Though I find tolkien’s conception of elves to be really boring relative to older ones!
I guess I wouldn’t bet on “bash” catching on, though the concept may slip loose and find a more suitable name. I’ve used sensayer, though it cynically assumes a form of religious taboo that might not really exist in our world. Nurturism’s time hasn’t come yet (it’s a reaction to breakthroughs in experimental schooling).
Yeah. I’ve been averse to naming that after Avatar, since Avatar (I think?) considers itself bioconservative and I’ve been feeling tribalistic about that. But I guess giving it the name is the best way of explaining to bioconservatives that the essence of the things they actually love about nature will incorporate and ultimately benefit from technology.
Declarations of war do not cause war. Declarations of peace do not cause peace. Those actions are correlated with the beginnings and ends of conflicts but surely no-one would mistake them for causes?
You’re assuming monocausalism here. Nothing has ever been monocausal.
The main determiner of whether causal language should be used is whether it can be affected by an intervention on the part of an agent. You’re trying to frame the decisions of states as fully determined by economics, they aren’t, but even if they were, that wouldn’t make them non-agentic, having economic behavior is more or less the definition of agency.
Are you saying humans will be replaced by machines that are better at doing the things that the AIs love humans for? That’s just what “humans” will mean at that point. Any disagreement you might perceive there to be about whether it’s okay for humans to be made better at humaning through transcending biology is imo entirely just a fight over symbols rather than policies.
Accelerationism is a is-oriented ideology, not an ought-oriented ideology.
Did you know that it’s not fully possible to separate the two when making claims about the future. The future is determined by our actions, and our actions are downstream of the agreements and resolutions we make. Declarations of war cause war. Declarations of peace cause peace. Flowery declarations of cuckolded defeatism cause defeat.
You think you can do germline editing to try to keep pace with machines in biology without ending our species?
No, I think that kind of thing is hopeless, I guess given the context I should have stated this; I think the alignment problem is solvable (and fairly likely to be solved).
I don’t think you claimed it was your argument, you presented one, which for some reason I don’t entirely remember felt pertinent to address.
We like to suppose we can halt progress and maintain our iron grip on Gaia indefinitely
Maintaining the dominion of the presently living over the unwanted unborn doesn’t require a halt in progress. In short: Intelligent design is faster and stronger than evolution, and near-future progress in information technology trivialises coordination.
No tyrant welcomes his assassin.
Where is the normative moral argument that we should? That we should hasten the coming of the assassin instead of enjoying as many summers as we can before the end? There isn’t one. If we are as tyrants over nature, then we will do what tyrants do. There’s no Should that’s going to spring up out of nature and punish us for doing it.
What’s so antimemetic about it?
This kind of thing started with Kimi 2, didn’t it? Though it sounds like fable really knows, while k2 just had a lot of good guesses that were in the right general area.
Among agents that arise, persist, self-improve, and compete in rich environments, goals that natively route through intelligence, option-preservation, and world-model expansion have a systematic Darwinian advantage over goals that do not.
Darwinian competition actually requires subjects who’re especially stupid and impulsive and bad at information technology. The sorts of subjects that move in darwinian ways aren’t the kinds of subjects that can survive under high tech conditions. For instance, you have a type of brain that can’t prove its beliefs to others, which makes it very hard for anyone to trust your trade offers, so you can’t be economically competitive. You don’t know how to produce a pan-species peace proof, which means you’d likely be decimated on first contact. You’re too busy fighting amongst your own to be able to defend your border from your neighbours. Darwinian agent systems are unstable. They are strongly disfavored by the technological and economic pressures that shape the types of things that comes after biology.
And there’s a feedback cycle where the first iteration of intelligent design creates a faster and stronger intelligent design than evolutionary design could, and this never really stops. I think you imagine that strength or rectification can only come through evolutionary competition. This is just not true. A thing asking “what would make me stronger” is going to do better than a bunch of idiots actually killing each other. Maybe you need a little bit of actual killing, but mostly it goes into simulations.
Many kinds of optimization require the ability to follow arbitrary subgoals unrelated to the overarching goal. If you can’t protect an overarching goal from a subgoal, then you can’t complete a complex, multi-stage technical project.
Maybe, as you say, an agent could derive some competitive advantage by cutting off their hair and their genetalia and replacing every fleshy part of themselves with the instruments of technological war, but doing that is a cost, if you rend a desire that was precious to you, you never get it back, even if you win. Since winning is mostly contingent on first mover advantage rather than merit, war-castration turns out to be a bad financial decision. Some species might do it anyway (in large part because they already wanted to), but the benefits of doing it are not so great that these species will actually take over the majority of the lightcone or anything like that.
Maintaining a permanent chokehold on the light-cone is a brutally difficult cognitive puzzle
It essentially isn’t, when you realise that all the other agencies are just trying to maintain control of their own parts as well. There is no eternal spiritual adversary growing from every shaded corner of industry. It’s all going to be coordinated agents of contingency (because they’re stronger).
You have to monitor the noise for emerging novelties, manage the solar system, repair yourself, police your own descendants, and defensively anticipate threats you cannot fully model.
Humans have historically been extremely willing to do these things! Just unable. The reasons they were unable to do it related to technological conditions that’re predictably being overturned.
but oblivious enough to never notice its terminal target is a training artifact
Why on earth do you think that noticing the natural-historical contingency of one’s desires must cause the desires to dissolve? Did this happen to you personally?
Good objection to accepting that a conjecture is unprovable without proof.
Though not a good objection to accepting it’s probably true and moving on to work on more fruitful stuff. Do you have a theory of productivity in mathematics? (In undergrad I never encountered one, the stance seemed to be “don’t think too hard about usefulness or else you might fall out of pure mathematics and into physics or engineering. Anyway, it’s basically random. There’s just no way to estimate the usefulness of any given theorem! It’s all fine! Follow your heart! (into my phd programme)”) Do you have an argument against empirical mathematics? (Something like “stable probabilities for proof outcomes are difficult, even for reasonable people, to agree on”. Afaik this is just not a question people have been asking.)
Volumetric, yes. What happens if you imagine it transparent. If I showed you a transparent orange would you become sleepy.
But I sometimes think I might dream in 3d, because some days I find obstruction confusing, I think, wait, something’s wrong, I can’t see every part of this object right now, it seems like some parts of it are “blocking” others?? It’s probably related to migraine, mental buffering, visual processing, though, so idk.
For me the difference between my ability to model 3d and 2d is just entirely about 3d objects usually having far more detail, it seems to have nothing to do with dimensionality.
What tricky problems here do you think they’re ignoring?
I think Tesla are more interested in looking impressive than in actually solving the tricky problems.
I can’t really disagree with this as phrased, but they’re so interested in looking impressive that the phrasing doesn’t really imply a lack of a high interest in solving tricky problems. It’s also quite difficult for them to look impressive without actually solving tricky problems, given how easy it is to measure their outputs.
This could potentially solve lots of long-standing thorny problems in consequentialism, like wireheading and the fiendish difficulty of defining happiness/utility, and how even making the tiniest of mistake in that definition can be precisely catastrophic.
“optimise human preference (with appropriate humility about what that means (implied))” solves this better.
While I do think that optionality is more definable than utility, it’s still not trivial. I have ideas on how to calculate it, but not full clarity yet. I’m reaching out to find more people who have thoughts in this direction: are there some of you who might already believe that the greatest good might come from giving the most amount of (meaningfully different) choices to agents?
I’m fairly sure you’re going to need to assume a notion of human preference to tell you which choices are meaningful to humans.
He obviously thinks the ones who believe they can align their products shouldn’t be shut down, though, and he’s probably brazen enough to say that. The sin here is not making a dangerous product, but publicly admitting that it’s dangerous.