The paper is annoying trapped inside a Word document,
Thank you for contributing to open science by freeing it.
Pattern
Karma: 798
How common are Condorcet winners?
This seems very useful.
Errata:
wasy to read
easy
In other words, if the Predict-O-Matic knows it will predict P = A, it assigns probability 1 to the proposition that it will predict P = A.
It’s a predictor—it produces probabilities (or expected value?). There’s also some rules about probability that it might follow—like if asked to guess the probability it rains next wednesday, it will give the same answer as if asked to guess the probability it will give when asked tomorrow.
[1] Whoever gets control of the share gets control of the company for one year, and gets dividends based on how well the company did that year. [2] Each person bids based on what they expect they could make. [3] So the highest bidder is the person who can run the company the best, and they can’t be out-bid. [4] So, you get the best possible person to run your company, and they’re incentivized to do their best, so that they get the most money at the end of the year.
[3] doesn’t seem to follow. The person who wins an auction is usually the person who bids the most on it.
In Relaxed adversarial training for inner alignment, I argued that one way of mechanistically verifying an acceptability condition might be to split a model into a value-neutral piece (its optimization procedure) and a value-laden piece (its objective).
1. That summary might be useful as a TL:DR on that post, unless the description was only referencing what aspects of it are important for (the ideas you are advancing in) this post.
2. It seems like those would be hard to disentangle because it seems like a value piece only cares about the things that it values, and thus, its “value neutral piece” might be incomplete for other values—though this might depend on what you mean by “optimization procedure”.
This seems relevant to the connection between strategy stealing and objective impact.
So, you get the best possible person to run your company, and they’re incentivized to do their best,
The details on this one didn’t fit together.
Betting that someone will live is equivalent to putting a price on their heads;
Unless there’s more details. Betting person X will not die of cancer might create an incentive for a cancer assassin, but that’s different then killing someone using any means. (The issue of how the bets are cashed out might be important—are we betting on the truth, or using a proxy like the announced cause of death?) Though betting someone will die of cancer might be an incentive to save them from dying that way, or killing them another way. (Cancer might be unusual w.r.t how hard preventing that method of death would be.)
Predict-O-Matic runs on a local search which only represents a single hypothesis at a time, and modifies the hypothesis.
The inner optimizer was a surprise—one doesn’t usually think of a hypothesis as an agent.
Some people can tell that you are either lying or believing false things, due to your boldly claiming things in this uncertain world. They will then suspect your epistemic and moral fiber, and distrust everything you say.
At most only the subset of people for which this changes depending on what you do/say should be taken into account.
People are already enduring the truth(1), therefore, they can stand what is true(2)?
Your examples are mostly cases where 1 doesn’t exactly hold, so 2 not following in those scenarios doesn’t seem like an invalidation. (If someone argues “If A is true then B is true.” and you argue that ‘there are cases where A isn’t true that B isn’t true’ that doesn’t really address the argument.)
The Litany of Gendlin is conjecture (1) supported by fallacy (2), with no evidence for it(3), and a great many plausible disproofs(4).
It is true that I can think of times that it is better to face the truth, hard though that might be. But that only proves that some knowledge is better than some ignorance, not that all facts are better to know than not.
(1) A conjecture is “an opinion or conclusion formed on the basis of incomplete information”. On what basis is this a conclusion based on information? (In context if you said it was an incorrect statement that would make sense. Claims about the process by which it was generated require evidence, and are beside the point, which is whether it is correct or incorrect.)
(2) What fallacy?
(3) You have already stated there is evidence—narrow enough it does not fit the conclusion in full, but that is different from “no evidence for”. (Unless you think there’s no evidence for gravity.)
(4) Most of your “disproofs” are the same. Aside from its logic, you are arguing as if some divine authority might force the truth upon everyone if we accept this Litany, or some diabolical force might do so only in the worst possible cases. The Litany does not say that seeking out all knowledge should be your first priority—you would die of starvation before proving the primality (or compositeness) of every positive integer.
What could it mean for a statement to be “true but not provable”? Is this just because there are some statements such that neither P nor not-P can be proven, yet one of them must be true? If so, I would (stubbornly) contest that perhaps P and not-P really are both non-true.
Can you give an example where both P and not-P are both non-true?
Basic bet: if [candidate] wins the next election, you pay me $5, if he loses, I pay you $5.
Conditional bet: Conditional on the next president being [specific candidate], if marijuana is legalized (between 2020 and 2024), I pay you $5, else you pay me $5. If the condition is not met, no one pays anyone.
EDIT: added next, and (between 2020 and 2024). That’s kind of important.
If the point is to not just know what the market says, but to know how the world works, then prediction markets in themselves may not be of much help. Here are two quick examples to demonstrate illustrate the point:
They can also be made conditional.
Question: Why doesn’t AlphaGo ever try to spell out death threats on the board and intimidate its opponent into resigning? This seems like it would be a highly effective strategy for winning.
At a guess AlphaGo doesn’t because it isn’t an agent. Which just passes the buck to why isn’t it an agent, so at a guess it’s a partial agent. What this means is kind of like, it’s a good sport—it’s not going to try to spell out death threats. (Though this seems more to do with it a) it not knowing language—imagine trying to spell out threats to aliens you’ve never seen on a Go board, when a1) you don’t have a language, a2) the aliens don’t know your language, and b):
It’s answering a question about its model of the world which is different from the real world.
) Though it was trained via simulation/watching pro games (depending on the version). If you just trained such a program on a database where that was a strategy, maybe you’d get something that would. Additionally, AI has a track record of also being (what some might call) a bad sport—using “cheats” and the like. It’s kind of about the action space and the training I’d guess.
Basically, if you’re looking for an AI to come up with new ways of being evil, maybe it needs a head start—once a bot understands that some patterns spelled out on the board will work well against a certain type of opponent*, maybe it’ll try to find patterns that do that. Maybe it’s an “architecture” issue, not a training issue—Monte Carlo Tree Search might be well suited to beating Go, but not to finding ways to spell out death threats on a Go board in the middle of a game. (I also don’t think that’s a good strategy a priori.)
*You could test how different ways of training turn out if you add a way to cheat/cheatcodes—like if you spell out “I WIN” or one swear word** you win.
**I imagine trying to go all the way to threats immediately (inside the game Go) isn’t going to go very fast, so you have to start small.
And this induction should be used not only in the discovery of axioms but also in drawing boundaries around notions. It is in this induction that our chief hope lies.
[[Here Bacon again mentions the importance of Looking Into the Dark.]]
Looking into the Dark, and something of how words* should be fashioned. (May relate to sequence material, may be more broad.)
*This may apply beyond words, he uses the word notions—for example, if “axiom” refers to “hypothesis” perhaps by notions he means something specific other than words—theories or ontology or departments? Or he means multiple things.
if you had conducted yourself perfectly yet still ended up in your present ·miserable· condition, you would have not even a hope of improvement. But as things stand, with your misfortunes being due not to the circumstances but to your own errors, you can hope that by abandoning or correcting these errors you can make a great change for the better.
An uplifting point.
In other words: if I seem to eat the same foods quite often (despite claiming to like variety), you might conclude that I like familiarity when it’s actually just that I like what I like. I’ve found a set of foods which I particularly enjoy (which I can rotate between for the sake of variety). That doesn’t mean it is familiarity itself which I enjoy.
Agents trade off exploring and exploiting, and when they’re exploiting they look like they’re minimizing prediction error?
How would you test the conjecture?
Errata:
and it’s unavoidably possible for 51% of the participants can conspire
to conspire
The same for 4 significant digits.
Systematic error is conflated with conflict (in b), following sections (in the vicinity of a) which claim error is not conscious. Even if I accept b,
why should c follow? Why not tune out what can’t be verified, or isn’t worth verifying? (I say this as someone who intends to vote on this post only after running the code.)