The final set of images looks a bit like someone zooming in on a map*. (The blue part of the first image looks like the head of a cat.)
*ETA: specifically the yellow region. (Not because it’s small.)
People are right/just to fear you developing political language if you appear to be actively
The end of this sentence appears to be missing.
More generally, I appreciate this post, and I think it’s a good distillation—as someone who can’t read what it’s a distillation of.
I also think that evaluating distillation quality well is easier with access to the conversation/data being distilled.
Absent any examples of conversations becoming public, it looks like distillation is the way things are going. While I don’t have any reason to suspect there are one or more conspiracies, given this:
It’s *also* the case that private channels enable collusion
was brought up, I am curious how robust distillations are (intended to be) against such things, as well as how one goes about incentivizing “publishing”. For example, I have a model where pre-registered results are better* because they limit certain things like publication bias. I don’t have such a model for “conversations”, which, while valuable, are a different research paradigm. (I don’t have as much of a model for, in general, how to figure out the best thing to do, absent experiments.)
*”Better” in terms of result strength, and not necessarily the best thing (in a utilitarian sense).
For those who prefer to read the post on Putanumonit:
The post says there are 13* comments, but only 6* can be seen.
*After this is posted: 14 and 7, respectively.
I feel like this same set of problems gets re-solved a lot. I’m worried that it’s a sign of ill health for the field.
Maybe the problem is getting everyone on the same page.
(This is meant to be purely illustrative, not taken seriously. Also, given how hard it was to come up with frames, it might be better to replace using lenses this way with ‘questions that are always good to ask’.)
Lenses:__Economic__Narrative  Empiricism 
 Movies these days seem to be lacking realism/X.
Is there not an audience for realism/X? (Or is this a market failure?)
Are most movies produced by studios that aren’t good at writing realism/X?
Is it more expensive to produce movies which are more realistic/X?
Harder to make money off of?
Is this the result of government regulation? Self-regulation?
Do the people involved in making movies (scriptwriters, directors, etc.) prefer less realistic**/X movies? Find it easier to make such movies?
 What is the alternative to realism that is more common?
(Realistically, the best way to make progress on a question like this is probably by unpacking ‘What you mean by “realism”/X.’)
 A struggle between the forces of good and evil.
Who controls Hollywood, good or evil? Both? Neither?
What reasons might they have for doing this?
It’s easier to have the good guys win in movies if you’re less realistic. It also delivers a particular message ’you will win if you’re good, no matter how ridiculous that sounds.
It lulls people into a false sense of security. “All evil needs to prevail is every good person doing nothing.” As it is hard to get people to do nothing, the nothing must be obscured by an illusion of doing something—thus, meaningless visual media, Netflix, etc.
Life isn’t perfect. People go to the movies to get away from it all/see the people they agree with win. It doesn’t have to make sense, it just has to be entertaining and end happily.
Movie makers don’t care about realism. Conflicts of Good versus Evil, where the good guys always win, in movies that don’t make sense aren’t about Good versus Evil. They’re just another opportunity for movie makers to set up their side as “Good” and the other side as “Evil”. This is why movies today are getting political (to the detriment of their quality).
 Looking at data
Assess the quality of a sample of movies, perhaps across time periods, perhaps highly rated/popular movies.
Has the factor we’re interested in changed over time?
(Is realism going down, up, in a cycle, or randomly—say, based on really popular movies coming out which do or don’t have features (such as realism), and then more movies like that getting made, Y number of Years later.)
Have other factors? Are there any relationships in the data?
(Update: it might be better to just come up with a list of questions that are always good to ask/have been really useful in the past (in other domains) and use that instead. The chart is just 1) a row of such questions 2) a row where you add a checkmark after you’ve answered that question (about the topic you’re trying to understand.)
A lens/frame/framework is a way of looking at things. I meant this to be a suggestion to see how a lens can be applied to other domains by constructing a chart/checklist as follows:
There are n columns and 2 rows, where n is the number of lenses. The first column tells you what is in each row. The top element of the first column contains ‘the name of your idea’. The bottom element of the first column can contain the word “lens”. The bottom row (after the first element) contains the name of each lens (that you have given it). The top row (after the first element) contains either blank spaces or check marks*.
Coming up with a procedure to see if you’ve thought through all the implications of a model may also be useful.
*One could also put a page number in it, and write about that idea through that lens on that page.
Great post by the way, it was really useful to see all these, together as one system.
and experiences you interact with
Relatedly, I’d suggest ‘Things you create or skills you have.’
you develop the ability to eliminate the channels/methods which present the most negative feedback.
The least useful feedback?
Start by developing ideas from activities you enjoy:
Try more things, see if you enjoy them. (Also, sometimes you can learn from things you don’t like—a story with a specific form of bad storytelling might teach you something about the right way to tell stories.)
Structure of Information Flows Record every idea you have:
This section didn’t have bold parts.
Idea Rate = Number of Ideas / Number of Opportunities
Why wouldn’t “Idea Rate = Ideas per (Unit of Time)” ? It would seem one could increase the amount/rate of ideas they have, not only by increases their ideas per opportunity, but also by increasing their number of opportunities.
Constants: Number of Opportunities = 800 Ideas/Opportunity = Idea Rate = 20% Good Ideas/Idea = Success Rate = 10%
Number of Opportunities = 800
Ideas/Opportunity = Idea Rate = 20%
Good Ideas/Idea = Success Rate = 10%
I didn’t understand this until I copied it here, and the formatting clicked, and then it all made sense.
Arming yourself with a vast knowledge of any particular situation or topic gives you a better chance of coming up with the correct solution to a given problem because as your network of understanding grows,
Problems are important, as is coming up with them/having good sources for them.
Each individual you come in contact with is an opportunity to glean unique and valuable information from.
It might be useful to come up with frames and give them names and put them in a list, so you can do this:
[New Idea]: (check)
Frames [Frame 1] [Frame 2] etc.
Reduce Cognitive Delay[**]
Also see how you can implement an idea, particularly—quickly. (This one can be hard.)
More generally, release “Delays” period. Having a model of the process can help with this. (It’s possible the low hanging fruit has been picked in communication technology on the raw speed front*, but it’s useful to note how this can speed up things we do.)
*To such an extent some may find it detrimental. One could compare the quality of comments on twitter with the quality of letters, or the quality of of moves in a live chess game versus one by post. (It’s also easier to draw on paper.)
Looking for HARSH criticism
That’s hard to do with such a good idea.
[**]####5. Gain Around Positive Feedback Loops a. Find a receptive audience:
The section above is related to that fact that “idea quality” can be subjective—coming up with ideas that sound great is all well and good, but reality is the final arbiter. (Though this kind of depends on what you’re working on.) Finding ways to implement things or ideas/testing things out might help. I’d also ask where these “first principles” come from.
If you enjoy something, you might not learn as much. Consider the popular Lord of the Rings. Did you learn something by reading/watching it? The first time? The n-th time?
By engaging in more conversation about your ideas, you develop a better grasp of why you receive negative feedback about particular topics.
Eh. Can you really change the world with an idea that doesn’t upset people?
This ties in to the idea that it is possible for you to produce a synthesis of contrarian idea.
This could use some elaboration.
This isn’t quite embedded agency, but it requires the base optimizer to be “larger” than the mesa-optimizer, only allowing mesa-suboptimizers, which is unlikely to be guaranteed in general.
Size might be easier to handle if some parts of the design are shared. For example, if the mesa-optimizer’s design was the same as the agent, and the agent understood itself, and knew the mesa-optimizer’s design, then it seems like them being the same size wouldn’t be (as much of) an issue.
Principal optimization failures occur either if the mesa-optimiser itself falls prey to a Goodhart failure due to shared failures in the model, or if the mesa-optimizer model or goals are different than the principal’s in ways that allow the metrics not to align with the principals’ goals. (Abrams correctly noted in an earlier comment that this is misalignment. I’m not sure, but it seems this is principally a terminology issue.)
1) It seems like there’s a difference between the two cases. If I write a program to take the CRT, and then we both take it, and we both get the same score (and that isn’t a perfect score), because it solved them the way I solve them, that doesn’t sound like misalignment.
2) Calling the issues between the agents because of model differences “terminology issues” could also work well—this may be a little like people talking past each other.
Lastly, there are mesa-transoptimizers, where typical human types of principle-agent failures can occur because the mesa-optimizer has different goals. The other way this occurs is if the mesa-optimizer has access to or builds a different model than the base-optimzer.
Some efforts require multiple parties being on the same page. Perhaps a self driving car that drives on the wrong side of the road could be called “unsynchronized” or “out of sync”. (If we really like using the word “alignment” the ideal state could be called “model alignment”.)
But arbitrarily low probabilities need not exist.
I believe for practical purposes, “I (or you) buy a cheap lottery ticket, and if it’s the winning ticket, then you pay me $1” is low enough.
Create a machine that creates lightning strikes.
The first issue isn’t humans abusing the system. It’s opening your brain/etc. up to attack by parasites, to say nothing of disease.
And that would probably be an issue way before the system would be developed enough to have a lot of, if any, upsides from functionality, let alone downsides.
I think the issue isn’t trial and error making one (securely) - it’s that it’d be expensive, and 2 parties would have to have it, to use it.
What’s the closest thing to this we see in any species?
I recommend asking this as its own question, and note that communication across species may be more interesting.
Links still work, I think it’s a cross-posting issue.
(Details: If you click on it, it goes to: https://www.lesswrong.com/posts/XzetppcF8BNoDqFBs/https://www.lesswrong.com/out?url=https%3A%2F%2Fwww.replicationmarkets.com%2F
If you shorten that to:
It redirects to:
What does it take for something to qualify as agent AI?
Consider something like Siri. Suppose you could not only ask for information (“What will the weather be like today?”), but you could also ask for action (“Call 911/the hospital”). Does this cross the line from “Oracle” to “Agent”?
when the capabilities of an unconstrained Agent AI will essentially always surpass those of an Oracle-human synthesis.
Nitpick: the capabilities of either a) unconstrained Agent AI/s, or b) Artificial Agent-human synthesis, will essentially always surpass those of an Oracle-human synthesis. We might have to work our way up to AIs without humans being more effective.
Perhaps degree of investment. Consider the amount of time it takes for someone to grow up, and the effort involved in teaching them (how to talk, read, etc.). (And before that, pregnancy.)
There is at least one book that plays with this—the protagonist finds out they were stolen from ‘their family’ as a baby (or really small child), and the people who stole them raised them, and up to that point they had no idea. I don’t remember the title.
A more extreme version of the interpersonal is that (one might suppose) you could have two (otherwise*) identical universes such that in the first Bob answers “3” and in the second Bob answers “5″, where both Bob’s feel (currently) the same way (largely*), but a) used different reference points within their life, or b) focus on different things—perhaps in universe 1 Bob thinks or says “my life could be better − 3”, but in universe 2 Bob thinks or says “my life could be worse** − 5′.
*This might require some assumptions about Bob, that don’t necessarily apply to everyone.
**Perhaps in universe 2, it occurs to Bob that he’s never had cancer, or anything of similar magnitude.
Assume all guns were pink by law tomorrow.
All guns, or all new guns made from here on out?