ozziegooen
Karma: 904 (LW), 0 (AF)
NewTop
Can We Place Trust in Post-AGI Forecasting Evaluations?
I like exploration. Could also see synonyms of exploration. “discovery”, “disclosure”, “origination”, “introduction”
Thanks! Good point about the division. I agree that the different parts to be done by different groups, I’m not sure with the best way of doing each one is. My guess is that some experts should be incorporated into the foundational understanding process, but that they would want to use many other tools (like the ones you mention). I would imagine all could be either done in the private or public sector.
I didn’t mean that all analogies around diseases were good, especially around psychology. The challenges are quite hard; even in cases where a lot of good work goes into making ontologies, there could be tons of edge cases in similar.
That said, I think medicine is one of the best examples of the importance and occasional effectiveness of large ontologies. If one is doing independent work in the field, I would imagine it is rare that they would be best served by doing their own ontology development, given how much is been done so far.
Good point, thanks. Even though we data could have values multiple sources, that could still be more useful than nothing, but it’s probably better where possible to use specific sources like the CIA World Factbook, if you trust that they will have the information the future.
I think I’ll be quite doable, but take some infrastructural work of course. Could you be a bit more specific about the predictions you want to make?
The Prediction Pyramid: Why Fundamental Work is Needed for Prediction Work
Yup, in general, I agree.
I really like this formulation, will consider the specifics more. It kind of reminds me of Pokemon types or Guilds in Magic.
Short story: An AGI’s Repugnant Physics Experiment
Thanks for the ideas!
I was also hesitant to use “clarification”, but do kind of think about it as one “clarifying” their messy thoughts on paper, and clarifying them with a few people. I feel like “sketch” implies incompleteness, which is not exactly the case, and description and delineation are not descriptive enough.
Some other related terms. Blueprint, survey, first pass, embryonic, pioneering, preliminary, germinal, prosaic.
Good to know. Can you link to another resource that states this? Wikipedia says “the amount a decision maker would be willing to pay for information prior to making a decision”, LessWrong has something similar “how much answering a question allows a decision-maker to improve its decision”.
https://en.wikipedia.org/wiki/Value_of_information https://www.lesswrong.com/posts/vADtvr9iDeYsCDfxd/value-of-information-four-examples
Changed. I originally moved accessibility to the bottom, out of order, just because the other three are more similar to each other, but don’t have a strong preference.
That sounds interesting, I look forward to eventually reading more about it, if that is published online.
Good question. It’s more technical than most of the ones I was considering and I have a harder time judging it because I haven’t really gone through it. I think with pure-math posts the line is blurry because if it’s sufficiently formal, it can be hard to make more explanatory, for dedicated readers.
I imagine that there are very math-heavy posts on Agent foundations that are both optimized for readability from other viewers, and ones more made as a first pass at writing down the ideas. That specific post seems like it’s doing a good amount of work to be clear and connect it to other useful work, so I would tend to think of it as explanatory. Of course, if it’s the first time the main content is online, then to viewers it would fulfill both purposes of being the original source, and also the most readable source.
Good to know. These are the main three types I’ve been thinking about working on so it seemed personally useful, I assumed the grouping may be useful to others as well. I could definitely imagine many other kinds of groupings for other purposes.
Three Kinds of Research Documents: Clarification, Explanatory, Academic
There’s no real difference, RAIN just a quick choice because I figured people may prefer it. Happy to change if people have preferences, would agree ARIN sounds cool.
Do other commenters have thoughts here?
Glad you like it :)
There’s definitely a ton of stuff that comes to mind, but I don’t want to spend too much time on this (have other posts to write), but a few quick thoughts.
Novelty
The Origin of Consciousness in the Breakdown of the Bicameral Mind
On the Origin of SpeciesAccessibility
Zen and the Art of Motorcycle Maintenance
3Blue1Brown’s Videro SeriesRobustness
Euclid’s Elements
The Encyclopædia BritannicaImportance is more relative to the reader and is about positive expected value, so is harder to say. Perhaps one good example is an 80,000 Hours article that gets one to change their career.
I’m also interested in others here have recommendations or good examples.
Thanks for the considered comment.
I think the main crux here is how valuable money will be post-AGI. My impression is that it will still be quite valuable. Unless there is a substantial redistribution effort (which would have other issues), I imagine economic growth will make the rich more money than the poor. I’d also think that even though it would be “paradise”, many people would care about how many resources they have. Having one-millionth of all human resources may effectively give you access to one-millionth of everything produced by future AGIs.
Scenarios where AGI is friendly (not killing us) could be significantly more important to humans than ones in which it is not. Even if it has a 1% chance of being friendly, in that scenario, it’s possible we could be alive for a really long time.
Last, it may not have to be the case that everyone thinks money will be valuable post-AGI, but that some people with money think so. In those cases, they could exchange with others pre-AGI to take that specific risk.
So I generally agree there’s a lot of uncertainty, but think it’s less than you do. That said, this is, of course, something to apply predictions to.