This post compiles my personal comments on the Epistemology book from the Lesswrong 2018 bookset. I put almost no effort into making it legible or relevant to anyone else.
For logistics team, okay to err on side of comparing shallower nodes just to come to agreement.
<a id=”org7acebea”></a>
Has implications for #sbub . Centralized and federated systems look an awful lot like the bad case of communication. The only way to converse is to straight up adopt the output. If I want #sbub to empower people trying to deeply understand the world, there has to be a lot of space just for interacting with other people’s models and drawing intricate connections to your own.
Currently I need to tune down my prune step and increase my babble. A lot. #habits
#sbub in my current imagination is extremely babble and prune. Some concretizations include:
Computers are capable of producing babble. That’s an idea for a feature right there: integrated gpt-2.
Run another Lesswrong poll. Seed statements from babble and prune article. Seed claims of which the reader uses and how well this serves them, what others use and need. Seed claims of good sources of babble, good algorithms of prune, good combinations.
ask LW team about make a post with the convo embedded
make a link post, send delayed invitations to answer and comment
“if you notice a hint of confusion or not-rightness, sit with it for a moment. Say a short sentence out loud expressing some aspect of the intuition, write it down and submit it if you will.”
provide a submission form for seed statement ideas, things you don’t automatically want to vote in favor of.
Scrabble babble as an example of the kind of play I want to do with my language generation project.
Advice in this article seems geared from an assumption of the brain as an implicit graph, namely that “individual edges and neighborhoods can be computed in reasonable time”. This assumption may not hold for me. I really struggle to generate and re-compact local planning. Expansion may be related? Author clarifies this is not the same thing as connectivity, and doesn’t clarify which actions in their example address which metric. I find myself trying to capture any babble I have 2 steps ahead
Better if this post had offered its own explanations of expander graph and aji. Wikipedia threw a lot of math symbols at me and I can’t tell off-hand how it relates to this context. Go metaphors gave me very aesthetic feels but not a model with moving parts. What are the implications?
“building an argument from scraps lying on the side of the road” captures my experience
I have insisted for a couple of years now that everyone should randomize their decisions more. If you’re stuck in a bad rut, randomize all the details and then use prune-discernment to tweak it until you have something that looks workable. If your pruning can’t keep up with your babble, then prune by weighted randomization or pick your favorite out of the first couple draws. (see the secretary problem)
Mysticism is so great for babble. Absolute best. Form tight connections between everything and anything. Encode important ideas in inscrutable, babbling poetry. Find the essence of hope in a dropped kernel of corn.
“To save the world, I will start by doing the humble and proper things I know how to do within the confines of my own life”—one for my #TumblrLikes. #habits
Sounds Confucian
<a id=”org3e0bd7a”></a>
Prune
Prune has multiple layers, yes. Subconscious, Conscious thought, Spoken word, Written word, Published word. (At times, Written and Spoken are the other way around for me.)
River of Babble: that’s powerful imagery right there.
This essay has awesome style, it moves the reader in its currents like deep mysticism. We need some of that. We need it in addition to prized technical, specific writing.
Contains one of the better woo-free explications for meditation practice.
<a id=”org9a2ddb8″></a>
Local Validity as a Key to Sanity and Civilization
<a id=”orgd254bb6″></a>
Third example at the beginning is shaky. I get stuck on figuring out whether I agree with the implicit judgment of ape-mammalian social instincts as default, naive, and destructive.
<a id=”orgfca63f7″></a>
Your faith in your priors gates your understanding of the rules of allowable argument steps. Being convincable of unintuitive truths by logical validity could get REALLY REALLY BAD OUTCOMES IN PRACTICE if not coupled with the willingness and ability to rethink your foundational assumptions. That skill does not come easily. “Fair to one side but not the other” speaks to this problem; their machinery to determine local validity works but they won’t use it to evaluate an argument from premises that don’t fit snugly into their understanding of the world.
<a id=”orgdadbc9b”></a>
I have the weaker version of the low-level attribute, even as I’ve explicitly championed local validity.
#habits—When I notice myself steel-manning my own side I will state the original claim’s bogus-ness out loud (or write it down) as a simple statement with no equivocation. Only after this will I attend to adjacent, more substantive claims.
<a id=”org537d959″></a>
Comment by bryjnar—“this throws up an important constraint for people designing systems that humans are supposed to interact with: you must make it possible to reason simply and locally about them.”
good faith surveys of evidence take effort. I don’t want to prune my efforts to dredge up faded recall or verbalize intuitions about the high-level generator of disagreement.
<a id=”org4f0a339″></a>
Going over a public political debate with this framework might train a good habit of thought. Combing over a thread in a rationalist space with this framework may generate useful feedback.
<a id=”orga77ea98″></a>
surprised to see double-crux described as a pruning technique. i would think of it as directing seeking: it has a function for determining if you’re on track but it also keeps introducing a whole bunch of new avenues to explore.
<a id=”org3be4839″></a>
Naming the Nameless
<a id=”org640e094″></a>
She was not exaggerating about chinese opera.
<a id=”org738d769″></a>
Consider how to detect style blindness in others. At its best, style is a high level generator of disagreement. So, “Varieties of Argumentative Experience” may help in describing specific error cascades to style-blind individuals.
<a id=”org26cbe60″></a>
Reaction is where the atheism movement was when I first entered it. I have concerns. I would like to co-opt their style to talk rationality to them. On Gab probably.
<a id=”orgf6f0927″></a>
project in the vicinity of “making amends between creators and expanders”: disco.coop.
<a id=”org1439911″></a>
New distinction: bets of expectation vs. bets of taste. The former requires specificity, the latter requires ????emotional connection????.
<a id=”orgf23342b”></a>
Toolbox-thinking and Law-thinking
<a id=”org87c407c”></a>
I’m itching to rewrite this with Scott’s Argument pyramid as a rubric. If you knows anyone would find that useful to read. Eliezer’s writing style may turn some people off for a reason.
<a id=”orgc6f33c2″></a>
How Law vs. Tool is relevant for #sbub . Defining the Ideal of a solution to a problem differs from grabbing the best available tool for it. I define the ideal because I believe the best available tools are not good enough; I want a comprehensible metric to make other people more useful help in coordinating towards better tools.
<a id=”org48189a2″></a>
When I find myself internally protesting that I don’t know what math to trust because reality is nearly always messier than proof premises… It’s not wrong… but something is funky about that and I need to reframe.
Mathematical law is timeless. There is no “running the algorithm”, really just twisting and turning your head until you hit the right angle to see what was by necessity true the whole time. Math, if you’re doing it right, is a whole lot of cleverly disguised tautology. #habits
<a id=”orgdb1b821″></a>
Third, related thinking style: Animus. When you don’t know the laws, when the metrics you care about are more nebulous and proxied than “distance”, you may still need some way to tie together your strategy and experience and hypotheses about a subject. In comes a form of animism: things and concepts have an essential spirit. It is your spirit, of course, the shadow of your own mind that reflects the subject and how you interact with it. It is the memes and archetypes that suffuse your imagination. It is the norms and incentives that push-pull on your actions. You don’t use animus-thinking: animus-thinking uses you.
<a id=”orgfdbe149″></a>
Thought Eliezer was wrong in the example of sticking to the left-hand rule in a maze. Surely you can just treat the path as a new entrance and resume the left-hand rule? I was wrong. I had fun thinking through it.
People who I’d want to introduce rationality to could benefit from having the fallback ability to understand whatever-it-is as a fact instead of a normative ideal. It was really hard to find the words for at the time. I kept gesturing to something like “cultivate the skill of knowing what it is right and choosing to do the wrong thing” or “face god and walk backwards into hell”.
<a id=”org8d1c5eb”></a>
Toward a New Technical Explanation of Technical Explanation
<a id=”org99afff0″></a>
If I recall correctly, (and I did), the theory of Logical induction uses prediction markets to eliminate the assumption of logical omniscience from Bayes. Logical Induction was a big breakthrough. Would implementing a market to that logic in #sbub constitute a (haha) market gamechanger?
Markets aren’t simple. Why would Bayes simplify to this weird thing humans invented to do with other humans?
The market assumption “tastes” like a tool shoved in to approximate the mathematical laws governing options space and credit propagation. Markets are a really good tool for searching option space; but still just a tool. MIRI, I accuse thee of wanting to privilege Truth when the actual laws are more about the distribution and properties of different paths through option space which are reachable by evolution.
<a id=”org4e4fe25″></a>
“I focus on the X, with only a minimal account on how Y makes it work” what a nice, succinct way to establish a context I frequently operate in, 10⁄10 would plagiarize
Table of Contents
Epistemology
Intro
This post compiles my personal comments on the Epistemology book from the Lesswrong 2018 bookset. I put almost no effort into making it legible or relevant to anyone else.
A sketch of good communication
For research, we want to exchange deep models
For logistics team, okay to err on side of comparing shallower nodes just to come to agreement.
Has implications for #sbub . Centralized and federated systems look an awful lot like the bad case of communication. The only way to converse is to straight up adopt the output. If I want #sbub to empower people trying to deeply understand the world, there has to be a lot of space just for interacting with other people’s models and drawing intricate connections to your own.
Babble and Prune Sequence
Babble and Prune
More Babble
Prune
Local Validity as a Key to Sanity and Civilization
Third example at the beginning is shaky. I get stuck on figuring out whether I agree with the implicit judgment of ape-mammalian social instincts as default, naive, and destructive.
Your faith in your priors gates your understanding of the rules of allowable argument steps. Being convincable of unintuitive truths by logical validity could get REALLY REALLY BAD OUTCOMES IN PRACTICE if not coupled with the willingness and ability to rethink your foundational assumptions. That skill does not come easily. “Fair to one side but not the other” speaks to this problem; their machinery to determine local validity works but they won’t use it to evaluate an argument from premises that don’t fit snugly into their understanding of the world.
I have the weaker version of the low-level attribute, even as I’ve explicitly championed local validity.
Comment by bryjnar—“this throws up an important constraint for people designing systems that humans are supposed to interact with: you must make it possible to reason simply and locally about them.”
Broken alarms
My alarms #habits
Compare to Umeshisms.
Varieties of Argumentative Experience
good faith surveys of evidence take effort. I don’t want to prune my efforts to dredge up faded recall or verbalize intuitions about the high-level generator of disagreement.
Going over a public political debate with this framework might train a good habit of thought. Combing over a thread in a rationalist space with this framework may generate useful feedback.
surprised to see double-crux described as a pruning technique. i would think of it as directing seeking: it has a function for determining if you’re on track but it also keeps introducing a whole bunch of new avenues to explore.
Naming the Nameless
She was not exaggerating about chinese opera.
Consider how to detect style blindness in others. At its best, style is a high level generator of disagreement. So, “Varieties of Argumentative Experience” may help in describing specific error cascades to style-blind individuals.
Reaction is where the atheism movement was when I first entered it. I have concerns. I would like to co-opt their style to talk rationality to them. On Gab probably.
project in the vicinity of “making amends between creators and expanders”: disco.coop.
New distinction: bets of expectation vs. bets of taste. The former requires specificity, the latter requires ????emotional connection????.
Toolbox-thinking and Law-thinking
I’m itching to rewrite this with Scott’s Argument pyramid as a rubric. If you knows anyone would find that useful to read. Eliezer’s writing style may turn some people off for a reason.
How Law vs. Tool is relevant for #sbub . Defining the Ideal of a solution to a problem differs from grabbing the best available tool for it. I define the ideal because I believe the best available tools are not good enough; I want a comprehensible metric to make other people more useful help in coordinating towards better tools.
When I find myself internally protesting that I don’t know what math to trust because reality is nearly always messier than proof premises… It’s not wrong… but something is funky about that and I need to reframe.
Third, related thinking style: Animus. When you don’t know the laws, when the metrics you care about are more nebulous and proxied than “distance”, you may still need some way to tie together your strategy and experience and hypotheses about a subject. In comes a form of animism: things and concepts have an essential spirit. It is your spirit, of course, the shadow of your own mind that reflects the subject and how you interact with it. It is the memes and archetypes that suffuse your imagination. It is the norms and incentives that push-pull on your actions. You don’t use animus-thinking: animus-thinking uses you.
Thought Eliezer was wrong in the example of sticking to the left-hand rule in a maze. Surely you can just treat the path as a new entrance and resume the left-hand rule? I was wrong. I had fun thinking through it.
People who I’d want to introduce rationality to could benefit from having the fallback ability to understand whatever-it-is as a fact instead of a normative ideal. It was really hard to find the words for at the time. I kept gesturing to something like “cultivate the skill of knowing what it is right and choosing to do the wrong thing” or “face god and walk backwards into hell”.
Toward a New Technical Explanation of Technical Explanation
If I recall correctly, (and I did), the theory of Logical induction uses prediction markets to eliminate the assumption of logical omniscience from Bayes. Logical Induction was a big breakthrough. Would implementing a market to that logic in #sbub constitute a (haha) market gamechanger?
“I focus on the X, with only a minimal account on how Y makes it work” what a nice, succinct way to establish a context I frequently operate in, 10⁄10 would plagiarize
Agency
Lotus-eating
rohit said he resented the idea that life isn’t just a series of lotuses to eat. the writing made it sound so easy not to fall into them, did not acknowledge let alone offer any concrete solutions to help with its very real difficulty.
<a id=”org11dc8f3″></a>
Epistemology
<a id=”org650c01a”></a>
Intro
<a id=”org8794c96″></a>
This post compiles my personal comments on the Epistemology book from the Lesswrong 2018 bookset. I put almost no effort into making it legible or relevant to anyone else.
<a id=”orgee2a32d”></a>
A sketch of good communication
<a id=”org652d52a”></a>
For research, we want to exchange deep models
<a id=”org10e7c49″></a>
For logistics team, okay to err on side of comparing shallower nodes just to come to agreement.
<a id=”org7acebea”></a>
Has implications for #sbub . Centralized and federated systems look an awful lot like the bad case of communication. The only way to converse is to straight up adopt the output. If I want #sbub to empower people trying to deeply understand the world, there has to be a lot of space just for interacting with other people’s models and drawing intricate connections to your own.
<a id=”orgc259ade”></a>
Babble and Prune Sequence
<a id=”orga3edf00″></a>
Babble and Prune
Currently I need to tune down my prune step and increase my babble. A lot. #habits
#sbub in my current imagination is extremely babble and prune. Some concretizations include:
Computers are capable of producing babble. That’s an idea for a feature right there: integrated gpt-2.
Run another Lesswrong poll. Seed statements from babble and prune article. Seed claims of which the reader uses and how well this serves them, what others use and need. Seed claims of good sources of babble, good algorithms of prune, good combinations.
ask LW team about make a post with the convo embedded
make a link post, send delayed invitations to answer and comment
“if you notice a hint of confusion or not-rightness, sit with it for a moment. Say a short sentence out loud expressing some aspect of the intuition, write it down and submit it if you will.”
provide a submission form for seed statement ideas, things you don’t automatically want to vote in favor of.
<a id=”orgd060fd3″></a>
More Babble
Akjash is a graph theorist, good to know
Scrabble babble as an example of the kind of play I want to do with my language generation project.
Advice in this article seems geared from an assumption of the brain as an implicit graph, namely that “individual edges and neighborhoods can be computed in reasonable time”. This assumption may not hold for me. I really struggle to generate and re-compact local planning. Expansion may be related? Author clarifies this is not the same thing as connectivity, and doesn’t clarify which actions in their example address which metric. I find myself trying to capture any babble I have 2 steps ahead
Better if this post had offered its own explanations of expander graph and aji. Wikipedia threw a lot of math symbols at me and I can’t tell off-hand how it relates to this context. Go metaphors gave me very aesthetic feels but not a model with moving parts. What are the implications?
“building an argument from scraps lying on the side of the road” captures my experience
I have insisted for a couple of years now that everyone should randomize their decisions more. If you’re stuck in a bad rut, randomize all the details and then use prune-discernment to tweak it until you have something that looks workable. If your pruning can’t keep up with your babble, then prune by weighted randomization or pick your favorite out of the first couple draws. (see the secretary problem)
Mysticism is so great for babble. Absolute best. Form tight connections between everything and anything. Encode important ideas in inscrutable, babbling poetry. Find the essence of hope in a dropped kernel of corn.
“To save the world, I will start by doing the humble and proper things I know how to do within the confines of my own life”—one for my #TumblrLikes. #habits
Sounds Confucian
<a id=”org3e0bd7a”></a>
Prune
Prune has multiple layers, yes. Subconscious, Conscious thought, Spoken word, Written word, Published word. (At times, Written and Spoken are the other way around for me.)
River of Babble: that’s powerful imagery right there.
This essay has awesome style, it moves the reader in its currents like deep mysticism. We need some of that. We need it in addition to prized technical, specific writing.
Contains one of the better woo-free explications for meditation practice.
<a id=”org9a2ddb8″></a>
Local Validity as a Key to Sanity and Civilization
<a id=”orgd254bb6″></a>
Third example at the beginning is shaky. I get stuck on figuring out whether I agree with the implicit judgment of ape-mammalian social instincts as default, naive, and destructive.
<a id=”orgfca63f7″></a>
Your faith in your priors gates your understanding of the rules of allowable argument steps. Being convincable of unintuitive truths by logical validity could get REALLY REALLY BAD OUTCOMES IN PRACTICE if not coupled with the willingness and ability to rethink your foundational assumptions. That skill does not come easily. “Fair to one side but not the other” speaks to this problem; their machinery to determine local validity works but they won’t use it to evaluate an argument from premises that don’t fit snugly into their understanding of the world.
<a id=”orgdadbc9b”></a>
I have the weaker version of the low-level attribute, even as I’ve explicitly championed local validity.
#habits—When I notice myself steel-manning my own side I will state the original claim’s bogus-ness out loud (or write it down) as a simple statement with no equivocation. Only after this will I attend to adjacent, more substantive claims.
<a id=”org537d959″></a>
Comment by bryjnar—“this throws up an important constraint for people designing systems that humans are supposed to interact with: you must make it possible to reason simply and locally about them.”
<a id=”org41a40c7″></a>
Broken alarms
<a id=”org0831c79″></a>
My alarms #habits
there is too much noise.
I need to not bother people.
make sense.
make myself busy.
defensiveness
<a id=”org4f2200b”></a>
Compare to Umeshisms.
<a id=”org519d045″></a>
Varieties of Argumentative Experience
<a id=”orgba5bd31″></a>
good faith surveys of evidence take effort. I don’t want to prune my efforts to dredge up faded recall or verbalize intuitions about the high-level generator of disagreement.
<a id=”org4f0a339″></a>
Going over a public political debate with this framework might train a good habit of thought. Combing over a thread in a rationalist space with this framework may generate useful feedback.
<a id=”orga77ea98″></a>
surprised to see double-crux described as a pruning technique. i would think of it as directing seeking: it has a function for determining if you’re on track but it also keeps introducing a whole bunch of new avenues to explore.
<a id=”org3be4839″></a>
Naming the Nameless
<a id=”org640e094″></a>
She was not exaggerating about chinese opera.
<a id=”org738d769″></a>
Consider how to detect style blindness in others. At its best, style is a high level generator of disagreement. So, “Varieties of Argumentative Experience” may help in describing specific error cascades to style-blind individuals.
<a id=”org26cbe60″></a>
Reaction is where the atheism movement was when I first entered it. I have concerns. I would like to co-opt their style to talk rationality to them. On Gab probably.
<a id=”orgf6f0927″></a>
project in the vicinity of “making amends between creators and expanders”: disco.coop.
<a id=”org1439911″></a>
New distinction: bets of expectation vs. bets of taste. The former requires specificity, the latter requires ????emotional connection????.
<a id=”orgf23342b”></a>
Toolbox-thinking and Law-thinking
<a id=”org87c407c”></a>
I’m itching to rewrite this with Scott’s Argument pyramid as a rubric. If you knows anyone would find that useful to read. Eliezer’s writing style may turn some people off for a reason.
<a id=”orgc6f33c2″></a>
How Law vs. Tool is relevant for #sbub . Defining the Ideal of a solution to a problem differs from grabbing the best available tool for it. I define the ideal because I believe the best available tools are not good enough; I want a comprehensible metric to make other people more useful help in coordinating towards better tools.
<a id=”org48189a2″></a>
When I find myself internally protesting that I don’t know what math to trust because reality is nearly always messier than proof premises… It’s not wrong… but something is funky about that and I need to reframe.
Mathematical law is timeless. There is no “running the algorithm”, really just twisting and turning your head until you hit the right angle to see what was by necessity true the whole time. Math, if you’re doing it right, is a whole lot of cleverly disguised tautology. #habits
<a id=”orgdb1b821″></a>
Third, related thinking style: Animus. When you don’t know the laws, when the metrics you care about are more nebulous and proxied than “distance”, you may still need some way to tie together your strategy and experience and hypotheses about a subject. In comes a form of animism: things and concepts have an essential spirit. It is your spirit, of course, the shadow of your own mind that reflects the subject and how you interact with it. It is the memes and archetypes that suffuse your imagination. It is the norms and incentives that push-pull on your actions. You don’t use animus-thinking: animus-thinking uses you.
<a id=”orgfdbe149″></a>
Thought Eliezer was wrong in the example of sticking to the left-hand rule in a maze. Surely you can just treat the path as a new entrance and resume the left-hand rule? I was wrong. I had fun thinking through it.
https://photos.app.goo.gl/oDF1ZUV1Em3fCMHX9
<a id=”orgedb7530″></a>
People who I’d want to introduce rationality to could benefit from having the fallback ability to understand whatever-it-is as a fact instead of a normative ideal. It was really hard to find the words for at the time. I kept gesturing to something like “cultivate the skill of knowing what it is right and choosing to do the wrong thing” or “face god and walk backwards into hell”.
<a id=”org8d1c5eb”></a>
Toward a New Technical Explanation of Technical Explanation
<a id=”org99afff0″></a>
If I recall correctly, (and I did), the theory of Logical induction uses prediction markets to eliminate the assumption of logical omniscience from Bayes. Logical Induction was a big breakthrough. Would implementing a market to that logic in #sbub constitute a (haha) market gamechanger?
Markets aren’t simple. Why would Bayes simplify to this weird thing humans invented to do with other humans?
The market assumption “tastes” like a tool shoved in to approximate the mathematical laws governing options space and credit propagation. Markets are a really good tool for searching option space; but still just a tool. MIRI, I accuse thee of wanting to privilege Truth when the actual laws are more about the distribution and properties of different paths through option space which are reachable by evolution.
<a id=”org4e4fe25″></a>
“I focus on the X, with only a minimal account on how Y makes it work” what a nice, succinct way to establish a context I frequently operate in, 10⁄10 would plagiarize