[i’m sure people on LW would love to tell me where i can find the answer or how they think about it]
How to properly defer to someone else’s belief instead of their evidence?
If Alice knows much more physics than me, learning that Alice assigns 90% to X is obviously evidence for X.
But blindly averaging toward expert beliefs seems to double-count correlated evidence.
It also conflates “probability emanating from a coherently-integrated world-model I can defend the gears-level of” with “probability if I was guessing on a prediction market.”
What is a cleaner way to think about epistemic deference?
How to think about the relative information efficiency of RL vs SFT?
People keep telling me “RL is very informationally inefficient.” It only receives a single bit of information (success/fail) over an entire rollout (tens of thousands of tokens!). Meanwhile, SFT gets a bit or more of information on every token.
But RL can make my warm-started model learn to do what I want in like a dozen steps while I can’t get SFT to do like anything useful in a dozen steps.
And RL and SFT on LLMs seem to reduce to the same cross-entropy loss anyway, just with a different coefficient. What is going on??
You know… like StackOverflow....
But I notice Q&A doesn’t seem to be the vibe here, nor quite fit with this particular web of knowledge.
So what can one do instead?
Maybe instead of …
forum-style “Hallo how do you ask questions?” post (tonally inappropriate, creates spam, doesn’t mesh with the web of knowledge)
… one could make a …
argument-style “LW should have a questions feature” post (and count on people to quickly tell you you’re wrong and LessWrong already supports this via [X], or no, LessWrong shouldn’t have questions, but you can do X)
how-to-style “How to ask questions on LW” post (include a thought or two, and let the comments fill in the rest — like writing a continuation-style base LLM prompt)
observation-style “I notice I don’t know where to put questions on LW” post (adding an observation to the pile; inviting discussion; maybe noticing a subtle confusion others didn’t realise they were confused about too)
Edit: I’m glad I made this post, because the comments have fully answered my original question, and given me a concrete way to direct this urge in the future:
Is this a thing? Was there ever an era where people did this?
Have you wanted to do ask questions? What did you do instead?
Notice question arise. Spend 5–20 minutes with Fable/Sol. Intellectually digest it, noticing what feels answered and what doesn’t, and how you’d explain it to someone else. Put THAT in a post maybe!
Are questions allowed on LessWrong?
Sometimes I want to post things like:
[thing i’m wondering about]
[here’s my initial stab at it]
[but i have no idea if this is right or wrong]
[i’m sure people on LW would love to tell me where i can find the answer or how they think about it]
How to properly defer to someone else’s belief instead of their evidence?
If Alice knows much more physics than me, learning that Alice assigns 90% to X is obviously evidence for X.
But blindly averaging toward expert beliefs seems to double-count correlated evidence.
It also conflates “probability emanating from a coherently-integrated world-model I can defend the gears-level of” with “probability if I was guessing on a prediction market.”
What is a cleaner way to think about epistemic deference?
How to think about the relative information efficiency of RL vs SFT?
People keep telling me “RL is very informationally inefficient.” It only receives a single bit of information (success/fail) over an entire rollout (tens of thousands of tokens!). Meanwhile, SFT gets a bit or more of information on every token.
But RL can make my warm-started model learn to do what I want in like a dozen steps while I can’t get SFT to do like anything useful in a dozen steps.
And RL and SFT on LLMs seem to reduce to the same cross-entropy loss anyway, just with a different coefficient. What is going on??
You know… like StackOverflow....
But I notice Q&A doesn’t seem to be the vibe here, nor quite fit with this particular web of knowledge.
So what can one do instead?
Maybe instead of …
forum-style “Hallo how do you ask questions?” post (tonally inappropriate, creates spam, doesn’t mesh with the web of knowledge)
… one could make a …
argument-style “LW should have a questions feature” post (and count on people to quickly tell you you’re wrong and LessWrong already supports this via [X], or no, LessWrong shouldn’t have questions, but you can do X)
how-to-style “How to ask questions on LW” post (include a thought or two, and let the comments fill in the rest — like writing a continuation-style base LLM prompt)
observation-style “I notice I don’t know where to put questions on LW” post (adding an observation to the pile; inviting discussion; maybe noticing a subtle confusion others didn’t realise they were confused about too)
Edit: I’m glad I made this post, because the comments have fully answered my original question, and given me a concrete way to direct this urge in the future:
Is this a thing? Was there ever an era where people did this?
Yes! There was a questions feature as recently as July of this year (deprecated for UI complexity).”
Have you wanted to do ask questions? What did you do instead?
Notice question arise.
Spend 5–20 minutes with Fable/Sol.
Intellectually digest it, noticing what feels answered and what doesn’t, and how you’d explain it to someone else.
Put THAT in a post maybe!
Simple bar.
“Are questions allowed?” was a somewhat mistaken framing, but was useful anyway.