My past occasional blogging included a few exercises that might be of interest. I’m pretty sure #4 is basically an expanded version of something from the Sequences, although I don’t recall which post exactly. Others are more open ended. (Along the lines of #5 I’ve been casually collecting examples of scientific controversy and speculation with relatively clear-cut resolutions for the purposes of giving interested laypeople practice evaluating these things, to the extent that’s possible. I don’t know if I’ll ever get around to writing something up, but if anyone has their own examples, I’d love to hear about them.)
Maybe we’re talking about different things, but from the page I’m on now where I’m looking at and replying to the discussion of the link (https://www.lesserwrong.com/posts/vhAJ4DBXZukE7SNtq/how-popper-killed-particle-physics/) the only link to the actual article is still gjm’s. In particular, the title of the blog post is not a link, although I would have expected it to be. To get to the actual article I have to click on the linkpost title in one of the other post listings (Featured/Frontpage/All). This happens to me for all link posts and for different browsers on both mobile and desktop.
Content note: This is a collection/expansion of stuff I’ve previously posted about elsewhere. I’ve gathered it here because it’s semi-related to Eliezer’s recent posts. It’s not meant to be a response to the “inadequacy” toolbox or a claim to ownership of any particular idea, but only one more perspective people may find useful as they’re thinking about these things.
For what it’s worth, I was another (the other?) person who downvoted the comment in question early (having upvoted the post, mostly for explaining an unfamiliar interesting thing clearly).
Catching up on all this has been a little odd to me. I’m obviously not a culture lord, but also my vote wasn’t about this question of “the bar” except (not that I would naturally frame it this way) perhaps as far as I read CoolShirtMcPants as doing something similar to what you said you were doing—”here is my considered position on this, I encourage people to try it on and attend to specifically how it might come out as I imply”—and you as creating an impasse instead of recognizing that and trying to draw out more concrete arguments/scenarios/evidence. Or that even if CSMP wasn’t intentionally doing that, a “bar” should ask that you treat the comment that way.
On one hand, sure, the situation wasn’t quite symmetric. And it was an obvious, generic-seeming objection, surely already considered at least by the author and better-expressed in other comments. But on the other hand, it can still be worth saying for the sake of readers or for starting a more substantive conversation; CSMP at least tried to dig a little deeper. And in this kind of blogging I don’t usually see one person’s (pseudonymously or otherwise) staking out some position as stronger evidence than another’s doing so. Neither should really get you further than deciding it’s worth thinking about for yourself. This case wasn’t an exception.
(I waffled on saying anything at all here because your referendum, if there is one, appears to have grown beyond this, and all this stuff about status seems to me to be a poor framing. But reading votes is a tricky business, so I can at least provide more information.)
Two more thoughts: the above is probably more common in [what I intuitively think of as] “physical” problems where the parameters have some sort of geometric or causal relationship, which is maybe less meaningful for neural networks?
Also, for optimization more broadly, your constraints will give you a way to wind up with many parameters that can’t be changed to decrease your function, without requiring a massive coincidence. (The boundary of the feasible region is lower-dimensional.) Again, I guess not something deep learning has to worry about in full generality.
Hm. Thinking of this in terms of the few relevant projects I’ve worked on, problems with (nominally) 10,000 parameters definitely had plenty of local minima. In retrospect it’s easy to see how. Saddles could be arbitrarily long, where many parameters basically become irrelevant depending on where you’re standing, and the only way out is effectively restarting. More generally, the parameters were very far from independent. Besides the saddles, for example, you had rough clusters of parameters where you’d want all or none but not half to be (say) small in most situations. In other words, the problem wasn’t “really” 10,000-dimensional; we just didn’t know how or where to reduce dimensionality. I wonder how common that is.
I think the main thing I want to say [besides my response to Oliver below] is that this post was not framed in my head as starting a conversation in response to your post, but as gesturing in the direction of some under-emphasized considerations as one contribution in a long-running conversation about rationalist jargon. Of course, I ended up opening with and only taking quotes from you, and now it looks the way it does, i.e. targeting your “bid” but somewhat askew. So that was a mistake, for which I apologize.
Also, I know I basically asked for your “actually a defeater” response, but I really was non-rhetorically hoping people would think about what I was leaning upon and accomplishing (or not) by using the Names that I chose throughout that might not align with their prior ideas about what the Names are for.
Pretty much agreed. I might go beyond “provisional” to “disposable”. I really do take maintaining fluidity and not fooling yourself to be more important/possible than creating common vocabulary or high-level unitary concepts or introspective handles [though I don’t introspect verbally, so maybe I would say that]; I really do think the way the community treats words is a good lever for that.
(Of course, this is all very abstract, isn’t a full elaboration of what I believe, and certainly has no force of argument. At best, I’m pointing towards a few considerations I could readily abstract out of the sum of my observations, in the hopes that people can recontextualize some of their reading with concerns along these lines.)
I’d also like to see someone try your last suggestion. (If nothing else, I might use it in a fiction project.)
I appreciate your outspokenness on these things. Writing like yours on EA has made me pause after having been resigned for a long time that these communities weren’t (and maybe never were) growing towards my idealizations of them. I don’t know how much we want the same things, and anyway I’m perhaps too much of an outsider with other commitments these days to make too much noise, but I’ll continue to look forward to your posting.
Taking up your framework, I’m not sure how much of what I see is predatory behavior by sociopaths (though there is that, malicious or otherwise) versus ordinary selection pressure in a loose coalition of different sorts of geeks, some whom may think they’re the same sort. Either way, it seems like I’ve connected with more like-minded people by dimming my beacon even into obscurantism than otherwise.
(I don’t consider this rude at all, and will welcome your post-mulling thoughts should you choose to add them. I can also say more about where I’m coming from when I get the chance.)
Yeah, my autocorrect guessed what he meant easily enough, but I’m convinced. I think I just needed to see someone else say this.
Woah! That sounds very unusual—it might be valuable for you talk about all that explicitly rather than write more like this post (which was presumably generated from your internalization of all that study, but which doesn’t go out of its way to show it).
(Also, for what it’s worth, I thought the title “Theodicy in Humans” was good—good enough for me to generate an approximation of the post before even reading it, although with slightly different context I’d have expected “theodicy” to be a derogatory analogy. And to bikeshed a bit, I might have used “theodicy for humans” [or maybe “of”], as you do in the text; it seems more accurate, and for your purposes it would make sense to use the title verbatim at least once.)
Also in favor of not only reserving judgment but ideally deferring exposure until one can seriously evaluate things, You Can’t Not Believe Everything You Read; and then there’s the mere-exposure effect to worry about, especially from prolific authors or in environments with a lot of repetition. (This is again the weird thing where you have apparently opposite biases which show up in similar situations, and it may not be obvious which direction you’ll be taken. In this case I’d guess it depends on one’s initial disposition and the level of conscious attention the idea is getting. [In particular, “inferential distance” isn’t the determinant—with the illusion of transparency, the gap can go unrecognized by either party and lead to unjustified agreement.] Luckily, one is led to similar reading/discussion policies either way.)
Venue also matters a lot through the social context it brings. Individual Wordpress blogs often feel like you’re saying “this is where my writing lives; by commenting, you’re coming into my house”, which can be challenging to take lightly—especially when you’re talking about a neighborhood of individual blogs, few of which get regular comments. Meanwhile social media is a weird mix of jokes and personal content with discussion-oriented ideas, where there’s an uncertain rudeness in potentially burying someone with attention or notifications by Starting Discourse. And in both of these, if it’s not controversy or gossip or dilettantism, then posting the most makes you king.
So I was/am hopeful about posting more to LW 2.0 largely for the sake of better defaults around “this is for having a conversation”—both “formally” in responding directly to or building on the OP, and more “socially” or indirectly by contributing thoughts on the same subject, and in a venue with moderation and karma where things can bubble up without the speculative/social/everyone’s-an-expert elements (or sheer consistent quantity).
I find that my writing seems to actively repel comments compared to stuff that gets comparably received by other metrics. I do try to go out of my way to write mostly on the rare occasions I have something unambiguously sensible or useful to contribute; it earns me a high upvote/downvote ratio, but little sense of how people are engaging with what I have to say.
At the same time, maybe this makes me part of the problem of silence on the best writing. I’m also interested in learning to be a better commenter, but I’m not someone who thinks they can or should always have something to say. For my part, I think this mostly indicates that I should comment more with thoughtful questions, but I’m very interested in you or anyone else fleshing out your “being a better commenter” open problem—I think this is potentially more important for success here than writing the right kinds of posts.
Also from Scott, Malthusianisms and Anthropicisms.
I appreciate this perspective! My first instinct is to zoom out from stock phrases to entire ideas or arguments while drafting (when everything is working well, sentences or paragraphs get translated atomically like this), then use ‘close reading’ as an editing tactic. But you’re right that zooming in to find the exact word when stuck on the page can also be very focusing (as it were). And there’s a lot of room for interplay between the two approaches, as far as there’s even a clean separation between self-expression and self-editing in the first place.