silentbob

Fake Blog Posts as a Prob­lem Solv­ing Device

• silentbobAug 23, 2024, 5:58 AM

  1 point

  0

  on: Mu­sic in the AI World

  While much of this can surely happen to varying degrees, I think an important aspect in music is also recognition (listening to the same great song you know and like many times with some anticipation), as well as sharing your appreciation of certain songs with others. E.g. when hosting parties, I usually try to create a playlist where for each guest there are a few songs in there that they will recognize and be happy to hear, because it has some connection to both of us. Similarly, couples often have this meme of “this is our song!”, which throws them back into nostalgic memories of how they first met.

  None of this is to disagree with the post though. I mostly just wanted to point out that novelty and “personal fit” are just two important aspects in any person’s music listening experience, and I think it’s unlikely these two aspects will dominate the future of music that much.

• silentbobAug 12, 2024, 5:55 AM

  10 points

  1

  in reply to: WitheringWeights’s comment on: You don’t know how bad most things are nor pre­cisely how they’re bad.

  I once had kind of the opposite experience: I was at a friend’s place, and we watched the recording of a System of a Down concert from a festival that we both had considered attending but didn’t. I thought it was terrific and was quite disappointed not to have attended in person. He however got to the conclusion that the whole thing was so full of flaws that he was glad he hadn’t wasted money on a ticket.

  Just like you, I was baffled, and to be honest just kind of assumed he was just trying to signal his high standards or something but surely didn’t actually mean that.

  Given that he was quite the musician himself, playing multiple instruments, and I’m quite the opposite, I now for the first time seriously consider whether he really did dislike that concert as much as he said.

• silentbobAug 10, 2024, 7:55 PM

  1 point

  0

  in reply to: wslafleur’s comment on: Failures in Kindness

  I appreciate your perspective, and I would agree there’s something to it. I would at first vaguely claim that it depends a lot on the individual situation whether it’s wise to be wary of people’s insecurities and go out of one’s way to not do any harm, or to challenge (or just ignore) these insecurities instead. One thing I’ve mentioned in the post is the situation of a community builder interacting with new people, e.g. during EA or lesswrong meetups. For such scenarios I would still defend the view that it’s a good choice to be very careful not to throw people into uncomfortable situations. Not only because that’s instrumentally suboptimal, but also because you’re in a position of authority and have some responsibility not to e.g. push people to do something against their will.

  However, when you’re dealing with people you know well, or even with strangers but on eye level, then there’s much more wiggle room, and you can definitely make the case that it’s the better policy to not broadly avoid uncomfortable situations for others.

• silentbobAug 10, 2024, 7:36 PM

  4 points

  3

  in reply to: Gunnar_Zarncke’s comment on: We’re not as 3-Di­men­sional as We Think

  Thanks for sharing your thoughts and experience, and that first link indeed goes exactly in the direction I was thinking.

  I think in hindsight I would adjust the tone of my post a bit away from “we’re generally bad at thinking in 3D” and more towards “this is a particular skill that many people probably don’t have as you can get through the vast majority of life without it”, or something like that. I mostly find this distinction between “pseudo 3D” (as in us interacting mostly with surfaces that happen to be placed in a 3D environment, but very rarely, if ever, with actual volumes) and “real 3D” interesting, as it’s probably rather easy to overlook.

• silentbobAug 10, 2024, 7:30 PM

  1 point

  0

  in reply to: Archimedes’s comment on: We’re not as 3-Di­men­sional as We Think

  I find your first point particularly interesting—I always thought that weights are quite hard to estimate and intuit. I mean of course it’s quite doable to roughly assess whether one would be able to, say, carry an object or not. But when somebody shows me a random object and I’m supposed to guess the weight, I’m easily off by a factor of 2+, which is much different from e.g. distances (and rather in line with areas and volumes).

• silentbobAug 8, 2024, 6:16 AM

  1 point

  0

  in reply to: L Rudolf L’s comment on: Me, My­self, and AI: the Si­tu­a­tional Aware­ness Dataset (SAD) for LLMs

  That github link yields a 404. Is it just an issue with the link itself, or did something change about the dataset being public?

We’re not as 3-Di­men­sional as We Think

• silentbobAug 4, 2024, 1:34 PM

  1 point

  0

  in reply to: Mark Xu’s comment on: Failures in Kindness

  Indeed! I think I remember having read that a while ago. A different phrasing I like to use is “Do you have a favorite movie?”, because many people actually do and then are happy to share it, and if they don’t, they naturally fall back on something like “No, but I recently watched X and it was great” or so.

• silentbobAug 4, 2024, 1:31 PM

  2 points

  1

  in reply to: mikbp’s comment on: Failures in Kindness

  Good point. I guess one could come up with examples that have less of this inefficiency but still are “computationally unkind”. Although in the end, there’s probably some correlation between these concepts anyway. So thanks for adding that. 👌

• silentbobAug 4, 2024, 1:28 PM

  1 point

  0

  in reply to: Unirt’s comment on: Failures in Kindness

  I would add 3) at the start of an event, everyone is asked to state their hopes and expectations about the event. While it’s certainly useful to reflect on these things, I (embarassingly?) often in such situations don’t even have any concrete hopes or expectations and am rather in “let’s see what happens” mode. I still think it’s fair to ask this question, as it can provide very benefitial feedback for the organizer, but they should at least be aware that a) this can be quite stressful for some participants, and b) many of the responses may be “made up” on the fly, rather than statements backed by a sufficient level of reflection. Of course just being honest there and saying “I don’t have any expectations yet and just thought the title of the event sounded interesting” is probably the best option, but I think 10-years-ago-me would probably not have been confident enough to say that, and instead made up some vague plausible sounding claims that had a higher chance of signaling “I’ve got my shit together and definitely thought deeply about why I’m attending this event beforehand”.

• silentbobAug 4, 2024, 1:21 PM

  1 point

  0

  in reply to: magic9mushroom’s comment on: Failures in Kindness

  I think it’s a fair point. To maybe clarify a bit though, while potentially strawmanning your point a bit, my intention with the post was not so much to claim “the solution to all social problems is that sufficiently-assertive people should understand the weaknesses of insufficiently-assertive people and make sure to behave in ways that don’t cause them any discomfort”, but rather I wanted to try to shed some light on situations that for a long time I found confusing and frustrating, without being fully aware of what caused that perceived friction. So I certainly agree that one solution to these situations can be to “tutor the insufficiently-assertive”. But still, such people will always exist in this world, and if you’re, say, a community builder who frequently interacts with new people, then it can still be valuable to be aware of these traps.

• silentbobAug 2, 2024, 3:33 PM

  1 point

  0

  on: “Frac­tal Strat­egy” work­shop report

  Thanks a lot for the write-up, very interesting and a good resource to get back to for future workshops.

• silentbobJul 29, 2024, 8:03 PM

  3 points

  0

  in reply to: Lennart Finke’s comment on: A Vi­sual Task that’s Hard for GPT-4o, but Doable for Pri­mary Schoolers

  I would expect that they fare much better with a text representation. I’m not too familiar with how multimodality works exactly, but kind of assume that “vision” works very differently from our intuitive understanding of it. When we are asked such a question, we look at the image and start scanning it with the problem in mind. Whereas transformers seem like they just have some rather vague “conceptual summary” of the image available, with many details, but maybe not all for any possible question, and then have to work with that very limited representation. Maybe somebody more knowledgeable can comment on how accurate that is. And whether we can expect scaling to eventually just basically solve this problem, or some different mitigation will be needed.

• silentbobJun 25, 2024, 3:31 PM

  3 points

  0

  in reply to: Joel Burget’s comment on: silent­bob’s Shortform

  Maybe I accidentally overpromised here :D this code is just an expression, namely 1.0000000001 ** 175000000000, which, as wolframalpha agrees, yields 3.98e7.

• silentbobJun 25, 2024, 10:30 AM

  44 points

  0

  on: silent­bob’s Shortform

  One crucial question in understanding and predicting the learning process, and ultimately the behavior, of modern neural networks, is that of the shape of their loss landscapes. What does this extremely high dimensional landscape look like? Does training generally tend to find minima? Do minima even exist? Is it predictable what type of minima (or regions of lower loss) are found during training? What role does initial randomization play? Are there specific types of basins in the landscape that are qualitatively different from others, that we might care about for safety reasons?

  First, let’s just briefly think about very high dimensional spaces. One somewhat obvious observation is that they are absolutely vast. With each added dimension, the volume of the available space increases exponentially. Intuitively we tend to think of 3-dimensional spaces, and often apply this visual/​spatial intuition to our understanding of loss landscapes. But this can be extremely misleading. Parameter spaces are utterly incredibly vast to a degree that our brain can hardly fathom. Take GPT3 for instance. It has 175 billion parameters, or dimensions. Let’s assume somewhat arbitrarily that all parameters end up in a range of [-0.5, 0.5], i.e. live in a 175-billion-dimensional unit cube around the origin of that space (as this is not the case, the real parameter space is actually even much, much larger, but bear with me). Even though every single axis only varies by 1 – let’s just for the sake of it interpret this as “1 meter” – even just taking the diagonal from one corner to the opposite one in this high-dimensional cube, you would get a length of ~420km. So if, hypothetically, you were sitting in the middle of this high dimensional unit cube, you could easily touch every single wall with your hand. But nonetheless, all the corners would be more than 200km distant from you.

  This may be mind boggling, but is it relevant? I think it is. Take this realization for instance: if you have two minima in this high dimensional space, but one is just a tiny bit “flatter” than the other (meaning the second derivatives overall are a bit closer to 0), then the attractor basin of this flatter minimum is vastly larger than that of the other minimum. This is because the flatness implies a larger radius, and the volume depends exponentially on that radius. So, at 175 billion dimensions, even a microscopically larger radius means an overwhelmingly larger volume. If, for instance, one minimum’s attractor basin has a radius that is just 0.00000001% larger than that of the other minimum, then its volume will be roughly 40 million times larger (if my Javascript code to calculate this is accurate enough, that is). And this is only for GPT3, which is almost 4 years old by now.

  The parameter space is just ridiculously large, so it becomes really crucial how the search process through it works and where it lands. It may be that somewhere in this vast space, there are indeed attractor basins that correspond to minima that we find extremely undesirable – certain capable optimizers perhaps, that have situational awareness and deceptive tendencies. If they do exist, what could we possibly tell about them? Maybe these minima have huge attractor basins that are reliably found eventually (maybe once we switch to a different network architecture, or find some adjustment to gradient descent, or reach a certain model size, or whatever), which would of course be bad news. Or maybe these attractor basins are so vanishingly small that we basically don’t have to care about them at all, because all the computer & search capacity of humanity over the next million years would have an almost 0 chance of ever stumbling onto these regions. Maybe they are even so small that they are numerically unstable, and even if your search process through some incredible cosmic coincidence happens to start right in such a basin, the first SGD step would immediately jump out of it due to the limitations of numerical accuracy on the hardware we’re using.

  So, what can we actually tell at this point about the nature of high dimensional loss landscapes? While reading up on this topic, one thing that constantly came up is the fact that, the more dimensions you have, the lower the relative number of minima becomes compared to saddle points. Meaning that whenever the training process appears to slow down and it looks like it found some local minimum, it’s actually overwhelmingly likely that what it actually found is a saddle point, hence the training process never halts but keeps moving through parameter space, even if the loss doesn’t change that much. Do local minima exist at all? I guess it depends on the function the neural network is learning to approximate. Maybe some loss landscapes exist where the loss can just get asymptotically closer to some minimum (such as 0), without ever reaching it. And probably other loss landscapes exist where you actually have a global minimum, as well as several local ones.

  Some people argue that you probably have no minima at all, because with each added dimension it becomes less and less likely that a given point is a minimum (because not only does the first derivative of a point have to be 0 for it to be a minimum, also all the second derivatives need to be in on it, and all be positive). This sounds compelling, but given that the space itself also grows exponentially with each dimension, we also have overwhelmingly more points to choose from. If you e.g. look at n-dimensional Perlin Noise, its absolute number of local minima within an n-dimensional cube of constant side length actually increases with each added dimension. However, the relative number of local minima compared to the available space still decreases, so it becomes harder and harder to find them.

  I’ll keep it at that. This is already not much of a “quick” take. Basically, more research is needed, as my literature review on this subject yielded way more questions than answers, and many of the claims people made in their blog posts, articles and sometimes even papers seemed to be more intuitive /​ common-sensical or generalized from maybe-not-that-easy-to-validly-generalize-from research.

  One thing I’m sure about however is that almost any explanation of how (stochastic) gradient descent works, that uses 3D landscapes for intuitive visualizations, is misleading in many ways. Maybe it is the best we have, but imho all such explainers should come with huge asterisks, explaining that the rules in very high dimensional spaces may look much different than our naive “oh look at that nice valley over there, let’s walk down to its minimum!” understanding, that happens to work well in three dimensions.

  What links here?

  • We’re not as 3-Di­men­sional as We Think by silentbob (Aug 4, 2024, 2:39 PM; 46 points)
  • silentbob's comment on List of most in­ter­est­ing ideas I en­coun­tered in my life, ranked by Lucien (Feb 25, 2025, 10:22 AM; 9 points)

silent­bob’s Shortform

• silentbobJun 25, 2024, 5:39 AM

  1 point

  0

  in reply to: slicko’s comment on: What are your great­est one-shot life im­prove­ments?

  So how did it work out for you?

Ap­ply­ing Force to the Wrong End of a Causal Chain

• silentbobJun 2, 2024, 6:25 AM

  2 points

  1

  in reply to: edge_retainer’s comment on: OpenAI: Fallout

  That seems like a rather uncharitable take. Even if you’re mad at the company, would you (at least (~falsely) assuming this all may indeed be standard practice and not as scandalous as it turned out to be) really be willing to pay millions of dollars for the right to e.g. say more critical things on Twitter, that in most cases extremely few people will even care about? I’m not sure if greed is the best framing here.

  (Of course the situation is a bit different for AI safety researchers in particular, but even then, there’s not that much actual AI (safety) related intel that even Daniel was able to share that the world really needs to know about; most of the criticism OpenAI is dealing with now is on this meta NDA/​equity level)