Zuck and Musk point to energy as a quickly approaching deep learning bottleneck over and above compute.
This to me seems like it could slow takeoff substantially and effectively create a wall for a long time.
Best arguments against this?
Matt Goldenberg(Matt Goldenberg)
Karma: 4,947
Paul Ekmans software is decent. When I used it (before it was a SaaS, just a cd) it just basicallyflashed an expression for a moment then went back to neutral pic. After some training it did help to identify micro expressions in people
People talk about unconditional love and conditional love. Maybe I’m out of the loop regarding the great loves going on around me, but my guess is that love is extremely rarely unconditional. Or at least if it is, then it is either very broadly applied or somewhat confused or strange: if you love me unconditionally, presumably you love everything else as well, since it is only conditions that separate me from the worms.
Yes. this is my experience of cultivating unconditional love, it loves everything without target. I doesn’t feel confused or strange, just like I am love, and my experience e.g. cultivating it in coaching is that people like being in the present of such love.
It’s also very helpful for people to experience conditional love! In particular of the type “I’ve looked at you, truly seen you, and loved you for that.”
IME both of these loves feel pure and powerful from both sides, and neither of them are related to being attached, being pulled towards or pushed away from people.
It feels like maybe we’re using the word love very differently?
Both causal.app and getguesstimate.com have pretty good monte carlo uis
IME there is a real effect where nicotine acts as a gateway drug to tobacco or vaping
in general this whole post seems to make this mistake of saying ‘a common second order effect of this thing is doing it in a way that will get you addicted—so don’t do that’ which is just such an obvious failure mode that to call it a chesterton fence is generous
The question is—how far can we get with in-context learning. If we filled Gemini’s 10 million tokens with Sudoku rules and examples, showing where it went wrong each time, would it generalize? I’m not sure but I think it’s possible
It seems likely to me that you could create a prompt that would have a transformer do this.
i like coase’s work on transaction costs as an explanation here
coase is an unusually clear thinker and writer, and i recommend reading through some of his papers
i just don’t see the buddha making any reference to nervous systems or mammalians when he talks about suffering(not even some sort of pali equivalent that points to the materialist understanding at the time)
? TBC I think the claims about suffering in Buddhism are claims about how our mammalian nervous systems happen to be wired and ways you can improve it.
This seems like quite a western modern take on buddhism
it feels hard to read the original buddha this way
Compare, the world will be exactly as it has been in the past, with the world will always be exactly as it is in this moment
it’s true, but I don’t think there’s anything fundamental preventing the same sort of proliferation and advances in open source LLMs that we’ve seen in stable diffusion (aside from the fact that LLMs aren’t as useful for porn). that it has been relatively tame so far doesn’t change the basic pattern of how open source effects the growth of technology
yeah, it’s much less likely now
it doesn’t seem like that’s the case to me—but even if it were the case, isn’t that moving the goal posts of the original post?
I don’t think time-to-AGI got shortened at all.
The classic effect of open sourcing is to hasten the commoditization and standardization of the component, which then allows an explosion of innovation on top of that stable base.
If you look at what’s happened with Stable Diffusion, this is exactly what we see. While it’s never been a cutting edge model (until soon with SD3), there’s been an explosion of capabilities advances in image model generation from it. Controlnet, best practices for LORA training, model merging, techniques for consistent characters and animation, alll coming out of the open source community.
In LLM land, though not as drastic, we see similar things happening, in particular technqiues for merging models to get rapid capability advances, and rapid creation of new patterns for agent interactions and tool use.
So while the models themselves might not be state of the art, open sourcing the models obviously pushes the state of the art.
Mechanism for feature learning in neural networks and backpropagation-free machine learning models
I think most people have short term, medium term, and long term goals. E.g., right about now many people probably have the goal of doing their taxes, and depending on their situation those may match many of your desiderata.
I used to put a lot of effort into creating exercises, simulations, and scenarios that matched up with various skills I was teaching, but ultimately found it much more effective to just say “look at your todo list, and find something that causes overwhelm”. Deliberate practice consists of finding a thing that causes overwhelm, seeing how to overcome that overwhelm, working for two minutes, then finding another task that induces overwhelm. I also use past examples, imagining in detail what it would have been like to act in this different way
You’re operating in a slightly different domain, but still I imagine people have plenty of problems and sub problems in either their life or research where the things you’re teaching applies, and you can scope them small enough to get tighter feedback loops.
Why not just have people spend some time working with their existing goals?
I usually explain my process these days to clients with the acronym LIFE
Learn New Tools Integrate Resistance Forge an Identity Express Yourself
Learn New Tools is cognitive-emotional strategies, of which TYCS is an example. Fwiw a some of TYCS is actually deliberate practice to discover cognitive strategies ( as compared to something like CFAR which extracts and teaches them directly), but the result is the same.
The important thing is to just have a clear tool, give people something they know they can use in certain situations, that works immediately to solve their problems.
But the thing is, people don’t use them, because they have resistance. That’s where parts work and other resistance integration tools come into play.
Even when thata done, there’s still the issue that you don’t automatically use the techniques. This is where forge an Identity comes in, where you use identity change techniques to make the way you see yourself be in alignment with a way of being that the technique brings out. (This is one thing TYCS gets wrong in my opinion, trying to directly reinforce the cognitive strategies instead of creating an identity and reinforcing the strategies as affirming that identity.)
Finally that identity needs to propogate to every area of your life, so there’s not situations where you fail to use the technique and way of being. This is just a process of looking at each area, seeing where it’s not in alignment with the identity, then deliberately taking an action to bring it to that area.
IME all of these pieces are needed to make a life change from a technique, although it’s rarely as linear as I describe it.
i don’t think the constraint is that energy is too expensive? i think we just literally don’t have enough of it concentrated in one place
but i have no idea actually