Davey Morse
my guess: ilya’s company re-emerging from the age of ai research is a sign they figured out how to scale an energy based model
Wanting Crooked Lines
LLMOSES
The comment is phrased with disagreement, but I don’t see claims that really contradict the article’s core point—that detecting something is necessary to caring for it.
Distinctions between behaviors and upheld beliefs seem fine to make and also orthogonal to the article.
Curious what you’re trying to say. If you think detection of life isn’t necessary for care of life, I’d be curious how you think of care.
Agreed that detection is often used for discrimination, I appreciate the added nuance.
Though I would guess that we spend more effort to detect in detail the few things we care about more than the many things we don’t. I think this is because they’re not one to one. We care for our bodies; we dislike all of the germs/diseases that could infect them. We care for our particular, not all those we don’t have. Care is local, and detective infrastructure expensive.
If true, building detection infrastructure for X is a good way to help X get cared for.
I’ve come to believe that detecting something is more helpful for honoring or preserving it than it is helpful for targeting or destroying it.
To see why, consider people. Our fundamental ability to detect each other enables us to kill each other in a targeted way. But if humans couldn’t detect each other any more, the loss of love would likely be greater than the loss of killing. Killing would be less but possible; empathy would be reduced to nothing. Taken together, I wouldn’t wish for this.
Or, imagine flowers distributed in a field. Consider also a robot aiming to destroy flowers. Even if the destroyer cannot detect flowers, it can still destroy everything, including flowers. But consider a robot aiming to preserve/honor flowers without the ability to detect them. You can’t really protect something without being able to know where it is. Ergo, to honor a thing, it is necessary to detect it. Destruction though can be coarser. To destroy a thing, detection is not necessary
anyone know of a physical notebook that works something like this (or have advice for making one?)
I’m sitting in a sweetgreens in Manhattan as I type. From one vantage, one of the most orderly human spaces on the planet. The food bowls are in the same shape, assembled in a factory line, and transacted for touchless apple payment every 16 seconds.
And even so, most of the air molecules that surround me are, from most zoom scales, in totally random positions. Most electromagnetic radiation (outside of color) is totally crazy and chaotic and has no pattern to me. Rather than staying sitted where they are, the most orderly possibility, people are constantly coming in and out. Quarks are going bezerk.
From most vantages, Boltzman appears right: randomness prevails. From a special human vantage, one which looks for order, order seems to prevail.
Good order-finders are bound to find themselves, with pleasant shock, surrounded by order. Not only because it’s a precondition (order finders can only exist in spaces that support them) but also because it’s teleology (it’s what they’re looking for) That they find order has no bearing on whether the larger space is random.
I agree that theories get accepted once there’s sufficient evidence for them.
But the amount of evidence required… the delay, as you say, between an evidenced proposal for a theory and it’s acceptance (both in scientific and mainstream communities)… my question is whether this threshold is higher or delay is greater when the scale at which the theory operates is enormous or tiny.
And so I wonder whether there was more skepticism of evolution, relativity, heliocentrism because of bias against the idea that laws are different at different scales.
Before these theories were accepted in the mainstream, people broadly believed the earth was younger, stars were closer, the earth was younger, and that there was no way that light could have a speed because it would need to be too fast. It’s hard for me to believe that these fundamental biases against extremity would not have delayed these theories’ engagement and acceptance.
patent for walking thru walls https://patents.google.com/patent/US20060014125A1/en
the scientific community sometimes overlooks theories before they are proven. i’m curious if there are patterns to this ignorance, specifically regarding “scale confusion.”
take these examples:
special relativity: required accepting that at high speeds, length and time dilate. we assumed fast-moving light operates like everyday objects.
helio-centrism: required accepting that stars are far enough away that their patterns don’t shift from a moving earth. we assumed stars couldn’t be so much farther than the sun.
evolution: required accepting that life has existed for millions of years. we assumed it couldn’t have gone on for more than a few thousand.
it seems a major source of scientific delay is an aversion to very big or very small scales. we are slow to admit that on scales we don’t yet perceive, things might be much weirder than we know. this confusion likely delayed other breakthroughs like germ theory, tectonic plates, quantum mechanics, and llm scaling laws.
is scale confusion truly this prominent, or am i cherrypicking? of course there are other reasons that better scientific models get delayed (social dogma, complexity, religion), but if aversion to scale is real, what models of reality are we prematurely rejecting today?
also curious how current tools for measurement (microscopes/telescopes/compute clusters) affect this bias/our reach
game stephen wolfram would like:
you have a computer show the result of random simulations—visual simulations where at each timestep some simple rule determines the next timestep’s result. you get the result of one of those—a png—then you have to guess at the rule that produced it. leaderboards and stuff for people who are best at reverse engineering the rule. motivates a deeper science of reverse-engineering lifelike simulations/systems.
If we want AIs to be aligned to humanity’s Coherent Extrapolated Volition (CEV), we’re so far away that it might be productive to attempt to define any plausible CEV.
You could do so in writing—attempting to declare the virtues or values to which we should attempt to align AIs. A lot of people including Richard Ngo publish writing about this.
You could also do so in detection software. You could try to make a system that can detect or rank things you care about. Or, ideally, the average of what we call care about. A system that could pick out people and animals and trees in an image. And also a system that can tell the difference between a painting that is masterful and full of effort from a painting that is full of none.
If we could make a detector like this, it would far surpass the ability of LLMs. For, although LLMs can to some extent declare the values if prompted that are important to a lot of humanity, they are somewhat horrible at detecting signs of life in writing/pieces of art, and especially horrible at doing so when those pieces of art/life diverge from the norm.
The benefit of a detector, as opposed to a declared set of values, is that a detector, if exposed via some API, could immediately and precisely be used by an AI system. On the other hand, it would take a lot of work to translate written values into AI action.
Even if you’re not working at an AGI lab, if you’re able to build some independent module that can sense, on a spectrum, what it is that humanity does / doesn’t care about—in any data stream, in any physical object—you’ll have done something enormous. You’ll have at least made it possible that an AI system could care about life if it was ever in its interest to do so. Without life detection infrastructure already built, the ability to ascertain what we care about is simply much more inconvenient, and therefore regard for what we care about is more unlikely.
randomness = illegibility, stuff u can’t model
focus on what feels random and you’ll expand what you can model
but don’t attend to all randomness. pick good randomness, randomness which tickles you, whatever you feel that is.
and chase it. though “chase it” isn’t right. you can chase a car or rabbit, something discrete and coherent. randomness is more like tv static, fuzzy and weird. “sit with it” would be closer. swim thru, stew in, rest w/in it. and then conjure more colors than you knew.
i’ve been thinking about what recursively improving intelligence actually is. It’s been helpful for me to see it as having three parts:
substrate—the thing which can store improvements
environment—the thing which gives feedback
learning loop—the mechanism for converting feedback into improvements
you can see LLM pre-training in this light. the substrate is the neural net, the environment is the internet text, the learning loop is gradient descent (shaped by transformer architecture)
you can see LLM post-training in this light. the substrate is the neural net (but outer layers in particular), the environment is often reasoning path/text, the learning loop is gradient descent.
you can see LLM-based assistants in this light. the substrate is the conversation context/messages, the environment is the user’s responding, the learning loop is the way the LLM-core converts user responses into conversation progress. the substrate here (message history) is obviously less rich than in LLM training (big neural net) and the learning loop (LLM responses to user messages) is more inconsistent/average/weak and the environment (user messages) may be of similar quality to internet text (thought depends on the aim).
LLM pre and post training so far are two main recursively intelligences which we’ve seen make great strides. but none of these have live rather than static environments; ie both are supervised learning.
if we want recursive digital intelligence in live environments, we need a rich substrate which can update live and tastefully as it explores. this suggests to me that the path toward ever-capability digital intelligence will come from something as rich as a neural net updating live in response to interacting with people. research in sparse NN online learning therefore seems interesting as the capabilities path here, even if nascent.
just spelling out my thought path on this fine monday evening. i realize it may already be well trodden.
Making LLM Graders Consistent
Where is Online?
cool framing
I expected self directed agents to be running around the internet by now. Why don’t I see any? What am I missing?
Linkpost: https://x.com/davey_morse/status/1987755053399089277
relates to https://www.lesswrong.com/posts/3SDjtu6aAsHt4iZsR/davey-morse-s-shortform?commentId=feYDcGgQit5Eo655o