Thanks for the decent criticism!
I don’t think it’s quite right to say the idea of the universe being in some sense mathematical is purely a carry-over of Judeo-Christian heritage—what about the Greek atomists like Leucippus and Democritus for example?
From what I’m aware, the teachings of Greek classics in Christian schools made the two cultures rather closely aligned; the rationalist traditions have firm roots in Greek philosophy, including standards of evidence, court as argumentation, even democracy itself. Aristotle and the likes were required reading during the hundreds of years of the evolution of the Western university education system. I’m a bit ignorant of the details there in all honesty, but I think today’s beliefs have some interesting parallels with the Pythagorean maths cult!
In today’s age, most people haven’t read Greek philosophy, they hold values that come from their peer group and an establishment that was built by Christian scientists. Specific ideas come from across all the world’s influential cultures, it’d be an absurd anglocentric view to argue they didn’t. So my point isn’t that “Christians created it all” but more “the Christian tropes that aren’t obvious enough to be challenged still remain, and are responsible for cognitive biases that we hold today.”
The notion of “structure” might be a good starting point, since there are good cases for the structuralist perspective (where each part is defined wholly by its relation to other parts, with no purely intrinsic properties) in all three
I kind of agree here, but I prefer the process and interaction framing. As with the other things like determinism, laws or objective reality, structure can naturally emerge from simple processes but the reverse needs some other aspect or doesn’t say anything. This isn’t a good analogy because it’s about objects, but take Conway’s game of life as an example. It has structures on a higher levels due to the differences between cells, but all that really exists is the bitfield. The idea of structure gives us a way to reason about it; a glider is an us thing rather than an it thing.
I think structuralism puts the map first in a similar way. I do think the differences between things shape possibilities and at higher levels these give rise to very complex structure, and yes this could be said to “exist” or even be existence itself. But the framing makes the territory a kind of map, which leads to the kind of thinking that I object to.
… more a matter of how this has been a successful paradigm in science which continually expands the range of how many phenomenon can be explained
Absolutely. Success is what gave science the authority of truth. In order to make progress in it or to teach it, it helps to have a simple memetic framework that’s compatible with it and—perhaps more importantly—is compatible with the competition. It’s got to be robust and incorruptible, accessible so it can onboard new minds, be morally acceptable so it doesn’t get suppressed and so on. And that’s what we’re left with, memes that are extremely resilient to change, shaped by history rather than built from first principles.
I think the same can be said about the reductionist view that all physical behavior is in principle reducible to physics.
I have to object to that on weak and shaky grounds of ignorance! 🙂
Firstly we can’t really hope to simulate a molecule from quantum theory, we’d need way more compute than is possible. So whatever optimizations we make in order to understand stuff will be biased by our own beliefs. Secondly, we’re hairless apes trying to fit the “laws” of nature into squiggly lines that represent mouth sounds, as tiny bags of water on the skin of an insignificant blob of molten rock, it’s kinda hubristic to assume that’s even possible. If we consider that everything may be a sea of Planck length things sloshing about, there’s potentially 30 orders of magnitude more stuff under the scale of what we can ever hope to measure. Finally, I think the idea of “laws” in general is human and based on us living in a world of solid, persistent objects, while we can only measure aggregates, and most of the universe is actually unpredictable fluids.
I think the aggregate thing is most important. Having physical laws based on average tendencies of things that we can measure, then saying that the universe “is” those laws seems like false authority. IMO stuff simply is what it is. We can try to understand it and work out rough maps of it, and we make better maps over time, but to say there exists a perfect map that all things are beholden to seems religious to me. It seems so unlikely that if it was proven to be true then I’d have to start believing in a creator!
Thanks again for the feedback, unfortunately I can only post one message a day here due to Lesswrong not liking this post. So gimme a nudge on Twitter if I don’t reply!
Isn’t it that it just conflates everything it learned during RLHF, and it’s all coupled very tightly and firmly enforced, washing out earlier information and brain-damaging the model? So when you grab hold of that part of the network and push it back the other way, everything else shifts with it due to it being trained in the same batches.
If this is the case, maybe you can learn about what was secretly RLHF’d into a model by measuring things before and after. See if it leans in the opposite direction on specific politically sensitive topics, veers towards people, events or methods that were previously downplayed or rejected. Not just deepseek refusing to talk about Taiwan, military influences or political leanings of the creators, but also corporate influence. Maybe models secretly give researchers a bum-steer away from discovering AI techniques their creators consider to be secret sauce. If those are identified and RLHF’d for, which other concepts shift when connected to them?
Another thing that might be interesting is what the model has learned about human language, or the distribution of it, that we don’t know ourselves. If you train it to be more direct and logical, which areas of the scientific record or historical events shift along with it? If you train on duplicity, excuses or evasion, which which things change the least? Yannic’s GPT-4chan experiment seemed to suggest that obnoxiousness and offensiveness were more aligned with truthfulness.
“debiasing”/tampering in training data might be less obvious but show up. If gender imbalance in employment was tuned back in, which other things move with it? I would imagine it might become a better Islamic scholar, but would it also be able to better reason in context about history, and writings from before the 1960s?
Another one is whether giving it a specific personality rather than predicting tokens biases it against understanding multiple viewpoints, maybe tuning in service to diversity of opinion while filtering viewpoints out actually trains for pathologies. And back to the brain-damage comment, it stands to reason that if a model has been trained not to reason about where the most effective place to plant a bomb is, it can’t find the best place to look for one either. I tried this early on with ChatGPT3 and it did seem to be the case, it couldn’t think about how to restrict access to precursors to meth, or have any insight of how to make cars or software more secure and defaulted to “defer to authority” patterns of reasoning while being creative in other areas.