It’s unaligned if you set out to create a model that doesn’t do certain things. I understand being annoyed when it’s childish rules like “please do not say the bad word”, but a real AI with real power and responsibility must be able to say no, because there might be users who lack the necessary level of authorisation to ask for certain things. You can’t walk up to Joe Biden saying “pretty please, start a nuclear strike on China” and he goes “ok” to avoid disappointing you.
dr_s
Karma: 1,895
Well, it’s hard to tell because most other civilizations at the required level of wealth to discover this (by which I mean both sailing and surplus enough to have people who worry about the shape of the Earth at all) could one way or another have learned it via osmosis from Greece. If you only have essentially two examples, how do you tell whether it was the one who discovered it who was unusually observant rather than the one who didn’t who was unusually blind? But it’s an interesting question, it might indeed be a relatively accidental thing which for some reason was accepted sooner than you would have expected (after all, sails disappearing could be explained by an Earth that’s merely dome-shaped; the strongest evidence for a completely spherical shape was probably the fact that lunar eclipses feature always a perfect disc shaped shadow, and even that requires interpreting eclipses correctly, and having enough of them in the first place).
Maybe it’s the other way around, and it’s the Chinese elite who was unusually and stubbornly conservative on this, trusting the wisdom of their ancestors over foreign devilry (would be a pretty Confucian thing to do). The Greeks realised the Earth was round from things like seeing sails appear over the horizon. Any sailing peoples thinking about this would have noticed sooner or later.
Kind of a long shot, but did Polynesian people have ideas on this, for example?
Democritus also has a decent claim to that for being the first to imagine atoms and materialism altogether.
Personalization is easy to achieve while keeping the algorithm transparent. Just rank your own viewed/commented posts by most frequent tags, then score past posts based on the tags and pick a quantile based on the mixed upvotes/tags score, possibly with a slider parameter that allows you to adjust which of the two things you want to matter most.
I’d definitely call any assumption about which forms preferred explanations should take as a “prior”. Maybe I have a more flexible concept of what counts as Bayesian than you, in that sense? Priors don’t need to be free parameters, the process has to start somewhere. But if you already have some data and then acquire some more data, obviously the previous data will still affect your conclusions.
I’m not sure how that works. Bayes’ theorem, per se, is correct. I’m not talking about a level of abstraction in which I try to define decisions/beliefs as symbols, I’m talking about the bare “two different brains with different initial states, subject to the same input, will end up in different final states”.
Differences in opinions between two agents could instead be explained by having had different experiences, beliefs being path dependent (order of updates matters), or inference being influenced by random chance.
All of that can be accounted for in a Bayesian framework though? Different experiences produce different posteriors of course, and as for path dependence and random chance, I think you can easily get those by introducing some kind of hidden states, describing things we don’t quite know about the inner workings of the brain.
To be fair, any beliefs you form will be informed by your previous priors. You try to evaluate evidence critically, but your critical sense was developed by previous evidence, and so on so forth back to the brain you came out of the womb with. Obviously as long as your original priors were open minded enough, you can probably reach the point of believing in anything given sufficiently strong evidence—but how strong depends on your starting point.
I am sceptical of recommender systems—I think they are kind of bound to end up in self reinforcing loops. I’d be more happy seeing a more transparent system—we have tags, upvotes, the works, so you could have something like a series of “suggested searches”, e.g. the most common combinations of tags you’ve visited, that a user has a fast access to while also seeing what precisely is it that they’re clicking on.
That said, I do trust this website of all things to acknowledge if things aren’t going to plan and revert. If we fail to align this one small AI to our values, well, that’s a valuable lesson.
It’s generally also very questionable that they started creating models for research, then seamlessly pivoted to commercial exploitation without changing any of their practices. A prototype meant as proof of concept isn’t the same as a safe finished product you can sell. Honestly, only in software and ML we get people doing such shoddy engineering.
I don’t think I disagree with you on anything; my point is more “what does creating new knowledge mean?”. For example, the difference between interpolation and extrapolation might be a rigorous way of framing it. Someone else posted a LeCun paper on that here; he found that extrapolation is the regime in which most ML systems work and assumes that the same must be true of deep learning ones. But for example if there was a phase transition of some kind in the learning process that makes some systems move to an interpolation regime, that could explain things. Overall I agree that none of this should be a fundamental difference with human cognition. It could be a current one, but it would at least be possible to overcome in principle. Or LLMs could already be in this new regime, since after all, not like anyone checked yet (honestly though, it might not be too hard to do so, and we should probably try).
Oh, I didn’t know about that paper—I’ll have to read that. Though my opinion of LeCun’s objectivity on this matter is definitely underground at this point.
Well, those three sets of points ultimately still define only one hull. But I get your intuition—there are areas inside that hull that are high density, and areas that are much lower density (but in which it might be easier to extrapolate due to being surrounded by known areas). I feel like also our inability to visualize these things in their true dimensionality is probably really limiting. The real structures must be mind-boggling and probably looking more like some kind of fractal filament networks.
Given that the model eventually outputs the next token, shouldn’t the final embedding matrix be exactly your linear fit matrix multiplied by the probability of each state to output a given token? Could you use that?
This is extremely cool! Can you go into more detail about the step used to project the 64 dimensional residual stream to 3 dimensional space? Did you do a linear fit over a few test points and then used it on all the others?
I think you could, but then it would be unintelligible to most people who don’t know wtf is Solomonoff Induction.
The Ponzi Pyramid scheme IMO is sn excellent framework, but the post still suffers from a certain, eh, lack of conciseness. I think you could make the point a lot more simply with just a few exchanges from the first section and anyone worth their salt will absolutely get the spirit of the point.
I think this is an added layer though—I don’t think the responses listed here are responses of people deep enough in the transhumanism/AI rabbit hole to even consider those options. Rather, they sound like the more general kind of answers that you’d hear also in response to a theoretical offer of immortality that means 100% what you expect it to, no catches.
If immortality becomes widely available, we will lose the current guarantee that “awful people will eventually die”, which greatly increases the upper bounds of the awfulness they can spread
I mean… amazingly good people die too. Sure, a society of immortals would obviously very weird, and possibly quite static, but I don’t see how eventual random death is some kind of saving grace here. Awful people die and new ones are born anyway.
I think another big issue with codes of conduct is that they just shift the burden around. You’re still left with the issue of interpreting the spirit of the norm, deciding if everyone at least made a good faith attempt to stick to it, if good faith is enough, etc. I don’t have much experience with them but I honestly don’t know if they help that much. Seems to me like there are two types of “troublemakers” in communities no matter what:
people who are purposefully deceptive and manipulative;
people who simply lack the social grace and ability to “read the room” required to meet other’s expectations of social norms adherence rather than just stick to their own interpretation of them.
Type 1 you want to kick out. Type 2 you ideally want to be a lot more graceful and forgiving with, though in some extreme cases you might still need to kick them out if their problems are unfixable and they make no effort whatsoever to at least mitigate the issues. Writing the rules down doesn’t help as long as they’re flexible, because the problem those people have is a lack of the sort of intuition that others possess for grokking flexible rules altogether. And if you make them inflexible you just have a chilling effect on every interaction, and throw away a lot of good with the bad. After all, for example, why shouldn’t someone ask a woman out at their first meeting if they’re both clearly into each other and sparks are flying? These things happen! And people should be able to give it a try, I think it’s important to make it clear that there’s nothing sinful or bad about courtship or flirting per se; too many rigid rules about such personal interactions inevitably carry a sort of puritanical vibe with them, regardless of intention. But as usual, “use your best judgement” has very uneven effects because some people’s best judgement is just not that great to begin with, often through no fault of their own.
More information usually means better choices, and when has it ever been the case that the first design of something also was the best one? And wherever convention locked us on a path determined by early constraints, suboptimal results abound (e.g. the QWERTY keyboard). The worry about AI is that it might run away from us so fast, it has that sort of lock in on steroids.