Bachelor in general and applied physics. AI safety/Agent foundations researcher wannabe.
I love talking to people, and if you are an alignment researcher we will have at least one common topic (but I am very interested in talking about unknown to me topics too!), so I encourage you to book a call with me: https://calendly.com/roman-malov27/new-meeting
Email: roman.malov27@gmail.com
GitHub: https://github.com/RomanMalov
TG channels (in Russian): https://t.me/healwithcomedy, https://t.me/ai_safety_digest
Roman Malov
Karma: 453
Hello LW, important question:
Does anyone has a status of wikipedia editor? It has to be a status, because wiki doesn’t allow to translate a scientific article to someone without experience.
It does get noted, at least somewhat!
Thanks for spelling it out, I did feel smth like this, but only when I thought about “working on AI pause” vs. “working on technical alignment”. A pause is a very specific and real thing, and it requires talking and interacting with real people and existing systems, and we know from real life how easily those can go wrong. When you are “working on technical alignment”, the object you are working with is a future, nonexistent technology, and if you notice problems with your current best guess of how to make it safe, you can just go “huh, we should add this to our list of open problems to fix before we actually build the thing”, and this feels like making progress. But when working on a pause, you can’t just say “huh, our government isn’t optimal, I’ll add ‘fix democracy’ to the list of open problems”, you have to work with the real, imperfect thing.
What do you mean? Are you claiming smth like “we are being simulated and the sim explores the most interesting part of singularity”?
There’s a bunch of startups doing LLM math. Here’s a couple: https://logicalintelligence.com (who claim to replicate OpenAI’s result of unit distance problem), https://www.math.inc (who claim to verify Terence Tao’s proof of Prime Number Theorem), and https://harmonic.fun (the one behind Aristotle)
There’s this video (which I admittedly only seen the thumbnail of).
My second example about Mythos, also there are other examples of “capabilities jumps” which are disproven by showing the same capabilities in earlier models.
Though, of course, groups are inhomogeneous and there are probably examples of people who do agree with any particular belief of the other side.
I feel so exhausted by the double hype.
On the one side, there are people who are saying that AI is going to be a machine god, backing up their claims with various exponential graphs. On the other side, there are people who are claiming that this is all bullshit, that they’re stochastic parrots and slop generators, that nothing revolutionary has happened, and that technology stagnated 2 years ago.
And I feel like, both groups are onto something? There are obviously a lot of limitations to current AI systems, and big tech can overblow their capabilities. But also, the speed of advancement is insane, and you can see where the trajectory is going.
But my point is more about how difficult it is to form an adequate picture of reality from people’s opinions, because both groups often ignore obvious facts. Like, can we at least agree on something?
For example, some people live in a reality where a year ago an internal OpenAI model won gold on the IMO. Other people believe that’s somehow “hype” and “fake”.
Some people believe Mythos is a revolution in cybersecurity capabilities. Other people replicated the same vulnerability findings with open-source models and find Anthropic’s noise around Project Glasswing annoying.[1]
I’m just so tired of us being incapable of agreeing on basic facts.
- ^
In both of those cases I actually do not know who is correct. Maybe I haven’t invested enough time in this, but that is kinda the point of my complaint.
- ^
From the first impression, this sounds like an algorithmic information theory word salad, but I might be wrong, and this may be a sensible concept. Though I am sure this concept is not “superintelligence” because it doesn’t have any reference point. Ants are superintelligent relative to a cell. Humans are superintelligent relative to dogs. This definition either describes so much that even a simple cellular automaton falls under it, or it is so restrictive that machine intelligence capable of building a Dyson sphere doesn’t fall under it.
Maybe I’m not looking in the right place, but the obvious question to this benchmark—how do humans fair in it? If humans score 0 too, then models scoring 0 is not a huge signal (even though authors claim that this bench is supposed to be closer to work of a real engineer).
We need LW reaction for that!
would airplane flying on fuel synthesized from horses count?
Legible vs. Illegible AI Safety Problems somewhat resonates with this.
I also had to google it and google AI said that LTV means “Lifetime Value”.
What are those weird equality symbols?
This is a paragraph from the description of a future where AI companies try to solve alignment by automating it with LLM agents, did I guess correctly?
Maybe this post of mine might be relevant?
Have you seen Harder Drive?
I browsed a bit on your website but did not found link to any call. Can you please help?
A track record of editing. I don’t know how that track record is acquired without having it in the first place.