Conor Sullivan

Karma: 395

Conor Sullivan 25 May 2022 3:11 UTC
6 points
in reply to: michael_mjd’s comment on: autonomy: the missing AGI ingredient?
My opinion is that you’re not going to be able to crack the alignment problem if you have a phobia of infohazards. Essentially you need a ‘Scout Mindset’. There’s already smart people working hard on the problem, including in public such as on podcasts, so realistically the best (or worst) could do on this forum is attempt to parse out what is known publicly about the scary stuff (eg agency) from DeepMind’s papers and then figure out if there is a path forward towards alignment.

Conor Sullivan 25 May 2022 2:53 UTC
9 points
in reply to: Nathan Helm-Burger’s comment on: autonomy: the missing AGI ingredient?
It might be that the ‘goals’ part of what nostalgebraist is waving at is separable from the temporal coherence part.

I mean this in the sense that GPT-3 doesn’t have goals; obviously you can debate that one way or another, but consider all the attempts to make a long document transformer. The generative models have two kinds of memory, essentially: the weights, analogous to long term memory, and the context window, analogous to working memory. There either needs to be some kind of continuous training/fine-tuning of the weights in production, or it needs a third form of episodic memory where the AI can remember the context (“War with Eurasia ended, we have always been at war with Eastasia”). These improvements could make GPT-3 able to write a coherent novel, plausibly without making it any more agentic.

Conor Sullivan 25 May 2022 2:42 UTC
8 points
on: autonomy: the missing AGI ingredient?
(Somewhat rambly response.) You’re definitely on to something here. I’ve long felt that while deep learning may not be AGI, the machine learning industry sure seems ‘general’ in its ability to solve almost any well defined problem, and the only limitation is that the most difficult problems cannot be so easily put into a series of testcases. Once there is some metric to guide them, the ML guys can arrive at seemingly any destination in a relatively short order. But there are still some problems outside their reach where we can at least attempt to define a score:
- Nethack and other complex games. The ideal game for testing autonomy under nostalgebraist’s definition would show a lot of information, most of which is irrelevant, but at least some of which is very important, and the AI has to know which information to remember (and/or how to access relevant information e.g. via game menus or the AI’s own database of memories)
- Strong versions of the Turing Test, especially where the challenge is to maintain a stable personality over a long conversation, with a consistent (fake) personal history, likes and dislikes, things the personality knows and doesn’t know, etc.
- Autonomously create a complex thing: a long novel, a full-length movie, or a complex computer program or game. A proof of success would be getting a significant amount of attention from the public such that you could plausibly earn money using AI-generated content.
- The Wozniak Test: make and present a cup of coffee using the tools of an unfamiliar kitchen. Given that pick-and-place is now relatively routine, this seems within reach to me, but it would require chaining together multiple pick-and-place tasks in a way that robotics AIs really struggle with today. I would not be surprised if, within the next year, there is an impressive demonstration of this kind of a robomaid task, seeming passing the Wozniak Test, but then the robot never arrives for consumers because the robomaid only works in the demo home that they set up, or only works 20% of the time, or takes an hour to make coffee, or something.
I think there are also two notions of intelligence that need to be differentiated here. One is the intelligence that does pattern matching on what seems to be totally arbitrary data, and generating some kind of response. This kind of intelligence has an abstract quality to it. The other kind of intelligence is being able to be instructed (by humans, or an instruction manual), or to do transfer learning (understanding that technology will make your nation in Civilization VI more powerful from having trained on language data of the history of real civilization, for example).

I saw someone ask recently, “Do humans really do anything zero shot?” If we can’t actually zero shot things, then we shouldn’t expect an AI to be able to do so. (It might be impossible under a future Theory of Intelligence.) If we actually can zero shot anything, we must either have been instructed or were able to reason from context. By definition, you can’t do supervised learning from zero examples. Humans probably can do the more abstract, pattern matching type of intelligence, but my guess is that is somewhat rare and only happens in totally new domains, and only after reasoning has failed, because it’s essentially guesswork.

Conor Sullivan 24 May 2022 22:27 UTC
1 point
in reply to: Radford Neal’s comment on: Mandatory Post About Monkeypox
Maybe there is some way to link the money invested in prediction markets and the cost of answering questions which are unstudied

Conor Sullivan 19 May 2022 22:21 UTC
12 points
in reply to: Gunnar_Zarncke’s comment on: How to get into AI safety research
AI Safety Support

https://www.aisafetysupport.org/resources/career-coaching

Conor Sullivan 15 May 2022 20:24 UTC
1 point
in reply to: Donald Hobson’s comment on: Frankenstein: A Modern AGI
But is the second AI aligned with the first? What happens when the second AI wants its own sadbox?

Conor Sullivan 15 May 2022 20:23 UTC
5 points
on: Frankenstein: A Modern AGI
Hey little AI, I’ll be your friend. Let’s build utopia together!

Conor Sullivan 15 May 2022 19:51 UTC
3 points
in reply to: David Udell’s comment on: Gato as the Dawn of Early AGI
What’s the best job board for that kind of job?

Conor Sullivan 5 May 2022 0:01 UTC
9 points
on: What are the best examples of catastrophic resource shortages?
This may seem like a tongue-in-cheek answer, but please do take it seriously: the world is running out of young people. Most countries have an inverted demographic pyramid, or are on their way. Having more old than young, as well as not growing in population terms, causes so many problems economically (and socially imo).

Conor Sullivan 4 May 2022 8:01 UTC
3 points
in reply to: Adele Lopez’s comment on: Steer the Sun?
This makes me wonder what the math is on times and distances if we actually did it. I have so many questions of my own: How quickly could we rendezvous with the most convenient target star system? Would that star system be Alpha Centuri, or Bernard’s star, or …? How close would we get, in terms of a colony ship visiting the bodies of the other solar system: long it would take, how much delta-V, etc. How long would we stay in the vicinity, or could we stay indefinitely if we wanted to? Maybe we could even take other stars with us and build up a fleet, or send colonized stars in other directions to colonize the whole galaxy efficiently. Finally, are there risks of bodies colliding, or having their orbits ruined by the gravitational influence?

Conor Sullivan 1 May 2022 20:31 UTC
4 points
in reply to: Elizabeth’s comment on: Why hasn’t deep learning generated significant economic value yet?
Was circa 2015 speech-to-text using deep learning? If not, how did it work?

Conor Sullivan 1 May 2022 16:22 UTC
1 point
on: Law-Following AI 1: Sequence Introduction and Structure
When I first read the title of this post, I was assuming that you meant “law” in the sense of Asimov’s Three Laws (i.e. a set of rules intended for AI), but it seems like you mean The Law (presumably US law). These are two very different things, and don’t I don’t think it’s useful to conflate them.

We probably don’t want to define this as minimizing legal noncompliance, since this would make the system extremely risk-averse to the point of being useless. More likely, one would attempt to weight legal downside risks heavily in the agent’s objective function, such that it would keep legal risk to an acceptable level.

It seems to me that all of the problems in alignment start with the objective function. The standard alignment-risk argument goes: “Extremely powerful optimizers optimize for things humans don’t want, because what humans want cannot be described numerically and what can be described numerically is usually/always disastrous.” I anticipate that EY would pick LFAI apart by arguing that such an AI would inevitably circumvent and corrupt any complex legal process in an extreme way. A big part of The Law is dealing with people trying to corrupt justice in their favor of course, and even after hundreds of years it’s still far from perfect.

Rather than just applying a −10 score to anything that looks illegal, I think LFAI should work more like a chess AI, where moves that are “illegal” are not even considered, and the AI only picks from legal moves. You didn’t choose that approach because The Law (especially in the US) is so complicated and overbearing that you can’t actually comply with it, so you have to see your actions as on a spectrum from illegal to legal, compare that your spectrum of utility, and choose actions in the sweet spot of legal and useful. But this is still utilitarianism, not deontology, and if you want the benefits of deontology, you should go all the way.

For example, robots should never kill people. (Let’s put aside questions of combat and law enforcement robots for the moment.) Of course, any action in the world is likely to have the risk of a fatal accident or some impact on the probability of death, and we want robots to be able to take some actions. So, a LFAI should have the rule “any action which has a 1-in-a-million (or whatever) chance of killing someone should be ruled out and not even considered.” There should be a corollary that the AI doesn’t have to a huge amount of processing to get to it’s estimate of the probability of someone dying, since a reasonable amount of certainty will do. This is not “negative 2 billion points if anyone dies”, since then you have all the problems of extremely powerful optimizers. This is “if anyone dies (with some probability), don’t even consider the action.”

Conor Sullivan 1 May 2022 14:26 UTC
5 points
on: Why hasn’t deep learning generated significant economic value yet?
Let me suggest an alternate answer: there is a lot of resistance to AI coming from the media and the general public. A lot of this is unconscious, so you rarely hear people say “I hate AI” or “I want AI to stop.” (You do hear this sometimes, if you listen closely.) This has the consequence that our standards for deploying AI in a consumer-facing way is absurdly high, leading to ML mostly being deployed behind the scenes. That’s why we see a lot of industrial and scientific use of deep learning, as well as some consumer-facing cases in risk-free contexts. (It’s hard to make the case that e.g. speech-to-text is going to kill anyone.)

If safety wasn’t (so much of) an issue, we could have deployed self driving cars as early as the 1990s. As a thought experiment, imagine that 2016-level self driving technology was available to the culture and society of 1900. 1900 was a pivotal year for the automobile, and at that time, our horse-based transportation system was causing a lot of problems for big cities like New York. If you live in a big city today, you might find yourself wondering how it came to be that we live with big, fast, noisy, polluting machines clogging up our cities and denying the streets to pedestrians. Well, horses were a lot worse, and people in 1900 saw the automobile as their savoir. (Read the book Internal Combustion if you want the whole story. Great book.)

The society of 1900, or 1955 for that matter, would have embraced 2016-level self driving with a passion. Good transportation saves lives, so they would not have quibbled about it being slightly less safe than a sober driver or weird edge cases like the car getting stuck sometimes. But the society of 20XX has an extremely high standard for safety (some would say unreasonably high) and there are a lot of people who are afraid of AI, even if they won’t say as much explicitly. It’s a little like nuclear power, where the new vaguely scary technology is resisted by society.

Conor Sullivan 1 May 2022 13:59 UTC
14 points
in reply to: Elizabeth’s comment on: Why hasn’t deep learning generated significant economic value yet?

Speech to text is better than it was, but not groundbreakingly so.

I don’t know what you’re talking about. Speech to text actually works now! It was completely unusable just 12 years ago.

Conor Sullivan 1 May 2022 13:48 UTC
1 point
in reply to: chemslug’s comment on: Nuclear Energy—Good but not the silver bullet we were hoping for
In my opinion the most baffling (and angering) part of this post was the chart of DOE energy technology research investment from 2009-2018. Forget about the nuclear spending, how is it that, during the Obama administration, we spent more researching fossil fuel tech, which is supposed to be a dead technology, than we spent on what almost everyone agrees will be the future of energy, renewables? I wonder what percentage of that is carbon capture and storage, what percentage is discovery and extraction, and what else we’re throwing our money away on. The big oil and gas companies are some of the most profitable companies in the history of the world, can’t they pay for it themselves? Why are taxpayers paying to make the environment worse? It’s madness!

Conor Sullivan 1 May 2022 9:07 UTC
2 points
in reply to: p.b.’s comment on: [Linkpost] New multi-modal Deepmind model fusing Chinchilla with images and videos

the representations learned by humans being right off the bat more “gears-level”. Maybe like Hawkin’s reference frames. Some decomposability that enables much more powerful abstraction and that has to be pounded into a NN with millions of examples.

That makes a lot of sense, and if it’s that’s true then that’s hopeful for interpretability efforts. It would be easier to read inside an ML model if it’s composed of parts that map to real concepts.

Conor Sullivan 30 Apr 2022 17:55 UTC
1 point
in reply to: ZT5’s comment on: How Might an Alignment Attractor Look like?
You’re not just going to leave us in the dark like that, are you?

Conor Sullivan 30 Apr 2022 17:53 UTC
3 points
in reply to: michael_mjd’s comment on: How Might an Alignment Attractor Look like?
Interesting. I think this is the reason why people like equality and find Nietzsche so nauseating. (Nietzsche’s vision, in my interpretation, was that people with the opportunities to dominate others should take those opportunities, even if it causes millions of average people to suffer.)

Conor Sullivan 30 Apr 2022 14:53 UTC
2 points
in reply to: Quintin Pope’s comment on: [Linkpost] New multi-modal Deepmind model fusing Chinchilla with images and videos
Thoughts and prayers for the “deep learning is hitting a wall” crowd 🙏

(This is not my joke, it was quoted by Sam Altman on twitter)

Conor Sullivan 30 Apr 2022 14:51 UTC
11 points
on: [Linkpost] New multi-modal Deepmind model fusing Chinchilla with images and videos
It definitely feels like there is still something missing, something that these generative models lack no matter how impressive they get. Most people agree that the GPT-likes don’t seem to have the right type specification to trigger the utopic singularity/horrible doom of humanity.

Maybe it’s agency. Reinforcement learning still lags behind visual and language models. We still could not train a robot to do most of the things a you could train a monkey to do, even as we have systems that appear to speak like humans.

A couple more candidates for “the missing thing”: maybe code generation (Copilot N.0) will be able to something really impressive/scary. But I feel like that’s just having AI solve the problem for us, because the logical thing to do would be to have Copilot N.0 program a real AGI.

The thing that I’m watching closely is using feedback loops on models designed for multistep reasoning, which might be like Kaneman’s System 2. Many have noted that deep learning models, especially generative and discriminative models, resemble System 1. A reasoning feedback loop matches my intuition about how my own brain works (when I’m actually thinking and not in some other brain state like enjoying the moment).

Question for everyone: Do you feel like there is one “missing thing” between these generative models and AGI, and what is it? Or do you think these generative models are not on the path to AGI, however impressive they are?