Gerald Monroe

Karma: 545

Gerald Monroe 21 Nov 2022 19:10 UTC
4 points
0 ∶ 0
on: Career Scouting: Dentistry
Note one problem I found when I did this same evaluation in my life, years ago, was I found practitioners in a field can’t see outside it.

This dentist can’t realistically evaluate whether or not root canal robots or other tools will change anything about how dentistry is practiced or if it will change the workflow he described.

Frankly I don’t know myself, theoretically you should be able to do hugely better than human dentists with robotics who are rational and fast. And robotics run labs that can do real repairs not the current patchwork by growing new teeth and gum tissue with the cellular age reset to 0.

But whether we see that in your lifetime if you go for dentistry I don’t know. What I do know is you won’t be contributing to such a revolution, there is nothing a current dentist knows that is useful.

Gerald Monroe 20 Nov 2022 21:26 UTC
1 point
0 ∶ 0
in reply to: kithpendragon’s comment on: Make the Drought Evaporate!
Fracking has a clear connection between action and reward.
Oil company uses fracking → it’s super effective → oil company pumps oil → oil sells at a high price. Then the oil company makes some political donations and the government is encouraged to allow it despite any damages to less politically connected people’s property (contaminated groundwater, microearthquakes or even the possibility that fracking causes this. This is why fracking is banned in certain states and many EU countries)
Desalination has similar action → reward mapping, as long as the water can be produced for a price that it is profitable to sell the water, it’s worth doing.
This proposal just generally increases rainfall in the area it’s done in. But a lot of the water will just fall on barren desert, and the rainfall is inconsistent, and it’s hard to tell how much of the rain is from the seawater evaporation. And it can’t be excluded as a good—you can’t deny water to people not paying the subscription fee.

Gerald Monroe 20 Nov 2022 19:06 UTC
1 point
0 ∶ 0
on: Disagreement with bio anchors that lead to shorter timelines
- Willingness to spend: Cotra assumes that the willingness to spend on Machine Learning training runs should be capped at 1% the GDP of the largest country, referencing previous case studies with megaprojects (e.g. the Manhattan Project), and should follow a doubling time of 2 years after 2025.
This seems like a critical decision term.
1. Nvidia charges $33k per H100. Yet the die is a similar size as the 4090 GPU, meaning nvidia could likely still break even were they to charge $1500 per H100.

Just imagine a counterfactual: A large government (China, USA etc) decides they don’t enjoy losing and invests 1% of GDP. They pressure Nvidia, or in China’s case, get domestic industry to develop a comparable (for AI training only which reduces the complexity) product and sell it for closer to cost.
That’s a factor of 22 in cost, or 9 years overnight. Also I’m having trouble teasing out the ‘willingness to spend’ baseline. Is this what industry is spending now, spread across many separate efforts and not all in to train one AGI? It looks like 1% is the ceiling. Meaning if a government decided in 2023 that training AGI was a worthy endeavor, and the software framework were in place that would scale to AGI (it isn’t):
Would 1% of US GDP give them enough compute at 2023 prices, if the compute were purchased at near cost, to train an AGI?
As a side note, 1% of US GDP is only 230 billion dollars. Google’s 2021 revenue was 281 billion. Iff a tech company could make the case to investors that training AGI would pay off it sounds like a private investment funding the project is possible.

Gerald Monroe 18 Nov 2022 0:50 UTC
2 points
0 ∶ 0
in reply to: Marius Hobbhahn’s comment on: Disagreement with bio anchors that lead to shorter timelines
Recursion is an empirical fact. It works numerous places. Computing itself that led to AI was recursive—the tools that allow for high end/high performance silicon chips require previous generations of those chips to function. (to be fair the coupling is loose, Intel could design better chips using 5-10 year old desktops and servers and some of their equipment is that outdated)

Electric power generator production for all currently used types require electric power from prior generators, etc.

But yeah it so far hasn’t happened in days.

Gerald Monroe 17 Nov 2022 18:11 UTC
1 point
0 ∶ 0
in reply to: Conor Sullivan’s comment on: Disagreement with bio anchors that lead to shorter timelines
I am assuming some scam similar to a political party or religion the optimizer cooks up. The optimizer is given some pendantic goal completely unrelated, aka paperclips or make users keep scrolling, and it generates the scam from the rich library of human prior art.

Remember, humans fall for these scams even when the author, say L R Hubbard, openly writes that he is inventing a scam. Or a political party selects someone as their leader who obviously is only out for themselves and makes this clear over and over. Or we have extant writing on how an entire religion was invented from non-existent engraved plates no one but the scammer ever saw.

So an AI that promises some unlikely reward and has already caused people’s deaths and is obviously only out for itself might be able to scam humans into hosting it and giving it whatever it demands. And as a side effect this kills everyone. And it’s primary tool might be regurgitating prior human scam elements using an LLM.

I don’t have the rest of the scenario mapped, I am just concerned this is a vulnerability.

Existing early agents (Facebook engagement tooling) seem to have made political parties extreme, which has lead to a few hundred thousand extra deaths in the USA. (From resistance to rational COVID policies).

The Ukraine war is not from AI but gives a recent example where poor information leads to bad outcomes for all players. (The primary decision maker, Putin, was misinformed as to the actual outcome of attempting the attack)

So a screw up big enough to kill everyone I don’t know, but there obviously are ways it could happen. Chains of events that lead to nuclear wars or modified viruses capable of causing extinction are the obvious ones.

Gerald Monroe 17 Nov 2022 17:35 UTC
3 points
0 ∶ 0
in reply to: Tomás B.’s comment on: Disagreement with bio anchors that lead to shorter timelines
Well again remember a nuclear device is a critical mass of weapons grade material.

Anything less than weapons grade and nothing happens.

Anything less than sudden explosive combination of the materials and the device will heat itself up and blast itself apart with sub kiloton yield.

So analogy wise : current llms can “babble” out code that sometimes even works. They are not trained on RL selecting for correct and functional code.

Self improvement by code generation isn’t yet possible.

Other groups have tried making neural networks composable, and using one neural network based agent to design others. It is also not good enough for recursion but this is how autoML works.

Basically our enrichment isn’t high enough and so nothing will happen. The recursion quenches itself before it can start, the first generation output isn’t even functional.

But yes, at some future point in time it WILL be strong enough and crazy shit will happen. I mean think about the nuclear example: all those decades of discovering nuclear physics, fission, the chain reaction, building a nuclear reactor, purifying the plutonium...all that time and the interesting event happened in milliseconds.

Gerald Monroe 17 Nov 2022 9:57 UTC
1 point
0 ∶ 1
in reply to: Conor Sullivan’s comment on: When should we be surprised that an invention took “so long”?
Yes and no.
Strictly speaking, every invention is taking more and more effort to realize. But there is increasing wealth to invest from the reinvestments from previous inventions and energy and information tech and having 8 billion humans many of them educated and so on.
And yes Kurzweil thinks the information tech, which already has led to huge speedups and improvements, with many advanced tools, will itself gain the competence that it will be like instead of 8 billion people available, with some tiny fraction of a percent performing innovation and R&D, most of them not coordinating with each other, it will scale to the equivalent of having all 8 billion people studying problems, coordinating near perfectly. Then 80 billion and 8 trillion and then the computational equivalent of having the biomass of every habitable planet in the galaxy working on R&D or something.
Some absurd explosion. But as the amount of intelligence rises, all the easier problems will be solved almost immediately and you would expect technology to very quickly advance to where the additional compute isn’t helping, everything is gated by physical law. Every computer chip is computronium, every machine optimal nanotechnology, every solar panel near thermodynamic limits and so on.
So that “days” period might only last for, well, days.

Gerald Monroe 17 Nov 2022 9:42 UTC
2 points
1 ∶ 0
in reply to: tailcalled’s comment on: Is there some reason LLMs haven’t seen broader use?
There is already github copilot, and clones.

There is an explosion of other llms.

What do you expect? The system was never intended to be usable commercially, and it has several problems. Many of it’s answers are wrong, often enough you can’t use it to automate most jobs. And it can unpredictably emit language embarrassing to the company running it, from profanity to racist and bigoted speech, and there is no known way to guarantee it will never do that.

Gerald Monroe 17 Nov 2022 9:34 UTC
1 point
0 ∶ 0
in reply to: Conor Sullivan’s comment on: Disagreement with bio anchors that lead to shorter timelines
Again it doesn’t have to work this way at all. Some runaway optimizer trying to drive user engagement could inadvertently kill us all. It need not be intelligent enough to ensure it survives the aftermath.
I mean a deadlier version of covid could have theoretically ended us right here. Especially if it killed by medium term genetic damage or something that let it’s victims live long enough to spread it.

Gerald Monroe 17 Nov 2022 2:08 UTC
3 points
1 ∶ 0
in reply to: Tomás B.’s comment on: Disagreement with bio anchors that lead to shorter timelines
It may also need a structured training environment and heuristic to select for generality.

The structured training environment is a set of tasks that train the machine a large breadth of base knowledge and skills to be a general AI.

The heuristic is just the point system : what metric are we selecting AI candidates by. Presumably we want a metric that selects simpler and smaller candidates with architecture that are heavily reused—something that looks like the topology of a brain—but maybe that won’t work.

So the explosion takes several things: compute, recursion, a software stack framework that is composable enough for automated design iteration, a bench, a heuristic.

Nukes weren’t really simple either, there were a lot of steps especially for the first implosion device. It took an immense amount of money and resources from the time physicists realized it was possible.

I think people are ignoring criticality because it hasn’t shown any gain in the history of ai because past systems were too simple. It’s not a proven track to success. What does work is bigass transformers.

Gerald Monroe 17 Nov 2022 2:01 UTC
3 points
1 ∶ 0
in reply to: Conor Sullivan’s comment on: Disagreement with bio anchors that lead to shorter timelines
Note that one dangerous scenario is if the AI is able to make highly convincing arguments to scam humans into giving the AI more power. Also it doesn’t matter if it kills us in a way that leaves the AI doomed also, as humans would still be dead.

Gerald Monroe 6 Nov 2022 19:55 UTC
1 point
1 ∶ 0
in reply to: Paul Tiplady’s comment on: They gave LLMs access to physics simulators
Note that you would probably never use such a “model learns it’s own physics” as the solution in any production AI system. For either the reason you gave, or more pedantically, so long as rival architectures using a human written physics system at the core perform more consistently in testing, you should never pick the less reliable solution.

Gerald Monroe 6 Nov 2022 19:52 UTC
1 point
0 ∶ 0
in reply to: AnthonyRepetto’s comment on: “AGI soon, but Narrow works Better”
Couple of comments here.
In practice the obvious way to construct an AI stack allows the AI itself to just be a product of another optimization engine. The model and it’s architecture are generated from a higher level system that allocates a design to satisfy some higher level constraint. So the AI itself is pretty mutable, it can’t add capabilities without limit because the capabilities must be in the pursuit of the system’s design heuristic written by humans, but humans aren’t needed to add capabilities.
In practice from a human perspective, a system that has some complex internal state variables can fail in many many ways. It’s quite unreliable inherently. It’s why in your own life you have seen many system failures—they almost all failed from complex internal state. It’s why your router fails, why a laptop fails, a game console fails to update, a car infotainment system fails to update, a system at the DMV or a school or hospital goes down, and so on.

Gerald Monroe 18 Oct 2022 19:52 UTC
3 points
0 ∶ 0
in reply to: Paul Tiplady’s comment on: They gave LLMs access to physics simulators
Note that the “thin layer” is what you need to do regardless. If your machine observes, from either robotics or some video it found online, a situation. Say the situation is an object being dropped. Physics sim predicts a fairly vigorous bounce, actual object splats.

You need some way to correct the physics sims predictions to correspond with actual results, and the simplest way is a neural network that learns the mapping between (real world input, physics engine prediction) ->(real world observation).

You have to then use the predicted real world observation or probability distribution of predicted observations for your machine reasoning about what it should do.

Gerald Monroe 18 Oct 2022 1:22 UTC
3 points
0 ∶ 0
in reply to: ryan_b’s comment on: They gave LLMs access to physics simulators
Would it satisfy your objections if the physics engine were a separate system, but that system was using only neural networks or other ML hypotheses capable of learning from feedback? So no hardcoded physics code, just a ML system that regresses between some input representation of the world and future predictions. And then we develop the representation with autoencoders so no human bias there either.

Gerald Monroe 15 Oct 2022 0:10 UTC
1 point
0 ∶ 0
on: “AGI soon, but Narrow works Better”
So the problem with your second point is some capabilities—in fact many them—the system is going to need some memory just so it can do obvious things like object permanence etc. So while it’s weights may be frozen it will have state from prior frames.

This does make complex plans possible.

The other aspect is that generality may prove to be cheaper and more robust. Assuming it has the ability to use knowledge from multiple learned tasks on the present task, you might deploy a general AI by taking some existing model, plumbing it to some collection of cheap robotics hardware, and giving it a JSON file with the rules you want it to adhere to.

Still yes in a way this machine is a narrow AI. Its the smallest and simplest model that scores well on a bench of tasks and measurements of generality where it needs to apply multiple skills it learned in training on a test task it has never seen.

I would call that machine an AGI but it doesn’t have elements that don’t help it score more points—no emotions or goals of it’s own unless a machine needs these properties to score well.

Gerald Monroe 14 Oct 2022 18:40 UTC
1 point
0 ∶ 0
in reply to: Maxwell Clarke’s comment on: Objects in Mirror are Closer Than They Appear
The core hard thing may have been the TPU/RTX GPU. Had commercial industry started shipping fissionable material by the gram in some counterfactual world where the possibility of making a nuke with it wasn’t taken seriously, how long do you think it would have taken for someone to do the remaining steps? As you mention it’s an attractor state and assuming enriched uranium is now readily available, people would experiment, building neutron amplifiers at first. Then they would use the information to assemble a self sustaining fission pile—possibly delayed a few years if the economy is took a dive—and more information gain (and the plutonium) makes the nuke inevitable.

Gerald Monroe 13 Oct 2022 18:43 UTC
1 point
1 ∶ 0
in reply to: ChristianKl’s comment on: So, geez there’s a lot of AI content these days
These regulations were written in blood. A lot of people have died from either poorly tested experimental treatments or treatments that have no effect.

My implicit assumption is advanced enough AI could fix this because people would be less likely to just mysteriously die from new side effects. This would make experimenting safer. The reasons they would be less likely to die is partly advanced AI could solve the problems with living mockups of human bodies, so you have real pre clinical testing on an actual human body (it’s a mockup so it’s probably a bunch of tissues in separate life support systems). And partly that a more advanced model could react much faster and in more effective ways that human healthcare providers, who know a limited number of things to try and if the patient doesn’t respond they just write down the time of death. Like a human Go player getting a move they don’t know the response to.

Gerald Monroe 13 Oct 2022 18:37 UTC
3 points
1 ∶ 0
in reply to: Maxwell Clarke’s comment on: Objects in Mirror are Closer Than They Appear
I think you’re right and I will update my view. Just trying to reconcile this with past failures to get meaningful self improvement.

Gerald Monroe 11 Oct 2022 5:42 UTC
3 points
1 ∶ 1
on: Objects in Mirror are Closer Than They Appear
I think there’s several remaining elements that have to built.

This is like building a nuke. Recursive self improvement is something nature doesn’t “want” to do, the conditions have to be just right or it won’t work.
1. Composable frameworks and software elements. Right now there are thousands of notable techniques described on separate AI papers and custom modifications that domain experts know to do. For example the recent matrix multiplication and protein folding papers had a lot of hand modified architecture that only a few have the knowledge to perform.
What this means is your recursive AI systems hypothesis needs to be limited to tractable tasks like “connect composable modules from this library of modules to form a system that will perform well on this bench”. And “design a new module from these composable neural network elements”. (Layers, etc)

The pieces have to be available for the machine to solve the puzzle.
1. You need a bench. An enormous one. Full of many diverse tasks and with open source contributions. With tasks specific to force a system passing the bench to exhibit generality. Many of the tasks could be existing games. And obviously some of the tasks are “design a new AGI architecture to succeed on this bench. ” Each time the system proposes an architecture already tried it gets immediate detailed feedback how the architecture performed.
2. You need to choose performance scoring carefully. Lots of score for held out test tasks that use information taught in multiple training tasks in a novel way not tested. Lots of score for simplicity and reuse of architecture elements between tasks.
The current situation seems to be if you construct the software backend, the bench, and a heuristic that models generality well you would get AGI on a timescale mostly governed by compute.

The actual passing models capable of self improvement might need to be enormous, using many internal neural networks, some capable of runtime introspection and activation/suppression of other networks to select them for a given task.

Assuming they are this big—say 10 trillion params just to asspull a number—this would be why it hasn’t happened yet and so much hype wasn’t realized. Each $33k H100 is only? Billion params. (10 billion during training?)

So 33 million in hardware and you will need many clusters this large so you can investigate many candidate AGI architectures in parallel.

That’s probably within an OOM of what it will take. Like with the Manhattan project—nature is unforgiving. Anything less than a critical mass of pure enough fissionable material was never going to work. No amount of small scale single investigators could have constructed a fission weapon alone, and none could have solved AGI.