I sensed you were highly skeptical of my “neural sim” variable until 2 days ago.
No. Not really. I wasn’t claiming that things like SORA couldn’t exist. I am claiming that it’s hard to turn them towards the task of engineering a bridge say.
Current SORA is totally useless for this. You ask it for a bridge, and it gives you some random bridge looking thing, over some body of water. SORA isn’t doing the calculations to tell if the bridge would actually hold up. But lets say a future much smarter version of SORA did do the calculations. A human looking at the video wouldn’t know what grade of steel SORA was imagining. I mean existing SORA probably isn’t thinking of a particular grade of steel, but this smarter version would have picked a grade, and used that as part of it’s design. But it doesn’t tell the human that, the knowledge is hidden in it’s weights.
Ok, suppose you could get it to show a big pile of detailed architectural plans, and then a bridge. All with super-smart neural modeling that does the calculations. Then you get something that ideally is about as good at looking at the specs of a random real world bridge. Plenty of random real world bridges exist, and I presume bridge builders look at their specs. Still not that useful. Each bridge has different geology, budget, height requirements etc.
Ok, well suppose you could start by putting all that information in somehow, and then sampling from designs that fit the existing geology, roads etc.
Then you get several problems.
The first is that this is sampling plausible specs, not good specs. Maybe it shows a few pictures at the end to show the bridge not immediately collapsing. But not immediately collapsing is a low bar for a bridge. If the Super-SORA chose a type of paint that was highly toxic to local fish, it wouldn’t tell you. If the bridge had a 10% chance of collapsing, it’s randomly sampling a plausible timeline. So 90% of the time, it shows you the bridge not collapsing. If it only generates 10 minutes of footage, you don’t know what might be going on in it’s sim while you weren’t watching. If it generates 100 years of footage from every possible angle, it’s likely to record predictions of any problems, but good luck finding the needle in the haystack. Like imagine this AI has just given you 100 years of footage. How do you skim through it without missing stuff.
Another problem is that SORA is sampling in the statistical sense. Suppose you haven’t done the geology survey yet. SORA will guess at some plausible rock composition. This could lead to you building half the bridge, and then finding that the real rock composition is different.
You need a system that can tell you “I don’t know fact X, go find it out for me”.
If the predictions are too good, well the world it’s predicting contains Super-SORA. This could lead to all sorts of strange self fulfilling prophecy problems.
No. Not really. I wasn’t claiming that things like SORA couldn’t exist. I am claiming that it’s hard to turn them towards the task of engineering a bridge say.
I would agree with that.
Let me give you a summary of my overall position: “Trust an ASI with a complex task it needs context awareness to complete? Not even once”.
Everything is about how to get a large amount of benefits in the orange area, and yes we should stay there for a prolonged period.
What benefits specifically? Vast amounts of material manufactured goods, robots, robots are doing repetitive tasks that can be clearly defined. ASI models in use are only used in limited duration sessions, and you strip away context awareness.
Context awareness is the bits that tell the ASI this is a real bridge, humans are really going to build it, and not just another task in the training sim. “Always bluepill”. It should not be possible for the ASI to know when the task is real vs sim. (which you can do by having an image generator convert real frames to a descriptor, and then regenerate them so they have the simulation artifacts...)
The architecture fees for a new bridge design are about 10% of the cost.
Other 90% is, well, all the back breaking labor to build one. Not just at the construction site, but where the concrete was made, building/maintaining the trucks to haul the materials, etc.
Sora’s role: For autonomous cars and general robots, this is used for training. General robots means one robot at a time, and for short task sessions. For example : “remove the part from the CNC machine and place it on the output table”.
What you do is record video from the human workers doing the task, many hours of it. You train a model using techniques similar Sora to classify what it sees and to predict frame for frame the next frame. This is 5-30 hz sampling rate, so 33 ms into the future. Then do this recursively.
Since this is an RL problem you go further and may model a few seconds ahead.
There are ways to reproject in a robot instead.
Then the robot tries to do the task in the real world once it has settled on a policy that covers it in the simulated world.
The robot may fail, but you train the model, and you policy iterate with each failure. This is where the log(real world episodes) comes in. It’s correct and aligns very well with real world experience.
Note on Sora : Tesla and probably Waymo already use a neural sim that is similar to training their robotic cars. I found the tweet last night from an insider, and I’ve seen demo videos before, i can produce evidence. But it is accurate to say it’s already in use for the purpose I am talking about.
Bridges : at least in the near future, you want to automate making the parts for repairing a bridge, doing the dangerous work of inspection via drone, converting a bunch of pictures from many places from the drones to an overall model of the bridge compatible with load testing software, and so on. <https://artbabridgereport.org/reports/2021-ARTBA-Bridge-Report.pdf> current USA bridges need 7% replaced and work done on about 40%.
Key note : I would not trust an ASI or AGI with designing a new bridge, the goal is to make it cheaper to replace or repair the designs you already have. If AI + robots can contribute between 10% and 95% of the labor to replace a bridge (starting at 10, gradually ramping...), that lets you swap a lot more bridges for the same annual budget.
If we have the technical capacity to get into the red zone, and enough chips to make getting there easy. Then hanging out in the orange zone, coordinating civilization not to make any AI too powerful, when there are huge incentives to ramp the power up, and no one is quite sure where the serious dangers kick in...
That is, at least, an impressive civilization wide balancing act. And one I don’t think we have the competence to pull off.
It should not be possible for the ASI to know when the task is real vs sim. (which you can do by having an image generator convert real frames to a descriptor, and then regenerate them so they have the simulation artifacts...)
This is something you want, not a description of how to get it, and one that is rather tricky to achieve. That converting and then converting back trick is useful. But sure isn’t automatic success either. If there are patterns about reality that the ASI understands, but the simulator doesn’t, then the ASI can use those patterns.
Ie if the ASI understands seasons, and the simulator doesn’t, then if it’s scorching sunshine one day and snow the next, that suggests it’s the simulation. Otherwise, that suggests reality.
And if the simulation knows all patterns that the ASI does, the simulator itself is now worryingly intelligent.
robots are doing repetitive tasks that can be clearly defined.
If the task is maximally repetitive, then the robot can just follow the same path over and over.
If it’s nearly that repetitive, the robot still doesn’t need to be that smart.
I think you are trying to get a very smart AI to be so tied down and caged up that it can do a task without going rouge. But the task is so simple that current dumb robots can often do it.
For example : “remove the part from the CNC machine and place it on the output table”.
Economics test again. Minimum wage workers are easily up to a task like that. But most engineering jobs pay more than minimum wage. Which suggests most engineering in practice requires more skill than that.
I mean yes engineers do need to take parts out of the CNC machine. But they also need to be able to fix that CNC machine when a part snaps off inside it and starts getting jammed in the workings. And the latter takes up more time in practice. Or noticing that the toolhead is loose, and tightning and recalibrating it.
The techniques you are describing seem to be next level in fairly dumb automation. The stuff that some places are already doing (like boston dynamics robot dog level hardware and software), but expanded to the whole economy. I agree that you can get a moderate amount of economic growth out of that.
I don’t see you talking about any tasks that require superhuman intelligence.
First of all. SORA.
No. Not really. I wasn’t claiming that things like SORA couldn’t exist. I am claiming that it’s hard to turn them towards the task of engineering a bridge say.
Current SORA is totally useless for this. You ask it for a bridge, and it gives you some random bridge looking thing, over some body of water. SORA isn’t doing the calculations to tell if the bridge would actually hold up. But lets say a future much smarter version of SORA did do the calculations. A human looking at the video wouldn’t know what grade of steel SORA was imagining. I mean existing SORA probably isn’t thinking of a particular grade of steel, but this smarter version would have picked a grade, and used that as part of it’s design. But it doesn’t tell the human that, the knowledge is hidden in it’s weights.
Ok, suppose you could get it to show a big pile of detailed architectural plans, and then a bridge. All with super-smart neural modeling that does the calculations. Then you get something that ideally is about as good at looking at the specs of a random real world bridge. Plenty of random real world bridges exist, and I presume bridge builders look at their specs. Still not that useful. Each bridge has different geology, budget, height requirements etc.
Ok, well suppose you could start by putting all that information in somehow, and then sampling from designs that fit the existing geology, roads etc.
Then you get several problems.
The first is that this is sampling plausible specs, not good specs. Maybe it shows a few pictures at the end to show the bridge not immediately collapsing. But not immediately collapsing is a low bar for a bridge. If the Super-SORA chose a type of paint that was highly toxic to local fish, it wouldn’t tell you. If the bridge had a 10% chance of collapsing, it’s randomly sampling a plausible timeline. So 90% of the time, it shows you the bridge not collapsing. If it only generates 10 minutes of footage, you don’t know what might be going on in it’s sim while you weren’t watching. If it generates 100 years of footage from every possible angle, it’s likely to record predictions of any problems, but good luck finding the needle in the haystack. Like imagine this AI has just given you 100 years of footage. How do you skim through it without missing stuff.
Another problem is that SORA is sampling in the statistical sense. Suppose you haven’t done the geology survey yet. SORA will guess at some plausible rock composition. This could lead to you building half the bridge, and then finding that the real rock composition is different.
You need a system that can tell you “I don’t know fact X, go find it out for me”.
If the predictions are too good, well the world it’s predicting contains Super-SORA. This could lead to all sorts of strange self fulfilling prophecy problems.
I would agree with that.
Let me give you a summary of my overall position: “Trust an ASI with a complex task it needs context awareness to complete? Not even once”.
Everything is about how to get a large amount of benefits in the orange area, and yes we should stay there for a prolonged period.
What benefits specifically? Vast amounts of material manufactured goods, robots, robots are doing repetitive tasks that can be clearly defined. ASI models in use are only used in limited duration sessions, and you strip away context awareness.
Context awareness is the bits that tell the ASI this is a real bridge, humans are really going to build it, and not just another task in the training sim. “Always bluepill”. It should not be possible for the ASI to know when the task is real vs sim. (which you can do by having an image generator convert real frames to a descriptor, and then regenerate them so they have the simulation artifacts...)
The architecture fees for a new bridge design are about 10% of the cost.
Other 90% is, well, all the back breaking labor to build one. Not just at the construction site, but where the concrete was made, building/maintaining the trucks to haul the materials, etc.
Sora’s role: For autonomous cars and general robots, this is used for training. General robots means one robot at a time, and for short task sessions. For example : “remove the part from the CNC machine and place it on the output table”.
What you do is record video from the human workers doing the task, many hours of it. You train a model using techniques similar Sora to classify what it sees and to predict frame for frame the next frame. This is 5-30 hz sampling rate, so 33 ms into the future. Then do this recursively.
Since this is an RL problem you go further and may model a few seconds ahead.
There are ways to reproject in a robot instead.
Then the robot tries to do the task in the real world once it has settled on a policy that covers it in the simulated world.
The robot may fail, but you train the model, and you policy iterate with each failure. This is where the log(real world episodes) comes in. It’s correct and aligns very well with real world experience.
Note on Sora : Tesla and probably Waymo already use a neural sim that is similar to training their robotic cars. I found the tweet last night from an insider, and I’ve seen demo videos before, i can produce evidence. But it is accurate to say it’s already in use for the purpose I am talking about.
Bridges : at least in the near future, you want to automate making the parts for repairing a bridge, doing the dangerous work of inspection via drone, converting a bunch of pictures from many places from the drones to an overall model of the bridge compatible with load testing software, and so on. <https://artbabridgereport.org/reports/2021-ARTBA-Bridge-Report.pdf> current USA bridges need 7% replaced and work done on about 40%.
Key note : I would not trust an ASI or AGI with designing a new bridge, the goal is to make it cheaper to replace or repair the designs you already have. If AI + robots can contribute between 10% and 95% of the labor to replace a bridge (starting at 10, gradually ramping...), that lets you swap a lot more bridges for the same annual budget.
If we have the technical capacity to get into the red zone, and enough chips to make getting there easy. Then hanging out in the orange zone, coordinating civilization not to make any AI too powerful, when there are huge incentives to ramp the power up, and no one is quite sure where the serious dangers kick in...
That is, at least, an impressive civilization wide balancing act. And one I don’t think we have the competence to pull off.
This is something you want, not a description of how to get it, and one that is rather tricky to achieve. That converting and then converting back trick is useful. But sure isn’t automatic success either. If there are patterns about reality that the ASI understands, but the simulator doesn’t, then the ASI can use those patterns.
Ie if the ASI understands seasons, and the simulator doesn’t, then if it’s scorching sunshine one day and snow the next, that suggests it’s the simulation. Otherwise, that suggests reality.
And if the simulation knows all patterns that the ASI does, the simulator itself is now worryingly intelligent.
If the task is maximally repetitive, then the robot can just follow the same path over and over.
If it’s nearly that repetitive, the robot still doesn’t need to be that smart.
I think you are trying to get a very smart AI to be so tied down and caged up that it can do a task without going rouge. But the task is so simple that current dumb robots can often do it.
Economics test again. Minimum wage workers are easily up to a task like that. But most engineering jobs pay more than minimum wage. Which suggests most engineering in practice requires more skill than that.
I mean yes engineers do need to take parts out of the CNC machine. But they also need to be able to fix that CNC machine when a part snaps off inside it and starts getting jammed in the workings. And the latter takes up more time in practice. Or noticing that the toolhead is loose, and tightning and recalibrating it.
The techniques you are describing seem to be next level in fairly dumb automation. The stuff that some places are already doing (like boston dynamics robot dog level hardware and software), but expanded to the whole economy. I agree that you can get a moderate amount of economic growth out of that.
I don’t see you talking about any tasks that require superhuman intelligence.