My thought is that LLMs can probably program physical “muscle memory” for themselves with basic multimodal inputs + tool calling capability then control them just fine. The muscle memory themselves would perhaps need a world model to execute akin to a brain’s motor cortex but those models probably don’t need any capability beyond functioning as a motor cortex, i.e. “follow simple instructions and forward important surprises to the LLM”, so there’s not really much doubt that they can be developed.
My thought is that LLMs can probably program physical “muscle memory” for themselves with basic multimodal inputs + tool calling capability then control them just fine. The muscle memory themselves would perhaps need a world model to execute akin to a brain’s motor cortex but those models probably don’t need any capability beyond functioning as a motor cortex, i.e. “follow simple instructions and forward important surprises to the LLM”, so there’s not really much doubt that they can be developed.