Fourth generation nuclear weapons are as many say in the industry are the “technology of the future and always will be”. I understand this is partially a thought experiment, but just to point out that the premise is far from reality.
Molecular laser isotope separation is a much more likely scenario to create fissionable material on the sly. Remember the first atomic bomb to kill people was a howitzer barrel and two lumps of Uranium 235 (not even weapons grade) shot into each other. The amount of material that actually fused would be about the mass of a penny. The tiny amount of fissioned material in little boy was the equivalent of 1.25 miles of box cars full of TNT.
The key to larger and efficient weapons is keeping the radioactive material together longer for more cycles of fission and creating more neutrons from the start. This is done by containment, implosion designs, neutron generators, neutron reflectors, and injecting deuterium and tritium to create more neutrons as the reaction starts.
A truck driver as a hobby built a copy of Little Boy with public sources. As this was easy to do by a single individual why don’t we already have these devices cooking off left and right. The given scenario also assumes that design is the only hurdle. Procurement of materials and concealment aren’t something that AI can teach.
Why would a technology like fusion be more likely than a technology that has been shown to work?
All of these technologies revolve around a huge amount of electricity up front. Governments already watch high electrical use locations for signs of marijuana growers and uranium refinement. Charging the huge banks of capacitors necessary to start a fusion reaction would easily trigger an investigation on anyone but state actors.
I would suggest that machine learning and gene editing using CRISPR technology to create pathogens would be much easier path to a weapon of mass destruction as they can be done far more covertly.
https://en.wikipedia.org/wiki/Molecular_laser_isotope_separation
Flatland for machine learning?