I think especially if you have a competent coding LLM in the air-gapped network, then probably yes, if you mean software engineering.
The biggest bottlenecks to me look like having nice searchable documentation for libraries the LLM doesn’t know well—the documentation for most projects can’t easily be downloaded, and the ones for which it can be downloaded easily aren’t in a nicely searchable format. Gitbook isn’t universal (yet. Growth mindset).
(Similar with the kiwix search—you need to basically know what you’re looking for, or you won’t find it. E.g. I was trying to think of the Nazi Bureaucrat who in the Sino-Japanese war in the 1930s had rescued many Chinese from getting killed in warcrimes, but couldn’t think of it—until LLaMa-2-13b (chat) told me the name was John Rabe—but I’ve despaired over slightly more obscure questions.)
A well-resourced actor could try to clone the Stackoverflow content and their search, or create embeddings for a ton of documentation pages of different software packages. That’d make it much nicer.
Also, a lot of software straight up doesn’t work without an internet connection—see e.g. the incident where people couldn’t do arithmetic in Elm without an internet connection. Thankfully it’s the exception rather than the norm.
Do you think this hints at “doing engineering in an air gapped network can be made somewhat reasonable”?
(I’m asking in the context of securing AI labs’ development environments. Random twist, I know)
I think especially if you have a competent coding LLM in the air-gapped network, then probably yes, if you mean software engineering.
The biggest bottlenecks to me look like having nice searchable documentation for libraries the LLM doesn’t know well—the documentation for most projects can’t easily be downloaded, and the ones for which it can be downloaded easily aren’t in a nicely searchable format. Gitbook isn’t universal (yet. Growth mindset).
(Similar with the kiwix search—you need to basically know what you’re looking for, or you won’t find it. E.g. I was trying to think of the Nazi Bureaucrat who in the Sino-Japanese war in the 1930s had rescued many Chinese from getting killed in warcrimes, but couldn’t think of it—until LLaMa-2-13b (chat) told me the name was John Rabe—but I’ve despaired over slightly more obscure questions.)
A well-resourced actor could try to clone the Stackoverflow content and their search, or create embeddings for a ton of documentation pages of different software packages. That’d make it much nicer.
Also, a lot of software straight up doesn’t work without an internet connection—see e.g. the incident where people couldn’t do arithmetic in Elm without an internet connection. Thankfully it’s the exception rather than the norm.