GPT-3 can already turn comments into code. We don’t expect the reverse case to be fundamentally harder
I would expect the reverse case to be harder, possibly fundamentally. In a lot of code the reader’s level of context is very important to code quality, and if you asked me to write code to follow a specification I would think it was boring but if you asked me to comment code that someone else wrote I would be very unhappy.
It’s possible that it’s just an order of magnitude harder or harder in a way that is bad for human attention systems that GPT-N would find easy. But I would predict that the project will have a stumbling block of “it gives comments but they are painful to parse,” and there’s at least some chance (10-30%?) that it will require some new insight.
I would expect the reverse case to be harder, possibly fundamentally. In a lot of code the reader’s level of context is very important to code quality, and if you asked me to write code to follow a specification I would think it was boring but if you asked me to comment code that someone else wrote I would be very unhappy.
It’s possible that it’s just an order of magnitude harder or harder in a way that is bad for human attention systems that GPT-N would find easy. But I would predict that the project will have a stumbling block of “it gives comments but they are painful to parse,” and there’s at least some chance (10-30%?) that it will require some new insight.
I also expect it to be harder as well, and 10-30% chance that it will require some new insight seems reasonable.