The APPS repository also gives the fine-tuned weights for GPT-Neo-2.7 and code to run it. Though without a GPU it takes roughly forever.
I asked Dan Hendrycks for the performance of GPT-J-6B on APPS on the Eleuther AI discord. He didn’t say they were definitely going to test it, but my take-away was that it might happen.
I could image a test driven automated programming evolving in the next ten to twenty years, were a LM-guided search tries to create functions according to a description that pass all the test cases.
The APPS repository also gives the fine-tuned weights for GPT-Neo-2.7 and code to run it. Though without a GPU it takes roughly forever.
I asked Dan Hendrycks for the performance of GPT-J-6B on APPS on the Eleuther AI discord. He didn’t say they were definitely going to test it, but my take-away was that it might happen.
I could image a test driven automated programming evolving in the next ten to twenty years, were a LM-guided search tries to create functions according to a description that pass all the test cases.