Unlike LLaMA it’s under CC SA-BY 4.0, a copyleft license that allows commercial use. Together with databricks-dolly-15k, which was released under CC SA-BY 3.0 last week, we are about to get an open sourced LLM that follows the compute optimal (Chinchilla) dataset/model ratio (1.5T dataset/65B model) at the scale of the largest LLaMA, with instruct tuning. In the LLaMA/Alpaca incarnation, this made the news only 1.5-2 months ago with mere noncommercial availability.
Not totally clear if/how they’ll publish the pre-training dataset though. And their currently released supervised instruct model variants are using instruct datasets that make them noncommercial (CC BY-NC-SA 4.0).
It’s not the same kind of thing as ChatGPT or text-davinci-003 yet, no RLHF. But the announcement says “we will be kicking off our crowd-sourced RLHF program”.
Unlike LLaMA it’s under CC SA-BY 4.0, a copyleft license that allows commercial use. Together with
databricks-dolly-15k, which was released under CC SA-BY 3.0 last week, we are about to get an open sourced LLM that follows the compute optimal (Chinchilla) dataset/model ratio (1.5T dataset/65B model) at the scale of the largest LLaMA, with instruct tuning. In the LLaMA/Alpaca incarnation, this made the news only 1.5-2 months ago with mere noncommercial availability.Not totally clear if/how they’ll publish the pre-training dataset though. And their currently released supervised instruct model variants are using instruct datasets that make them noncommercial (CC BY-NC-SA 4.0).
It’s not the same kind of thing as ChatGPT or
text-davinci-003yet, no RLHF. But the announcement says “we will be kicking off our crowd-sourced RLHF program”.