I used to spend $2/mo in AI tokens for SOTA GPT-3.5 in mid 2022 for all my SOTA LLM needs. I now spend thousands, despite the insane price per token decline.
I have some moderate confidence the following is happening:
Tokens used to solve a task with greater difficulty is increasing faster than the amount of pricing decline per token.
Pricing decline of tokens is in the neighborhood of 90% per year. Token per task at SOTA is increasing 100x to 10,000x per year, in my experience.
I used to spend $2/mo in AI tokens for SOTA GPT-3.5 in mid 2022 for all my SOTA LLM needs. I now spend thousands, despite the insane price per token decline.
I have some moderate confidence the following is happening:
Tokens used to solve a task with greater difficulty is increasing faster than the amount of pricing decline per token.
Pricing decline of tokens is in the neighborhood of 90% per year. Token per task at SOTA is increasing 100x to 10,000x per year, in my experience.