Sure: ~$100 between API credits (majority of the cost from proprietary models) and cloud GPUs. A few of the smaller models were evaluated on my M4 Macbook Pro with 24 gigs of unified ram. For larger open weight models, I rented A100s. Most runs took about 20 minutes at the 2 degree resolution.
Curious as I’m experimenting with LLM stuff myself these days, where did you rent the A100? I suppose it comes out to be cheaper than paying OpenAI or Anthropic for credits?
You can easily and somewhat cheaply get access to A100s with Google Colab by paying for the pro subscription or just buying them outright. They sell “compute credits” which are pretty opaque, hard to say the amount of usage time you’ll be able to get with X credits.
I assume that’s an Amazon thing but man that is unfortunate naming to anyone sufficiently familiar with web fiction (and possibly, intentionally cheeky that way).
I suppose subtlety is braindead, but its body will remain forcefully kept alive by being hooked to machines until someone launches an AI-powered defense system literally called Skynet.
Would you let us know how much money/credits you spent on it overall, and separately, how many hours on your laptop, and how much RAM?
Sure: ~$100 between API credits (majority of the cost from proprietary models) and cloud GPUs. A few of the smaller models were evaluated on my M4 Macbook Pro with 24 gigs of unified ram. For larger open weight models, I rented A100s. Most runs took about 20 minutes at the 2 degree resolution.
Curious as I’m experimenting with LLM stuff myself these days, where did you rent the A100? I suppose it comes out to be cheaper than paying OpenAI or Anthropic for credits?
You can easily and somewhat cheaply get access to A100s with Google Colab by paying for the pro subscription or just buying them outright. They sell “compute credits” which are pretty opaque, hard to say the amount of usage time you’ll be able to get with X credits.
I used https://www.runpod.io/. Pretty cheap.
There’s also vast.ai and lamda labs. And prime intellect.
I assume that’s an Amazon thing but man that is unfortunate naming to anyone sufficiently familiar with web fiction (and possibly, intentionally cheeky that way).
You’d think, but nope, it’s explicitly named after the web fiction.
I suppose subtlety is braindead, but its body will remain forcefully kept alive by being hooked to machines until someone launches an AI-powered defense system literally called Skynet.
https://en.wikipedia.org/wiki/SKYNET_(surveillance_program)
Like the Darwin Awards, we need the Torment Nexus Awards for stuff like this.
I’m afraid the people who are nominated would just make torment nexus themed laptop stickers.