GPT-4.5 was killed off after 3 months, presumably for inference cost reasons.
FWIW, GPT-4.5 is still available for Pro-tier users.
leni
Karma: 434
Inverse Rubric Optimization: A testbed for agent science
Tracking Difficulty with Feature Portfolios
Benchmarking Real Work
Introducing Lunette: auditing agents for evals and environments
Automated real time monitoring and orchestration of coding agents
AI agents and painted facades
Distillation Robustifies Unlearning
Update on Harvard AI Safety Team and MIT AI Alignment
The current SOTA models do very well (~90% accuracy) at few-shot learning tasks in the CIFAR-FS dataset [source], which has a comparable resolution to the images seen by bees, so I think that this task is quite solvable. Even bees and the models I discussed seem to do pretty well compared to chance.
Interesting to learn that compute figures can be brought down so much without accuracy loss! Could you point me to some reading material about this?
I think Rohin’s second point makes sense. Bees are actually pretty good at classifying abstract shapes (I mention a couple of studies that refer to this in the appendix about my choice of benchmark, such as Giurfa (1996)), so they might plausibly be able to generalize to stylized images.
Hey Ben! Thanks for formatting the doc into the post, it looks great!
How does bee learning compare with machine learning?
Hey everyone! I’m Eleni. I’m doing an AI timelines internship with Open Phil and am going to investigate that topic over the next few months.
It seems plausible to a lot of people that simply scaling up current ML architectures with more data and compute could lead to transformative AI. In particular, the recent successes of GPT-3 and the impressive scaling observed seem to suggest that a scaled-up language model could have a transformative impact. This hypothesis can be modeled within the framework of Ajeya’s report by considering that a transformative model would have the same effective horizon length as GPT-3 and assuming that the scaling will follow the same laws as current Transformer models. I’ve added an anchor corresponding to this view in an updated version of the quantitative model that can be found (together with the old one) here, where the filenames corresponding to the updated model begin with “(GPT-N)”. Please note that Ajeya’s best guess sheet hasn’t been updated to take this new anchor into account. A couple of minor numerical inconsistencies between the report and the quantitative model were also fixed.
Not sure—OpenAI’s website says that it’s “unlimited” (subject to their guardrails), but I don’t know what that means.