Any thoughts on Symbolica? (or “categorical deep learning” more broadly?)
All current state of the art large language models such as ChatGPT, Claude, and Gemini, are based on the same core architecture. As a result, they all suffer from the same limitations.
Extant models are expensive to train, complex to deploy, difficult to validate, and infamously prone to hallucination. Symbolica is redesigning how machines learn from the ground up.
We use the powerfully expressive language of category theory to develop models capable of learning algebraic structure. This enables our models to have a robust and structured model of the world; one that is explainable and verifiable.
It’s time for machines, like humans, to think symbolically.
How likely is it that Symbolica [or sth similar] produces a commercially viable product?
How likely is it that Symbolica creates a viable alternative for the current/classical DL?
I don’t think it’s that different from the intentions behind Conjecture’s CoEms proposal. (And it looks like Symbolica have more theory and experimental results backing up their ideas.)
Symbolica don’t use the framing of AI [safety/alignment/X-risk], but many people behind the project are associated with the Topos Institute that hosted some talks from e.g. Scott Garrabrant or Andrew Critch.
What is the expected value of their research for safety/verifiability/etc?
Sounds relevant to @davidad’s plan, so I’d be especially curious to know his take.
How likely is it that whatever Symbolica produces meaningfully contributes to doom (e.g. by advancing capabilities research without at the same time sufficiently/differentially advancing interpretability/verifiability of AI systems)?
(There’s also PlantingSpace but their shtick seems to be more “use probabilistic programming and category theory to build a cool Narrow AI-ish product” whereas Symbolica want to use category theory to revolutionize deep learning.)
thank you for sharing the job postings. We’re starting something really exciting, and as research leads on the team, we—Paul Lessard and Bruno Gavranović - thought wed provide clarifications.
Symbolica was not started to improve ML using category theory. Instead, Symbolica was founded ~2 years ago, with its 2M seed funding round aimed at tackling the problem of symbolic reasoning, but at the time, its path to getting there wasn’t via categorical deep learning (CDL). The original plan was to use hypergraph rewriting as means of doing learning more efficiently. That approach however was eventually shown unviable.
Symbolica’s pivot to CDL started about five months ago. Bruno had just finished his Ph.D. thesis laying the foundations for the topic and we reoriented much of the organization towards this research direction. In particular, we began: a) refining a roadmap to develop and apply CDL, and b) writing a position paper, in collaboration with with researchers at Google DeepMind which you’ve cited below.
Over these last few months, it has become clear that our hunches about applicability are actually exciting and viable research directions. We’ve made fantastic progress, even doing some of the research we planned to advocate for in the aforementioned position paper. Really, we discovered just how much Taking Categories Seriously gives you in the field of Deep Learning.
Many advances in DL are about creating models which identify robust and general patterns in data (see the Transformers/Attention mechanism, for instance). In many ways this is exactly what CT is about: it is an indispensable tool for many scientists, including ourselves, to understand the world around us: to find robust patterns in data, but also to communicate, verify, and explain our reasoning.
At the same time, the research engineering team of Symbolica has made significant, independent, and concrete progress implementing a particular deep learning model that operates on text data, but not in an autoregressive manner as most GPT-style models do.
These developments were key signals to Vinod and other investors, leading to the closing of the 31M funding round.
We are now developing a research programme merging the two, leveraging insights from theories of structure, e.g. categorical algebra, as means of formalising the process by which we find structure in data. This has twofold consequence: pushing models to identify more robust patterns in data, but also interpretable and verifiable ones.
In summary:
a) The push to apply category theory was not based on a singular whim, as the the post might suggest,
but that instead
b) Symbolica is developing a serious research programme devoted to applying category theory to deep learning, not merely hiring category theorists
All of this is to add extra context for evaluating the company, its team, and our direction, which does not come across in the recently published tech articles.
We strongly encourage interested parties to look at all of the job ads, which we’ve tailored to particular roles. Roughly, in the CDL team, we’re looking for either
1) expertise in category theory, and a strong interest in deep learning, or
2) expertise in deep learning, and a strong interest in category theory.
I’d bet against anything particularly commercially successful. Manifold could give better and more precise predictions if you operationalize “commercially viable”.
Is this coming from deep knowledge about Symbolica’s method, or just on outside view considerations like “usually people trying to think too big-brained end up failing when it comes to AI”.
Or at least that’s approximately true. I’ll have a post on why I expect the bitter lesson to hold eventually, but is likely to be a while. If you read this blog post you can probably predict my reasoning for why I expect “learn only clean composable abstraction where the boundaries cut reality at the joints” to break down as an approach.
I don’t think the bitter lesson strictly applies here. Since they’re doing learning, and the bitter lesson says “learning and search is all that is good”, I think they’re in the clear, as long as what they do is compute scalable.
(this is different from saying there aren’t other reasons an ignorant person (a word I like more than outside view in this context since it doesn’t hide the lack of knowledge) may use to conclude they won’t succeed)
By building models which reason inductively, we tackle complex formal language tasks with immense commercial value: code synthesis and theorem proving.
There are commercially valuable uses for tools for code synthesis and theorem proving. But structured approaches of that flavor don’t have a great track record of e.g. doing classification tasks where the boundary conditions are messy and chaotic, and similarly for a bunch of other tasks where gradient-descent-lol-stack-more-layer-ML shines.
Any thoughts on Symbolica? (or “categorical deep learning” more broadly?)
How likely is it that Symbolica [or sth similar] produces a commercially viable product?
How likely is it that Symbolica creates a viable alternative for the current/classical DL?
I don’t think it’s that different from the intentions behind Conjecture’s CoEms proposal. (And it looks like Symbolica have more theory and experimental results backing up their ideas.)
Symbolica don’t use the framing of AI [safety/alignment/X-risk], but many people behind the project are associated with the Topos Institute that hosted some talks from e.g. Scott Garrabrant or Andrew Critch.
What is the expected value of their research for safety/verifiability/etc?
Sounds relevant to @davidad’s plan, so I’d be especially curious to know his take.
How likely is it that whatever Symbolica produces meaningfully contributes to doom (e.g. by advancing capabilities research without at the same time sufficiently/differentially advancing interpretability/verifiability of AI systems)?
(There’s also PlantingSpace but their shtick seems to be more “use probabilistic programming and category theory to build a cool Narrow AI-ish product” whereas Symbolica want to use category theory to revolutionize deep learning.)
A new update
I’d bet against anything particularly commercially successful. Manifold could give better and more precise predictions if you operationalize “commercially viable”.
Is this coming from deep knowledge about Symbolica’s method, or just on outside view considerations like “usually people trying to think too big-brained end up failing when it comes to AI”.
Outside view (bitter lesson).
Or at least that’s approximately true. I’ll have a post on why I expect the bitter lesson to hold eventually, but is likely to be a while. If you read this blog post you can probably predict my reasoning for why I expect “learn only clean composable abstraction where the boundaries cut reality at the joints” to break down as an approach.
I don’t think the bitter lesson strictly applies here. Since they’re doing learning, and the bitter lesson says “learning and search is all that is good”, I think they’re in the clear, as long as what they do is compute scalable.
(this is different from saying there aren’t other reasons an ignorant person (a word I like more than outside view in this context since it doesn’t hide the lack of knowledge) may use to conclude they won’t succeed)
There are commercially valuable uses for tools for code synthesis and theorem proving. But structured approaches of that flavor don’t have a great track record of e.g. doing classification tasks where the boundary conditions are messy and chaotic, and similarly for a bunch of other tasks where gradient-descent-lol-stack-more-layer-ML shines.