Thanks for this great overview of what’s going on with the large language models.
I want to indulge in some limited knowledge speculation. Just want to put that out there first.
I’m going to predict that the larger language models are not going to be very much more useful from the current state.
I’m wondering if you know or can talk about what people are doing to improve the models other than making them bigger?
I read a Wolfram blog post that talked about using the Wolfram expert systems in conjunction with large language models of you looked at that?
The approach of combining llms with with expert systems seems to be a valuable path to me in terms of creating more useful systems. I e, results from a LLM could be evaluated by expert systems and fed back into the text that’s generated. Do you see this is the path to creating the even more intelligent systems?
Thanks for this great overview of what’s going on with the large language models. I want to indulge in some limited knowledge speculation. Just want to put that out there first. I’m going to predict that the larger language models are not going to be very much more useful from the current state. I’m wondering if you know or can talk about what people are doing to improve the models other than making them bigger? I read a Wolfram blog post that talked about using the Wolfram expert systems in conjunction with large language models of you looked at that? The approach of combining llms with with expert systems seems to be a valuable path to me in terms of creating more useful systems. I e, results from a LLM could be evaluated by expert systems and fed back into the text that’s generated. Do you see this is the path to creating the even more intelligent systems?
There are currently attempts to train LLMs to use external APIs as tools:
https://cognitiveai.org/wp-content/uploads/2022/10/wang2022-behavior-cloned-transformers-are-neurosymbolic-reasoners-arxiv.pdf
https://arxiv.org/abs/2302.04761