Sorry, I think I might have a superficial understanding of encoders and embeddings. Would you be able to try pointing out for me how decomposition is performed in that case (or point me toward a favorite reading on the subject)? When I think of feeding a sentence into an encoder, I can think of multiple ways in which some compositional structure might be inferred.
I’m drawing up a proof of concept with seq2seq learners right now, but my hypothesis is that they will be inadequate decomposers suitable only for benchmarking a baseline.
Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it’s possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that’s not very useful to factored cognition.
Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.
Why is the literature into reversible encoders/autoencoders/embedding generators not relevant for your specific usecase ?
Give an answer to that it might be easier to recommend stuff.
Sorry, I think I might have a superficial understanding of encoders and embeddings. Would you be able to try pointing out for me how decomposition is performed in that case (or point me toward a favorite reading on the subject)? When I think of feeding a sentence into an encoder, I can think of multiple ways in which some compositional structure might be inferred.
I’m drawing up a proof of concept with seq2seq learners right now, but my hypothesis is that they will be inadequate decomposers suitable only for benchmarking a baseline.
I was asking why because I wanted to understand what you mean by “decomposition”.
Defines many things.
Usually the goal is feature extraction (think Bert) or reducing the size of a representation (think autoencoders or simpler , PCA)
You need to narrow down your definition, I think, to get a meaningful answers.
I think Quinn means factored cognition, which is quite different from autoencoders/embeddings/PCA.
Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it’s possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that’s not very useful to factored cognition.
Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.