The traditional distance metric for vectors like that is angle—that is, you take the dot product of the two vectors, divide by the product of the lengths, then inverse-cosine the result, with 0 being maximally similar and pi being maximally dissimilar. 1D scales like political orientation are also represented as vectors, to which you measure a similarity; and you find interesting scales by putting a bunch of peoples’ belief vectors in a matrix and doing singular value decomposition. You weight the terms by scaling the values.
This similarity metric is very widely used in search, where a (sparse) vector has an entry in it for each term in the lexicon, and the value is the number of times that term appears.
However, I’m not sure a vector is really the right way to model peoples’ maps; there are a lot of things you can’t represent that way, or that require truly enormous vectors to represent that way. Many ontologies use graphs, instead—you have nodes for various concepts, and edges between them representing relationships. In that case, you would measure similarity using edit distance: the length of the shortest sequence of insertions and deletions that would make one map look like the other. However, while there are efficient algorithms for edit distance between strings, I don’t know whether there is one for edit distance between graphs, or if that’s even in P.
If we have the map be a list and the entries be probabilities, we’d need to somehow represent the difference between someone who believes in lots and someone who believes in very little, even if they have the same angle.
Something like chi-squared might work, but that would end up being dominated by the largest differences, which may be undesirable. But you can probably take roots until you get something that does what you want.
Not that you could compute it...
Maybe we should try to emulate what humans do when they assess how much they have in common with someone. Which is… uh… I don’t know what we do. Probably something like a chi squared, but starting with broad categories and stronger inferences, and including weaker terms rarely or not at all.
Hm, yeah that’s true, we can’t just represent probabilities. But also, we need a way to represent the emotional salience of the probabilities. That could probably be done by creating two new emotional salience vectors that correspond to each belief vector (appending the vectors might work, but would introduce complications in the metric calculations).
The traditional distance metric for vectors like that is angle—that is, you take the dot product of the two vectors, divide by the product of the lengths, then inverse-cosine the result, with 0 being maximally similar and pi being maximally dissimilar. 1D scales like political orientation are also represented as vectors, to which you measure a similarity; and you find interesting scales by putting a bunch of peoples’ belief vectors in a matrix and doing singular value decomposition. You weight the terms by scaling the values.
This similarity metric is very widely used in search, where a (sparse) vector has an entry in it for each term in the lexicon, and the value is the number of times that term appears.
However, I’m not sure a vector is really the right way to model peoples’ maps; there are a lot of things you can’t represent that way, or that require truly enormous vectors to represent that way. Many ontologies use graphs, instead—you have nodes for various concepts, and edges between them representing relationships. In that case, you would measure similarity using edit distance: the length of the shortest sequence of insertions and deletions that would make one map look like the other. However, while there are efficient algorithms for edit distance between strings, I don’t know whether there is one for edit distance between graphs, or if that’s even in P.
If we have the map be a list and the entries be probabilities, we’d need to somehow represent the difference between someone who believes in lots and someone who believes in very little, even if they have the same angle.
Something like chi-squared might work, but that would end up being dominated by the largest differences, which may be undesirable. But you can probably take roots until you get something that does what you want.
Not that you could compute it...
Maybe we should try to emulate what humans do when they assess how much they have in common with someone. Which is… uh… I don’t know what we do. Probably something like a chi squared, but starting with broad categories and stronger inferences, and including weaker terms rarely or not at all.
Very interesting replies.
Hm, yeah that’s true, we can’t just represent probabilities. But also, we need a way to represent the emotional salience of the probabilities. That could probably be done by creating two new emotional salience vectors that correspond to each belief vector (appending the vectors might work, but would introduce complications in the metric calculations).