Early career ML researcher with an interest in interpretability.
dkornai
Karma: 2
Pain is the consequence of a perceived reduction in the probability that an agent will achieve its goals.
In biological organisms, physical pain [say, in response to limb being removed] is an evolutionary consequence of the fact that organisms with the capacity to feel physical pain avoided situations where their long-term goals [e.g. locomotion to a favourable position with the limb] which required the subsystem generating pain were harmed.
This definition applies equally to mental pain [say, the pain felt when being expelled from a group of allies] which impedes long term goals.
This suggests that any system that possesses both a set of goals and the capacity to understand how events influence their probability of achieving such goals should posses a capacity to feel pain. This also suggests that the amount of pain is proportional to the degree of “setbacks” and the degree to which “setbacks” are perceived.
I think this is a relatively robust argument for the inherent reality of pain not just in a broad spectrum biological organisms, but also in synthetic [including sufficiently advanced AI] agents.
We should strive to reduce the pain we cause in the agents we interact with.
dkornai’s Shortform
I would say that if a concept is imprecise, more words [but good and precise words] have to be dedicated to faithfully representing the diffuse nature of the topic. If this larger faithful representation is compressed down to fewer words, that can lead to vague phrasing. I would therefore often view vauge phrasing as a compression artefact, rather than a necessary outcome of translating certain types of concepts to words.
I would certainly agree with part of what you are saying. Especially the point that many important lessons are taught by pain [correct me if this is misinterpreting your comment]. Indeed, as a parent for example, if your goal is for your child to gain the capacity for self sufficiency, a certain amount of painful lessons that reflect the inherent properties of the world are necessary to achieve such a goal.
On the other hand, I do not agree with your framing of pain as being the main motivator [again, correct me if required]. In fact, a wide variety of systems in the brain are concerned with calculating and granting rewards. Perhaps pain and pleasure are the two sides of the same coin, and reward maximisation and regret minimisation are identical. In practice however, I think they often lead to different solutions.
I also do not agree with your interpretation that chronic pain does not reduce agency. For family members of mine suffering from arthritis, their chronic pain renders them unable to do many basic activities, for example, access areas for which you would need to climb stairs. I would like to emphasise that it is not the disease which limits their “degrees of freedom” [at least in the short term], and were they to take a large amount of painkillers, they could temporarily climb stairs again.
Finally, I would suggest that your framing as a “contrast between the current state and the goal state” is basically an alternative way of talking about the transition probability from the current state to the goal state. In my opinion, this suggests that our conceptualisations of pain are overwhelmingly similar.