Based on how I experienced things when I had the experience that made enlightenment seem within reach, something like a lack of noticeable change is in fact exactly what I would expect from many people who become enlightened.
If this is the case, our experience becomes slightly surprising from an anthropics-ish point of view.
That is, if there are multiple ways to experience the world that are instrumentally the same (like suffering from pain or not), whichever one we happen to have is a random draw. It seems we could have evolved to have any of them with equal probability. The more options there are, the more it would be nice to have a hypothesis that put extra weight in ending up with the experience we have.
Of course, one could bite the bullet and just say “well, we had to end up with one of the possible experiences! we’re just kind of unlucky”. Also one could argue that though the behaviours are the same, this experience is somehow more costly (maybe awareness of the constituents of your experience is just energetically costly) and recover naturality that way
Related: https://www.lesswrong.com/posts/wqDRRx9RqwKLzWt7R/nonperson-predicates
Here, EY discusses the concept of a non-person predicate, which evaluates things and tells you if they are not-people. It it says it’s a person, it might be wrong, but it’s never wrong if it says it’s not a person. That way, if you get “not a person!”, you can be certain that you do not have to worry about its subjective experience (and therefore, for many moral theories, its moral patienthood).
This doesn’t affect the main post’s point that once we know which systems are conscious, we may find ourselves in a situation where all our best candidates for work-doing-systems are also consciousness-having-systems
Also related: https://www.lesswrong.com/posts/mELQFMi9egPn5EAjK/my-attempt-to-explain-looking-insight-meditation-and
Here Kaj Sotala suggests that an aspect of our qualitative experience (suffering) can be removed without much change in our behaviours. (Though I worry that makes our experience of suffering surprising)