It’s a trap. It’s a trap in UX, and an analogous trap in software design.
The problem with UX research, as normally practiced, is that it prioritizes first impressions and neglects what happens when the user has been using the system for awhile and understands it well. So you wind up doing things like adjusting all the buttons to control their prominence, to guide new users to the main interactions and away from the long-tail interactions, at the expense of those buttons having any sort of coherent organization.
In the case of software, UX-style user testing is going to lead straight into a well known, known bad attractor: brittle magic. There is a common pattern in libraries and frameworks where they do a great job making a few common development tasks look really easy, shielding the developer from a lot of details that they probably should not have been shielded from. And what happens with these systems, almost invariably, is that as soon as you start using them for real instead of in toy examples, they reveal themselves to be overcomplicated garbage which impedes attempts to understand what’s really happening.
In the case of software, UX-style user testing is going to lead straight into a well known, known bad attractor: brittle magic. There is a common pattern in libraries and frameworks where they do a great job making a few common development tasks look really easy, shielding the developer from a lot of details that they probably should not have been shielded from.
Indeed. This is, for example, possibly the biggest problem with Ruby on Rails.
I think I didn’t do a very good job of being clear about my point and thoughts.
The big thing I was trying to say is that user research should be thought of as the ultimate barometer, not whether the code you wrote happens to adhere to various acronyms like DRY.
Perhaps most UX-style user testing is done on new UIs, but you can also do such user testing on older UIs. For example, maybe you’ve had the same navbar for a long time, people have been using it, but you want to see if mobile users understand how to use it to navigate.
You make a great point about how one has to be careful to not only optimize for new users, and that the Third User (as Said describes) is also important. But I see this point as supplementary to the points I was trying to make, not in contrast to them. For example, suppose you have a bunch of DRY code in your codebase with weird abstractions that are confusing to the team. In that case, the user testing results are negative, and, I argue, those user testing results are what matter at the end of the day, not heuristics like whether the code adheres to principles like DRY.
Although, to caveat that point, there is probably some wisdom in giving those principles some weight as well, since our ability to introspect and answer the question of how easy it is to work with code is imperfect. Damn, that wasn’t very clear. It’s hard to explain what I mean here. Maybe this is a good analogy. Psychologists who study happiness find that long commutes make people unhappy. Maybe you have a long commute and think that it’s fine. So the user testing shows that it’s fine. But maybe you’re failing to introspect and the commute is actually causing you a good deal of frustration. In that case the user testing has mislead us. Because of this risk, it’s probably a good idea to give some weight to what things like happiness research have shown. Hopefully that makes some sense.
In the case of software, UX-style user testing is going to lead straight into a well known, known bad attractor: brittle magic.
It’s a trap. It’s a trap in UX, and an analogous trap in software design.
The problem with UX research, as normally practiced, is that it prioritizes first impressions and neglects what happens when the user has been using the system for awhile and understands it well. So you wind up doing things like adjusting all the buttons to control their prominence, to guide new users to the main interactions and away from the long-tail interactions, at the expense of those buttons having any sort of coherent organization.
In the case of software, UX-style user testing is going to lead straight into a well known, known bad attractor: brittle magic. There is a common pattern in libraries and frameworks where they do a great job making a few common development tasks look really easy, shielding the developer from a lot of details that they probably should not have been shielded from. And what happens with these systems, almost invariably, is that as soon as you start using them for real instead of in toy examples, they reveal themselves to be overcomplicated garbage which impedes attempts to understand what’s really happening.
Indeed. This is, for example, possibly the biggest problem with Ruby on Rails.
I think I didn’t do a very good job of being clear about my point and thoughts.
The big thing I was trying to say is that user research should be thought of as the ultimate barometer, not whether the code you wrote happens to adhere to various acronyms like DRY.
Perhaps most UX-style user testing is done on new UIs, but you can also do such user testing on older UIs. For example, maybe you’ve had the same navbar for a long time, people have been using it, but you want to see if mobile users understand how to use it to navigate.
You make a great point about how one has to be careful to not only optimize for new users, and that the Third User (as Said describes) is also important. But I see this point as supplementary to the points I was trying to make, not in contrast to them. For example, suppose you have a bunch of DRY code in your codebase with weird abstractions that are confusing to the team. In that case, the user testing results are negative, and, I argue, those user testing results are what matter at the end of the day, not heuristics like whether the code adheres to principles like DRY.
Although, to caveat that point, there is probably some wisdom in giving those principles some weight as well, since our ability to introspect and answer the question of how easy it is to work with code is imperfect. Damn, that wasn’t very clear. It’s hard to explain what I mean here. Maybe this is a good analogy. Psychologists who study happiness find that long commutes make people unhappy. Maybe you have a long commute and think that it’s fine. So the user testing shows that it’s fine. But maybe you’re failing to introspect and the commute is actually causing you a good deal of frustration. In that case the user testing has mislead us. Because of this risk, it’s probably a good idea to give some weight to what things like happiness research have shown. Hopefully that makes some sense.
I agree with what you’re saying here.