One question is whether 3 > 2. A separate question is 2 > 1. I am making the/a point in this post that 2 > 1.[2]
But since 3 > 2, shouldn’t we always be doing 3 instead of 2, making the point that 2 > 1 moot? I don’t think so.
3 takes more time than 2 and is not always practical. A given design is composed of lots of little decisions that are made[3], and there isn’t time to do proper user research on each of these component decisions. And so in practice, the status quo is that people currently do not do user research on all of these component decisions.
In which case, I think the question is whether 2 > 1. Or rather, as I mention in my second footnote to this comment, whether 2 can be used in addition to 1, adding value. I think it can and often does. Furthermore, I think that this point is underutilized/underappreciated/under-understood.
And there’s probably some sort of more general point to be made here that I’m struggling to think about and articulate that extends past usability and software design.
Although you seem to feel more strongly about how much 3 > 2 than I do. To make up numbers, how strongly I feel about it’s importance, I’d say is like a 7⁄10, whereas you seem to be more like a 9.5/10 or 10⁄10.
I have a feeling that we mostly agree with each other and are thinking about two different questions. Consider three options:
Think about whether a design matches various heuristics.
Think concretely about whether a design will actually be understandable to users (user research thought experiment).
Do user research to find out whether a design will actually be understandable to users (actual user research).
I think we are in agreement that 3 > 2 > 1.[1]
One question is whether 3 > 2. A separate question is 2 > 1. I am making the/a point in this post that 2 > 1.[2]
But since 3 > 2, shouldn’t we always be doing 3 instead of 2, making the point that 2 > 1 moot? I don’t think so.
3 takes more time than 2 and is not always practical. A given design is composed of lots of little decisions that are made[3], and there isn’t time to do proper user research on each of these component decisions. And so in practice, the status quo is that people currently do not do user research on all of these component decisions.
In which case, I think the question is whether 2 > 1. Or rather, as I mention in my second footnote to this comment, whether 2 can be used in addition to 1, adding value. I think it can and often does. Furthermore, I think that this point is underutilized/underappreciated/under-understood.
And there’s probably some sort of more general point to be made here that I’m struggling to think about and articulate that extends past usability and software design.
Although you seem to feel more strongly about how much 3 > 2 than I do. To make up numbers, how strongly I feel about it’s importance, I’d say is like a 7⁄10, whereas you seem to be more like a 9.5/10 or 10⁄10.
Well, they all work in conjunction with each other, but the idea of 3 > 2 > 1 still gestures at what I am trying to get at, I hope.
I’m having trouble articulating what I mean by this. Maybe you get it?
First, note that 1 is properly not just heuristic analysis, but also all the other formal and semi-formal methods of evaluation. That said:
1 > 2. (Because 1 is just 2 but systematized, with checklists, corrections for common biases, non-obvious considerations, etc.)
3 <> 1. (That is: 3 is incomparable with 1.) (As you say, a design is composed of many decisions, too many to effectively user-test, etc.)
3+1 > any of { 1, 2, 3 }. (Note that this constitutes the standard prescription for UX design practice.)