Multicriteria objective functions are really hard to get right. Weighting features from 10 to 1 is actually a decent first approach- it should separate good solutions from bad solutions- but if you’re down to narrow differences of the weighted objective function, it’s typically time to hand off to a human decision-maker, or spend a lot of time considering tradeoffs to elicit the weights. (Thankfully, a first pass should show you what features you need to value carefully and which features you can ignore.)
If you have relatively few choices and properties are correlated (as of course they are), I’m not sure how much it matters. I did a simulation of this for embryo selection with n=10, and partially randomized the utility weights made little difference.
Multicriteria objective functions are really hard to get right. Weighting features from 10 to 1 is actually a decent first approach- it should separate good solutions from bad solutions- but if you’re down to narrow differences of the weighted objective function, it’s typically time to hand off to a human decision-maker, or spend a lot of time considering tradeoffs to elicit the weights. (Thankfully, a first pass should show you what features you need to value carefully and which features you can ignore.)
If you have relatively few choices and properties are correlated (as of course they are), I’m not sure how much it matters. I did a simulation of this for embryo selection with n=10, and partially randomized the utility weights made little difference.