Yep! Here’s an example where the 50% horizon and 80% horizon can be lower for an agent whose success profile dominates another agent (i.e. higher success rate at all task lengths), even for (1) monotone nonincreasing success rates (i.e. longer tasks are harder) (2) success rate of 1 at minimum task length (3) success rate of 0 at maximum task length
Yep! Here’s an example where the 50% horizon and 80% horizon can be lower for an agent whose success profile dominates another agent (i.e. higher success rate at all task lengths), even for
(1) monotone nonincreasing success rates (i.e. longer tasks are harder)
(2) success rate of 1 at minimum task length
(3) success rate of 0 at maximum task length
before points are
[(0,1), (1, 1⁄15), (2, 0), (3,0)]
after points are
[(0,1), (1, 0.1), (2, 0.1), (3, 1⁄15)]
https://www.desmos.com/calculator/nqwn6ofmzq