I mean, we review all the decisions manually, so it’s the same legibility of criteria as before. What is the great benefit of making the LLM decisions more legible in-particular? Ideally we would just get the error close to zero.
I’d feel happy about being able to see the written criteria? And it seems nice to help refine the error rate by directly editing prompts rather than just feeding data maybe?
But yeah, on further thinking, this is pretty minor as an available improvement, automated with classifier plus review seems fine enough.
I mean, we review all the decisions manually, so it’s the same legibility of criteria as before. What is the great benefit of making the LLM decisions more legible in-particular? Ideally we would just get the error close to zero.
I’d feel happy about being able to see the written criteria? And it seems nice to help refine the error rate by directly editing prompts rather than just feeding data maybe?
But yeah, on further thinking, this is pretty minor as an available improvement, automated with classifier plus review seems fine enough.