Inevitable given the incentives, but still disappointing. Contractors are paid to evaluate whether chatbot responses are good. But they’re paid based on the assumption that each judgement takes a certain amount of time: not much.
“Three hours of research to complete a 60-second task, that’s a great way to frame the problem we’re facing right now”
It’s easy to forget how much very human but not very well-paid labour goes into many AI and other predictive models