The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can Help
The Challenge of Variable Effort Crowdsourcing and How Visible Gold Can Help
We consider a class of variable effort human annotation tasks in which the number of labels required per item can greatly vary (e.g., finding all faces in an image, named entities in a text, bird calls in an audio recording, etc.). In such tasks, some items require far more effort …