Who Said What: Modeling Individual Labelers Improves Classification
Who Said What: Modeling Individual Labelers Improves Classification
Data are often labeled by many different experts with each expert only labeling a small fraction of the data and each data point being labeled by several experts. This reduces the workload on individual experts and also gives a better estimate of the unobserved ground truth. When experts disagree, the …