Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED
Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED
DocRED is a widely used dataset for document-level relation extraction. In the large-scale annotation, a recommend-revise scheme is adopted to reduce the workload. Within this scheme, annotators are provided with candidate relation instances from distant supervision, and they then manually supplement and remove relational facts based on the recommendations. However, …