Abstract/Summary

Crowd sourcing has been ubiquitously used for annotating enormous collections of data. However, the major obstacles of using crowd-sourced labels are noise and errors from non-expert annotations. In this work, two approaches dealing with the noise and errors in crowd-sourced labels are proposed. The first approach uses Sharpness-Aware Minimization (SAM), an optimization technique robust to noisy labels. The other approach leverages a neural network layer called crowd layer specifically designed to learn from crowd-sourced annotations. According to the results, the proposed approaches can improve the performance of Wide Residual Network model and Multi-layer Perception model applied on two crowd-sourced datasets in image processing domain.