Affiliations: [a] Toshiba Digital Solutions Corporation, Tokyo 183-8512, Japan. E-mail: [email protected] | [b] Graduate School of Information Systems, The University of Electro-Communications, Tokyo 182-8585, Japan. E-mails: [email protected], [email protected]
Abstract: Current crowdsourcing platforms such as Amazon Mechanical Turk provide an attractive solution for processing numerous tasks at a low cost. The number of workers who process crowdsourcing tasks is increasing along with the expansion of domains in which crowdsourcing is utilized. However, there is insufficient support for crowdsourcing workers, such as education and improvement of their work environment. This problem may be due to crowdsourcing workers being numerous and unspecified, which also makes them easy to employ and terminate. Poor worker management could lead to declining quality of worker records and unjustified worker termination. In this study, we propose a grade-based training method for workers. Our training method utilizes probabilistic networks to estimate correlations between tasks based on worker records for 18.5 million tasks, then allocates pre-learning tasks to workers to raise the accuracy of target tasks according to task correlations. In an experiment, the method automatically allocated 31 pre-learning task categories for 9 target task categories, and after pre-learning task training, we confirmed that target task accuracy increased by 7.8 points on average. This result was comparatively higher than those for pre-learning tasks allocated using other methods, such as decision trees. We therefore confirmed that task correlations can be estimated from a large number of worker records, and that these are useful for grade-based training of low-quality workers.
