Paper @ RAI Workshop AAAI 2020
The aim of this workshop is to develop recommendations on how researchers can enhance the reproducibility of their work, and to encourage future AAAI conferences to implement them. Workshop site: https://folk.idi.ntnu.no/odderik/RAI-2020/
Our contribution to this workshop:
Towards Reproducible Artificial Intelligence: The Roles of Researchers and Publishers
Anita de Waard, Georgios George Tsatsaronis, Rinke Hoekstra, Sweitze Roffet, Catriona Fennell, Sergios Petrides, Thom Pijnenburg and Efthymios Tsakonas
Abstract
Over the past few years, we have seen a rapid growth in the field of artificial intelligence and its applications, such as machine learning, computer vision and natural language processing. This means that the issue of ensuring reproducible published scientific results is now becoming even more crucial for the AI community. With increased frequency, our community is experiencing critical reports of research results that cannot be reproduced. Some recent studies tried to shed light on the main reasons for this lack of reproducibility. A primary criticism is the lack of proper documentation and an appropriate explanation of the methods and the experimental setups and conditions in AI publications. In this paper, we share some of our own experiments in this arena, and present our opinion on the roles and responsibilities that both researchers and publishers have in the process of increasing reproducibility in AI. We suggest some potential directions for the community to consider, and identify some possible best practices that we can undertake, to improve the reproducibility of future research in artificial intelligence.