Missing information, unresponsive authors, experimental flaws: the impossibility of assessing the reproducibility of previous human evaluations in NLP

Belz, Anya;Thomson, Craig;Reiter, Ehud;Abercrombie, Gavin;Alonso-Moral, Jose M.;Arvan, Mohammad;Braggaar, Anouck;Cieliebak, Mark;Clark, Elizabeth;van Deemter, Kees;Dinkar, Tanvi;Dušek, Ondřej;Eger, Steffen;Fang, Qixiang;Gao, Mingqi;Gatt, Albert;Gkatzia, Dimitra;González-Corbelle, Javier;Hovy, Dirk;Hürlimann, Manuela;Ito, Takumi;Kelleher, John D.;Klubicka, Filip;Krahmer, Emiel;Lai, Huiyuan;van der Lee, Chris;Li, Yiru;Mahamood, Saad;Mieskes, Margot;van Miltenburg, Emiel;Mosteiro, Pablo;Nissim, Malvina;Parde, Natalie;Plátek, Ondřej;Rieser, Verena;Ruan, Jie;Tetreault, Joel;Toral, Antonio;Wan, Xiaojun;Wanner, Leo;Watson, Lewis;Yang, Diyi

2023