Skip to content

Toggle service links

Coronavirus: Course registration is open, but please be aware it may take us longer to respond than usual. Find out about our coronavirus response and current contact hours.

You are here

  1. Home
  2. Mr Jacopo Amidei

Mr Jacopo Amidei

Profile summary

Professional biography

Jan. 2020 - May 2020: Visiting Research Fellow, Universitat Pompeu Fabra (UPF), Spain

Oct. 2017 - present:  Ph.D in Computational Linguistics, The Open University, School of Computing & Communications 

Mar. 2016 - Oct. 2016:  Traineeship, ExB, Leipzig, Germany

Nov. 2012 - June. 2017:  Ph.D in Philosophy, Scuola Normale Superiore (SNS), Italy.  70/70 cum laude

Sept. 2008 - Sept. 2012:  Master in Philosophy, University of Siena, Italy    110/110 cum laude

Sept. 2004 - Apr. 2008:  Bachelor in Philosophy, University of Siena, Italy  110/110 cum laude

Research interests

1) Evaluation methodologies for Natural Language Generation systems

2) Machine Learning

3) Automatic Question Generation 

4) Natural Language Understanding

5) Philosophy of Language

Publications 

•  2016 

 

 

 

•  2018 

 

 

 

•  2019

 

 

•  2020

  • J. Amidei, P. Piwek and A. Willis,  Identifying  Annotator  Bias:  A new IRT-based method for bias identification, In Proceedings of The 28th International Conference on  Computational Linguistics (COLING), Barcelona, Spain, December 8-13, 2020.

 

  • M. Westera, J. Amidei and L. Mayol, Similarity or deeper understanding? Analyzing the TED-Q dataset of evoked questions, In Proceedings of The 28th International Conference on  Computational Linguistics (COLING), Barcelona, Spain, December 8-13, 2020.

 

•  To appear 

 

Impact and engagement

Local Organizing committee: XXV meeting of the Italian Association of Logic and its Applications, 14 - 17 April 2014 Scuola Normale Superiore, Pisa (Italy)

Publications

Trial and error mathematics: Dialectical systems and completions of theories (2019-01-23)
Amidei, Jacopo; Andrews, Uri; Pianigiani, Duccio; San Mauro, Luca and Sorbi, Andrea
Journal of Logic and Computation, 29(1) (pp. 157-184)


Identifying Annotator Bias: A new IRT-based method for bias identification (2020)
Amidei, Jacopo; Piwek, Paul and Willis, Alistair
In : Proceedings of The 28th International Conference on Computational Linguistics (COLING) (8-13 Dec 2020, Barcelona, Spain) (pp. 4787-4797)


Agreement is overrated: A plea for correlation to assess human evaluation reliability (2019)
Amidei, Jacopo; Piwek, Paul and Willis, Alistair
In : Proceedings of the 12th International Conference on Natural Language Generation (29 Oct - 1 Nov 2019, Tokyo, Japan)


The use of rating and Likert scales in Natural Language Generation human evaluation tasks: A review and some recommendations (2019)
Amidei, Jacopo; Piwek, Paul and Willis, Alistair
In : Proceedings of the 12th International Conference on Natural Language Generation (29 Oct - 1 Nov 2019, Tokyo, Japan)


Evaluation methodologies in Automatic Question Generation 2013-2018 (2018-11-06)
Amidei, Jacopo; Piwek, Paul and Willis, Alistair
In : Proceedings of The 11th International Natural Language Generation Conference (5-8 Nov 2018, Tilburg, The Netherlands) (pp. 307-317)


Rethinking the Agreement in Human Evaluation Tasks (2018)
Amidei, Jacopo; Piwek, Paul and Willis, Alistair
In : Proceedings of the 27th International Conference on Computational Linguistics (20-26 Aug 2018, Santa Fe, New Mexico) (pp. 3318-3329)