So good that it is bad. On the (re)use of Datasets in Machine Learning Security
Seminar Dagstuhl Seminars: Security of Machine Learning (Negative Results)
Dagstuhl, Germany
Oneliner: A very negative (informal) talk!
I really had a blast at Dagstuhl—definitely one of those “once in a lifetime” experiences. In particular, during the second day of the Seminar, there was a session entirely devoted to “negative results”.
Here, I let out a lot of negativity that I experienced during my first 2 years as a postdoc—negativity concealing a problem that (imho) is significantly affecting our community as whole.
Abstract: In this (informal!) talk, I will discuss about the pros and cons of (re)using “benchmark” datasets in the context of ML security. I will elucidate some aspects deriving from such “reuse” that intrinsically affect our entire community. My intention is to raise the attention to a “problem” that – despite being at the core of our research – is often overlooked, and that (potentially) is negatively impacting future developments in our domain.