FWIW, this claim doesn’t match my intuition, and googling around, I wasn’t able to quickly find any papers or blog posts supporting it.
“Explaining and Harnessing Adversarial Examples” (Goodfellow et al. 2014) is the original demonstration that “Linear behavior in high-dimensional spaces is sufficient to cause adversarial examples”.
I’ll emphasize that high-dimensionality is a crucial piece of the puzzle, which I haven’t seen you bring up yet. You may already be aware of this, but I’ll emphasize it anyway: the usual intuitions do not even remotely apply in high-dimensional spaces. Check out Counterintuitive Properties of High Dimensional Space.
adversarial examples are only a thing because the wrong decision boundary has been learned
In my opinion, this is spot-on—not only your claim that there would be no adversarial examples if the decision boundary were perfect, but in fact a group of researchers are beginning to think that in a broader sense “adversarial vulnerability” and “amount of test set error” are inextricably linked in a deep and foundational way—that they may not even be two separate problems. Here are a few citations that point at some pieces of this case:
“Adversarial Spheres” (Gilmer et al. 2017) - “For this dataset we show a fundamental tradeoff between the amount of test error and the average distance to nearest error. In particular, we prove that any model which misclassifies a small constant fraction of a sphere will be vulnerable to adversarial perturbations of size O(1/√d).” (emphasis mine)
I think this paper is truly fantastic in many respects.
The central argument can be understood from the intuitions presented in Counterintuitive Properties of High Dimensional Space in the section titled Concentration of Measure (Figure 9). Where it says “As the dimension increases, the width of the band necessary to capture 99% of the surface area decreases rapidly.” you can just replace that with the “As the dimension increases, a decision-boundary hyperplane that has 1% test error rapidly gets extremely close to the equator of the sphere”. “Small distance from the center of the sphere” is what gives rise to “Small epsilon at which you can find an adversarial example”.
“Intriguing Properties of Adversarial Examples” (Cubuk et al. 2017) - “While adversarial accuracy is strongly correlated with clean accuracy, it is only weakly correlated with model size”
I haven’t read this paper, but I’ve heard good things about it.
To summarize, my belief is that any model that is trying to learn a decision boundary in a high-dimensional space, and is basically built out of linear units with some nonlinearities, will be susceptible to small-perturbation adversarial examples so long as it makes any errors at all.
(As a note—not trying to be snarky, just trying to be genuinely helpful, Cubuk et al. 2017 and Goodfellow et al. 2014 are my top two hits for “adversarial examples linearity” in an incognito tab)
When evaluating whether there is a broad base of support, I think it’s important to distinguish “one large-scale funder” from “narrow overall base of support”. Before the Arnold foundation’s funding, the reproducibility project had a broad base of committed participants contributing their personal resources and volunteering their time.
To add some details from personal experience: In late 2011 and early 2012, the Reproducibility Project was a great big underfunded labor of love. Brian Nosek had outlined a plan to replicate ~50 studies—this became the Science 2015 paper. He was putting together spreadsheets to coordinate everyone, and hundreds of researchers who were personally committed to the cause were allocating their own discretionary funds and working in their spare time to get the replications done in their own labs. The mailing list was thriving. Researchers were paying subjects out-of-pocket. Reproducibility wasn’t a full-blown memetic explosion in the public eye, nor was there a major source of funding, but we were getting notable media coverage, and researchers kept joining.
Importantly, I think we were already firmly on track to write the 2015 Science paper before the Arnold Foundation took notice of the coverage that existing projects were getting and began reaching out to Nosek and others to ask if they could do more with more funding.
When the Center for Open Science was founded, it increased the scale of coordination that Brian and other coordinators were able to execute amongst participants. I’d guess that Brian himself was also able to spend more time talking to the media. The base of participating researchers remained broad and unpaid. I’d guess that the vast majority of researchers contributing personally to the reproducibility movement are still not getting any earmarked funds for it.
I wasn’t aware of the details of COS’s funding before reading this article, so I have no additional evidence about whether there are more large-scale funders. A brief round of Googling turns up a few other Open Science flavored sources of money (e.g. https://www.openscienceprize.org/res/p/FAQ/) but these are not specific to reproducibility; rather, they’re more broadly targeted towards open sharing of code, data, and methods.
A few suggested takeaways:
There may be other cases where an existing movement with an enthusiastic base of participants is funding-limited in the scale of coordination and publicity they can achieve, and a single motivated funder can make a substantial impact by adding that type of funding.
Under “normal” funding and incentive conditions, the reproducibility project was able to form and begin producing concrete and impactful output, but thereafter it appears that only one major source of funding materialized and no other dedicated large-scale funding has been available. I think this should make you feel optimistic about academic researchers as individuals and as a culture, but pessimistic about traditional academic funding routes, rather than monolithically pessimistic about academia as a whole.