Fig. 1

We curated annotations of Aβ neuropathologies from multiple experts, and found differing degrees of consensus. a Five experts (NP) and two undergraduate novices (UG) used a custom web portal for annotation. Each annotator labeled the same set of images in the same order. From the expert annotations, we constructed consensus-of-n labels (n = 1 to n = 5) for the same 20,099 images. b Average class distributions are consistent across the seven annotators. The y-axis plots average frequency, while the x-axis plots the Aβ class. c Representative images illustrating consensus-of-n strategies applied to each Aβ class, with rows progressing from top to bottom in order of increasing consensus. For a consensus-of-n image, at least n experts labeled the image as positive for the designated class. Each image was randomly and independently chosen from the set of images. d Positive annotation distributions differ by Aβ class. The x-axis plots the exact (not cumulative) number of annotators who gave a positive label. Hence, when e = 1 and e = 5 this is equivalent to a consensus-of-one and consensus-of-five respectively. For e = 2, 3, or 4, this is not equivalent to an at-least-n consensus strategy. The y-axis plots the frequency. Each class has a different count of total positive labels (indicated in the legend). This total count represents the total number of images with at least one expert identifying the class. Each image may have multiple classes present