Historical patents are a difficult target for digitization. Figures and labels are central to patents and present complex combinations of image and text elements. Sponsored by the CoECI in 2011, the U.S. Patent and Trademark Office (USPTO) Algorithm Challenge tasked competitors to develop algorithms that automatically detect figures and part labels in US patents (USPTO; White House blog; Topcoder problem statement). The challenge drew 232 two-person teams, of which 70 teams (30%) submitted solutions. The first place submission was able to correctly identify part labels within an image and correctly recognize the text in more than 70% of the test cases. For more information see Riedl et al. (2014).