The genesis of Leafsnap was the realization that many techniques used for face recognition developed by Professor Peter Belhumeur and Professor David Jacobs, of the Computer Science departments of Columbia University and the University of Maryland, respectively, could be applied to automatic species identification.

Professors Jacobs and Belhumeur approached Dr. John Kress, Research Botanist and Curator at the Smithsonian to start a collaborative effort for designing and building such a system for plant species. Columbia University and the University of Maryland designed and implemented the visual recognition system used for automatic identification. In addition, Columbia University designed and wrote the iPhone, and iPad apps, the leafsnap.com website, and wrote the code that powers the recognition servers. The Smithsonian was instrumental in collecting the datasets of leaf species and supervising the curation efforts throughout the course of the project. As part of this effort, the Smithsonian contracted the not-for-profit nature photography group Finding Species, which collected and photographed the high-quality photos available in the apps and the website).