Global climate change and other anthropogenic stressors have heightened the need to rapidly characterize ecological changes in marine benthic communities across large scales. Digital photography enables rapid collection of survey images to meet this need, but the subsequent image annotation is typically a time consuming, manual task. We investigated the feasibility of using automated point-annotation to expedite cover estimation of the 17 dominant benthic categories from survey-images captured at four Pacific coral reefs. Inter- and intra- annotator variability among six human experts was quantified and compared to semi- and fully- automated annotation methods, which are made available at coralnet.ucsd.edu. Our results indicate high expert agreement for identification of coral genera, but lower agreement for algal functional groups, in particular between turf algae and crustose coralline algae. This indicates the need for unequivocal definitions of algal groups, careful training of multiple annotators, and enhanced imaging technology. Semi-automated annotation, where 50% of the annotation decisions were performed automatically, yielded cover estimate errors comparable to those of the human experts. Furthermore, fully-automated annotation yielded rapid, unbiased cover estimates but with increased variance. These results show that automated annotation can increase spatial coverage and decrease time and financial outlay for image-based reef surveys.
Department of Computer Science and Engineering, University of California, San Diego, La Jolla, CA, United States of America;Department of Biology, California State University, Northridge, Northridge, CA, United States of America;Biophysical Remote Sensing Group, School of Geography, Planning and Environmental Management, University of Queensland, St. Lucia, QLD, Australia;Integrative Oceanography Division, Scripps Institution of Oceanography, University of California, San Diego, La Jolla, CA, United States of America;Integrative Oceanography Division, Scripps Institution of Oceanography, University of California, San Diego, La Jolla, CA, United States of America;Integrative Oceanography Division, Scripps Institution of Oceanography, University of California, San Diego, La Jolla, CA, United States of America;Catlin Seaview Survey, Global Change Institute, University of Queensland, St. Lucia, QLD, Australia;Joint Institute for Marine and Atmospheric Research, University of Hawaii at Manoa, Honolulu, HI, United States of America;Department of Biology, California State University, Northridge, Northridge, CA, United States of America;National Museum of Marine Biology and Aquarium, Checheng, Taiwan, Republic of China;National Museum of Marine Biology and Aquarium, Checheng, Taiwan, Republic of China;Department of Computer Science and Engineering, University of California, San Diego, La Jolla, CA, United States of America;Charney School of Marine Sciences, University of Haifa, Haifa, Israel;Department of Neuroscience, University of California, San Diego, La Jolla, CA, United States of America;Integrative Oceanography Division, Scripps Institution of Oceanography, University of California, San Diego, La Jolla, CA, United States of America;Department of Computer Science and Engineering, University of California, San Diego, La Jolla, CA, United States of America
Recommended Citation:
Oscar Beijbom,Peter J. Edmunds,Chris Roelfsema,et al. Towards Automated Annotation of Benthic Survey Images: Variability of Human Experts and Operational Modes of Automation[J]. PLOS ONE,2015-01-01,10(7)