Traditionally, image retrieval systems have been text-based, relying on the annotations or captions associated with the images. Although text-based information retrieval methods are mature and well-researched, they are limited by the quality and availability of the annotations associated with the images. Advances in techniques in computer vision have led to methods for using the image as the search entity. Our project aimed to create an image retrieval system with a set of 1500 upper endoscopic images from the Clinical Outcomes Research Initiative Collection.