The community now known as Ulmer goes back for many years. The surrounding lands belonged to only two persons - Ephriam Ulmer and John Williams. There are authentic records dating back to the 1700s. The ancestorial home of the John Williams family still stands. It was lived in until sometime in the 1950s. The first building was erected on the banks of the Salkehatchie River. The site was near a settlement know as Buford's Bridge - which was there prior to the Revolutionary War.

While a proper name was being decided for the town that was building around the depot, it was jokingly called "too nigh". Streets needed to be laid out for the new town, but this was going to take more of Mr. Ulmer's property. However, Mr. Ulmer agreed to reliquish extra property in exchange for the town being called "Ulmer" in his honor. This was agreed upon and the town of Ulmer got its charter on August 7, 1891.