The hamlet of Stone Mills, at first called Collins Mills, was one of the first areas of Orleans Township to be developed due to the availability of water power from the Chaumont River. Peter Pratt (born Saybrook, Connecticut, 1776) and J. B. Collins built a stone grist mill here in 1819 or 1820, and soon thereafter a stone, up-and-down saw mill. Both are no longer extant. These mills provided grain processing services to farmers in the southern part of town at a time before LaFarge purchased most of the land in the region. Stone Mills grew on its own with a school house, Stone Mills Union Church, and the Irwin Brothers Store constructed over the next few years. It was during this period that Elijah Horr built his large stone Elijah Horr House at the southern edge of the hamlet.