Early white settlers began inhabiting the area around 1790. The county was form from Tioga County in 1836; in 1854. The Chemung River (a tributary of the Susquehanna) flows through the southern part of the county.
The county is home to four historic districts that are listed on the National Register of Historic Places.
- Elmira Civic – comprised of six county buildings, the city hall, an armory used by the city for office space, a large post office, a YMCA and a YWCA, two private social clubs, a library, an art museum, a church and a building intended for the Chemung County Historical Society.
- Hanover Square – the business district of Horseheads Village; 18 of the district's 21 buildings are Romanesque Revival designs.
- 1855 Extension – primarily residential district enhanced by the Union School Building (ca. 1890 by Joseph H. Pierce and H. H. Bickford) at Grand Central Avenue and Fletcher Street.
- Near Westside – Elmira City; more than 70 acres with nearly 500 19th and 20th century domestic structures.