The first settlement at La Fargeville was made without title to Dr. Reuben Andrus, of Vermont, who in 1819 erected a log mill on Catfish Creek, within the present limits of the village. From this the place acquired the name of "Log Mills" which it long retained. On the occasion of a Fourth of July celebration in 1823, a resolution was passed giving the place its present name of La Fargeville, in honor of John La Farge, the proprietor of the site.
In 1838 the mansion and farm of Mr. La Farge, one mile south of La Fargeville, was purchased by Bishop Dubois, and a Catholic seminary named "Saint Vincent de Paul" was opened under Reverend Francis Guth and several assistants. It was designed to combine in this theological seminary for the education of priests, and a classical boarding school — the more advanced candidates for holy orders serving as teachers during part of the time. It was soon determined that the location was too remote and it was removed to New York where Saint John's College (Fordham) was founded soon after.