The community was once known as "Factory Hill" due to the number of wool factories. The village was incorporated in 1891. It derives its name from George P. Philip, who built a reservoir to provide water for his mill. This in turn led to construction of a reservoir up the "mountain" in the Taconic Mountains to guarantee water to the mills, thus creating the factory hill.
Philmont is located at . According to the United States Census Bureau, the village has a total area of, of which is land and, or 3.16%, is water. The village is along Agawamuck Creek, which drops over its High Falls near the center of the village. The creek is a tributary of Claverack Creek, which flows about to the Hudson River.
Demographics
As of the census of 2000, there were 1,480 people, 576 households, and 361 families residing in the village. The population density was 1,256.9 people per square mile. There were 644 housing units at an average density of 546.9 per square mile. The racial makeup of the village was 94.53% White, 1.69% African American, 0.34% Native American, 0.20% Asian, 1.96% from other races, and 1.28% from two or more races. Hispanic or Latino of any race were 3.72% of the population. There were 576 households out of which 34.2% had children under the age of 18 living with them, 45.0% were married couplesliving together, 11.6% had a female householder with no husband present, and 37.3% were non-families. 29.5% of all households were made up of individuals and 13.7% had someone living alone who was 65 years of age or older. The average household size was 2.55 and the average family size was 3.20. In the village, the population was spread out with 28.9% under the age of 18, 8.0% from 18 to 24, 29.4% from 25 to 44, 19.9% from 45 to 64, and 13.8% who were 65 years of age or older. The median age was 35 years. For every 100 females, there were 98.4 males. For every 100 females age 18 and over, there were 94.5 males. The median income for a household in the village was $31,094, and the median income for a family was $41,944. Males had a median income of $27,188 versus $23,274 for females. The per capita income for the village was $16,162. About 11.3% of families and 13.7% of the population were below the poverty line, including 14.9% of those under age 18 and 6.3% of those age 65 or over.
Sites of interest
The High Falls Conservation Area, a property in the village, with access from the town of Claverack, is owned and managed by the Columbia Land Conservancy. High Falls, Columbia County's highest waterfall, cascades before culminating in a large pool. The water flows southwest then northwest by Claverack Creek until eventually reaching the Hudson River. The village is part of the Taconic Hills Central School District.