Some of the known pioneers came to Indian territory before and just after Kemper County was formed in 1833. They built a church in May 1835 to worship. The charter members of the church known as "Mount Olivet United Baptist Church were as follows: Enoch Spinks, William Fox, Dunken Joiner, Lewis Stovall and James Stovall. Sisters Elizabeth Spinks, Prusilai Fox, Sarah Ann Fox, Macrinia Attwood, Marl Harrell, Nancy Carter and Harriett were women who were charter members.