They were scared of more than just jazz music. Once black people got the right to vote white people realized that they were quite diligent voters.
The passage of the Fifteenth Amendment in 1870 guaranteed the right to vote to men of all races, including former slaves. Initially, this resulted in high voter turnout among African-Americans in the South. In the 1880 United States presidential election, a majority of eligible African-American voters cast a ballot in every Southern state except for two. In eight Southern states, Black turnout was equal to or greater than White turnout. At the end of the Reconstruction era, Southern states began implementing policies to suppress Black voters.
After 1890, less than 9,000 of Mississippi's 147,000 eligible African-American voters were registered to vote, or about 6%. Louisiana went from 130,000 registered African-American voters in 1896 to 1,342 in 1904 (about a 99% decrease).