You should read Lance Hill's The Deacons for Defense: Armed Resistance and the Civil Rights Movement. Persuasive argument that, despite the revered narrative, nonviolence as a tactic wasn't particularly successful, and violence and the threat of violence were absolutely essential to the changes that took place. SNCC's peaceful local organizing in the early '60s actually didn't bring about many real, tangible gains: months-long campaigns that succeeded in registering minuscule numbers of voters. Intractable white power-structures, racism, relentless Klan violence; King's "moral suasion," his hopes to shame Southern whites out of racism, failed utterly. In the end the activists had to turn to the coercive power of the federal government. But the government refused to act for years: nonviolence, numberless complaints submitted to the Justice Department, moral appeals, didn't bring about legislative reform or enforcement of the laws. Finally in 1964 things were threatening to get out of hand, with riots and some white deaths, so the government was able to pass the Civil Rights Act -- and enforced it only sporadically, usually when compelled to by violence or its threat. The Deacons, who didn't eschew violence, were in some respects far more effectual than SNCC. They also did more to restore dignity and pride to working-class black men (think of Frantz Fanon), many of whom saw nonviolence as just a continuation of age-old submissiveness and an excuse for Southern whites to persist in their views of blacks as children and cowards. By fighting back, blacks forced local whites to take them seriously. Nonviolence was a useful tactic for getting white liberal support, but without the threat of black violence always lurking in the background it would have accomplished little. ("One of the great ironies of the civil rights movement was that black collective force did not simply enhance the bargaining power of the moderates; it was the very source of their power.") After all, states are not moral agents, as Chomsky says, that can be swayed by moral suasion; they are power-structures that respond to attacks on, or threats to, their power. In the end, even the reforms of the civil rights era did little to materially improve the situation of most blacks, blacks in ghettos, in rural poverty; they were better for the middle class. (It's the old revolutionary pattern of poor people doing most of the real work and middle-class people getting most of the rewards.)
Pacifism is admirable but wrong, at least if you want change. For that you have to look at tactics, not adhere blindly to principles.
 Also, it was sort of a fringe movement -- with the support of the powerful.