No, I don't. The market, or businesses are also able to use force.
Let's take the case of Mr. Zeku, a black American who lives with his family in Belleville, a small town, where only 20 black people live. To keep things familiar with this thread, let's say that suddenly a girl is raped and murdered by 4 black people, which Zaku doesn't even know. That girl happens to be the daughter of the CEO of ACME Electrics, an utility company providing electicity for all areas of Belleville. Needless to say, relations with the other 16 black people and the rest of the population are now much less friendly.
Our enraged CEO then decides his company will no longer offer service to any black people in that area, and he also convince his good friend who owns the two local groceries to deny black customers accesss to his stores. Now Zeku has to drive over 30 miles to do his grocery, which is still doable, but if he wants electricity, he'd have to shell out $100K for the closest competitor to reach his house, since they don't currently cover the area.
Since Zeku had a job working with the public, his boss, fearing for his sales, fire him. Now the only option left for Zeku is to sell his house, most likely (for a really low price, taking a loss since people are well aware of his situation), and move with his family to another place.
This may sound far-fetched today, but it was only a few decades ago that black people were forced to take the back seats in buses, or were forbidden to take buses reserved for whites.
Now I'm guessing you don't see anything wrong in what happened there?
Racial segregation and/or discrimination is deeply evil only when done by a government?