While it is true that after emancipation blacks were initially successful and did well, and that the previously pro-slavery people or people who had believed the widely prevalent idea that blacks were racially inferior to whites (this was true everywhere, not just here, and is a belief that persisted widely in Europe even up through and after World War II) pushed back with Jim Crow laws and such, blacks are not the only ones to have been treated adversely.
Take a look at the history of the Chinese in this country, who first started coming here with the Gold Rush, and later were used as almost slave labor to build the railroad from the west side. After they displayed their work ethic and business acumen, and started building numbers in many western cities (and New York), federal laws (not just state laws as with Jim Crow laws) were enacted like the Anti-Coolie Act of 1862, the Burlingame Treaty of 1868, the Page Act of 1875, the Angell Act of 1880, and the Chinese Exclusion Act of 1882, which was extended in 1892 and made permanent law in 1902, until the Magnuson Act of 1943 repealed it and allowed 105 Chinese peryear to enter the country.
Yet despite that, the Chinese have made it so they are no longer held down in any way, and indeed are outperforming in many areas.
Though white, Irish immigrants didn't always have the "white privilege" either. Many came as slaves and Irish were looked on with disdain by other whites as inferior, poor, uneducated, incorrigible, and so on.
Having been dealt a bad lot by other people groups does not have to result in permanent disadvantage of that people group. Even with the black people group, one cannot say that such perceived disadvantage is universal.
That's all a long way of saying that pulling the race card is weak, and it also isn't really the issue in the income inequality argument today. What has happened there has occurred in the last 2-3 decades. Though there were many underlying influences in our economy, the real accelerants towards the current situation were the federalization of poverty, and the whole subculture it created, the federalization of insurance, first with Medicare, then its influence on health insurance through the 60's and 70's, culminating with the ACA, which almost completed the goal of forcing America to believe the only solution for delivering good health care was socialized medicine run and paid for by the government. Along with that we had in 1963 Murray v. Curlett which kicked off the whole separation of church and state movement, which has resulted in the huge moral decline in our society that has resulted in a loss of altruism throughout the whole society. That, in turn, has affected the education received in all levels, but of interest, at the business school level. Thus you had our top business schools in the 1980's teaching their MBA's the whole corporate merger/rape theory that has vastly reduced competition in the business landscape and began the trend of consolidating money and power in the hands of fewer and fewer people. All those businesses which sprang up and flourished in the post-WWII era were whittled away. Finally, you add in the banking law changes brought about by these same business school types, and then the globalization of everything, and you have the perfect recipe for giving all the money to a very few and removing the ability for ordinary people to participate in America's wealth because their factory jobs provided them the excess needed to either enjoy life and be good consumers, or take an idea and become an entrepreneur themselves. If you couldn't see the predictions back in the Bush 1 days and before of what it meant when they said when America didn't need to do the "dirty" manufacturing any more and we would move to a service economy, you are pretty shortsighted and naive. The rapid increase in real estate prices brought on by things like the internet bubble and the rise of the ultra-rich, along with bailouts that prevented such overinflated prices on real estate and medicine from falling back to where they should have when the economy went into recession have also served to make things far too expensive for the wage earner.
