Hey Bing, how about all the crap being spewed at the "white, American, christian, veterans"? No one (and I repeat, no one) has done more for the world advancement generally than those, and no one is more universally ridiculed or even hated because of it.
What the hell do you feel guilty about? I don't. I've never owned a slave (or known one), never vilified anyone because of race, creed, color or national origin. I do take offense to the race card (in all of its incarnations) whenever played. Judge everyone on the content of their character, and then stand the hell back and watch them call you rascist, sexist, or whatever ist is applicable. If you accept that, live with your guilt.
vent over, have a nice day.