Words only have as much power as you give them. If you ban certain groups of people from using certain words, you only create disenfranchised classes, and prevent wounds from healing. Besides that, it's rather stupefying to me to propose that an entire race of people are so incredibly weak that they cannot handle the usage of a word. Should we really damage the entirety of a democracy so we can hold a word so thoroughly sacrosanct in the department of offensive terms, that it will never change as a result?
I mean, think about it. The more you hold something up as an example of damages to a group of people, and push it in everyone's faces as such, the slower and slower you're going to make real progressive change to happen. Besides that, you can make nigger out to be the "worst word ever made" as much as you like, some people who don't care will keep using it anyway. Therefore, the more logical course of action is to teach people not to care. The less people care about that word, the less power it has, the sooner we can move on past it.
Granted, is it polite to use slurs in every day conversation? No. There's a list of words that are considered impolite to use in modern conversation. "Fuck", "shit", "cracker", "paki", "nigger", so on. Social skills help. If someone is running around screaming "LOL CUCKHOLDING SHIT FUCKER NIGGER PAKI CRACKER POP SKULL NUGGERS~" You should probably just ignore them. They're probably a thirteen year old on XBox Live who just discovered the magical world of horrendous language. Let them get it out of their system, it's a phase, lots of people go through it. If they're an adult doing the same, they never learned, and if you choose to associate with them, that's your problem. If someone harasses you on the street screaming "fuck you nigger" or similar, ignore them. They're only going to hurt themselves in the end when their everlasting negativity comes to make them lonely.
Words only have as much power as you give them. People only have as much power over you with language as you give them. Just let it go... And you can, let it go. The only people who can't are so beyond damaged in the head that they need to see a psychiatrist.