When words hurts

It has been quite a while since I let people's words make any impact on me. I've promised myself that I won't let negative comments get to me and bring me down.
And for a long time, I was happy as I am. But for some reason, when an unreasonable comment (or a so-called joke) goes south and it caught me at the wrong time, I just couldn't avoid from getting hurt. I know that those negative comments are not true, but somehow, it still hurts.
I don't deserve to be treated this way, so I want to eliminate those who brings me down out of my life. I don't need them around, nor do I need to hear to those ill-comments. So please go away.. SHOOOO...!!