Is this a reason to just stop talking to a really close friend? Just recently broke off all contact?

Probably not. He makes jokes at your expense! Friendships are fluid. Sometimes, they serve their purpose and we move on without them.