In a way, this is correct. But some people's anxiety is so bad, that immediate exposure might make it worse for them. Like you said, gradual exposure might work the best. This is what I did to overcome my anxiety. I had a fear of talking to new people and it took me some time before I was even able to talk to strangers. What I did was, I tried talking to acquaintances first. And then, into friends of acquaintances until I was able to talk to new people already.
I also had this fear of public speaking. But to pass several classes, I had to do several reports. Believe me, it scared me at that time but I had no other choice. With practice and exposure, I was able to do these several reports and even defend my college thesis. The feeling after was quite unexpected, I was happy and excited.
So with time and gradual exposure, it can surely happen. Don't beat yourself up if it doesn't happen as quickly.