Well, I thought I was pretty hot-stuff because I can do B7's and Bm's no trouble. And I can, pretty much. The problem is the time it takes to get there. And if you're a second slower getting there than going between other chords, then it really shows up.
Practice! Practice! Practice!