I'm not sure how much I use it, but I'm pretty sure it's in my active vocabulary, and it's solidly in my passive vocabulary.
I tend to think that the phrase comes from the association of south with down, and possibly with certain places that are traditionally said to be in a downward direction.