The thing is Wizard can be used uni-sexually along with other names for magick beings but thanks to movies and games many of the terms are used for gender bias...like witch and wizard.
In the Vlad Taltos series by Steven Brust the main character (Vlad, a male) practices sorcery and witchcraft...the former having to do with pure magical power and the latter having a more nature and spiritual approach to performing spells. What I'm getting at is that Vlad is considered a Witch regardless of his sex.
When we think of wizard we think of Gandalf and with Witch we think of a cackling old lady boiling toads...or male vs female.
Sources: past experiences and this handy article:
Magician (fantasy) - Wikipedia, the free encyclopedia