I've worn my contacts at the Deid without any problems.  On some of the dustier days glasses might be more comfortable, but that's about it.  I don't like wearing the normal-looking glasses that the AF gave me when I fly because they bend easily/don't fit well with a headset, and are too narrow to work well with NVG's.  I've never met anyone who gave me a hard time about what glasses I wear, but per the reg the only ones you're technically allowed to wear are the ugly ones (including sunglasses).  Like I said though, nobody really seems to care, so do what works best for you.