Math ain't nothin' but trouble.

So, once again, I was wearing my math shirt this weekend and I get an inordinate amount of attention from a gay man. Except this time, I had completely forgotten about the shirt and just couldn't figure out why this guy was smiling at me so much. It wasn't until I was telling Anne about it that she pointed out I was wearing my math shirt at the time. It's definitely of flattering, but why does this only happen with the gay folk? I mean, I wear non-GLBT affiliated clothing all of the time and girls consistently fail to throw themselves at me. I just don't get it.