Bravo head honcho Andy Cohen writes in his new memoir, The Andy Cohen Diaries: A Deep Look at a Shallow Year, “[Neil Patrick Harris] and I had a quick meal after the show and talked about a lot of gay stuff, including debating Kevin Spacey; I still get enraged when I think about him talking about being in love with that woman on ’60 Minutes.’ Come out, sir.”

While some might be shocked to hear Cohen label Spacey as gay, rumors about the “House of Cards” star’s sexuality have regularly appeared on gossip sites for years. And he certainly isn’t the only celebrity who has faced gay rumors. In fact, it’s been said that you’re no one in Hollywood until someone has said you’re gay.

Though in recent years it’s become easier to come out and being called gay is in many ways less traumatic both personally and professionally for individuals, even the faintest whiff of queerness can still mean disaster for celebrities’ careers. Therefore, some of the biggest stars in film, television and music have reacted to these claims in a variety of ways. Some deny the rumors, some laugh them off, some see them as flattering and others flat out refuse to even address them. And sometimes the rumors turn out to be true.

In the slideshow below we’re taking a look at 40 stars who have faced gay rumors. We look forward to the day when, just as a court ruled in 2012 that calling someone gay could no longer be considered slanderous, someone’s sexuality is something that’s celebrated, not speculated about, and we no longer have the need for closets or beards.