Giuliana Rancic knows you probably think she’s too thin and she agrees. But the E! host says it’s not what you think.

In a new interview with People magazine, Rancic is finally speaking out about rumors that she’s battling an eating disorder, telling the publication things have “gotten out of hand.”

“Some people were saying, ‘The cancer is probably back.’ And they were accusing me of every eating disorder. I thought to myself, ‘God, if someone really thought I had an eating disorder, what a horrible way to approach it,” she told the magazine.

Rancic was diagnosed with breast cancer in 2011 and underwent a double mastectomy after an initial lumpectomy did not successfully remove all of the cancer from her breast.

Today, while the 40-year-old is healthy, she says the cancer-suppressing medication she’s been prescribed since 2012 has had an effect on her metabolism.

Rancic calls the rumors surrounding her appearance and the attention to her body “really hurtful.”

“I’m sorry that some people think I’m disgustingly skinny, as they put it, but there’s nothing I can do. I’m lucky that I even have the type of cancer that reacts to the medicine,” she told People. “I eat a very robust, healthy, balanced diet and dessert almost every night. I’m not hiding from anything.”

In fact, Rancic says the weight loss has been difficult for her.

“I look in the mirror and it’s hard for me. I am really thin. I want to look fit and beautiful and sexy, and I can’t,” she said.

For more with Guiliana Rancic pick up a copy of People magazine on sale April 3.