I take a number of meds, not all antidepressants, and have not noticed a dropping in my IQ level. I have noticed more lethargy, both physical and mental. That just means it take me a bit longer to pull together a thought or come up with that certain word.
It's interesting to note that most people don't hesitate to take meds for migraines, cancer, pain, osteoporosis...but there is a stigma with regard to taking antidepressants.
I would propose that many of the men and women to whom we look with respect are taking or have taken antidepressants. Certainly many great people suffered depression before a.d.'s were available. Would their IQs have been affected by taking meds? I think probably not.
If I believed that my IQ were being affected by my meds, I'd certainly have a talk with my psychiatrist who prescribed them, as well as to my primary care doctor and the psychotherapist who is counselling me. Yeah, there are side effects, but for the most part, the benefit has far outweighed the inconveniences. A case could be made that medicated people just have to work a bit harder to maintain their equanimity.
Don't be paranoid about antidepressants. Certainly, be informed. There may be a point at which you need to place some trust in a doctor who can help you with medication therapy. Don't avoid it simply by a fear of being dumbed down. Maybe avoid it because of allergies, but don't let fear drive your decisions.