I found that my aversion and dislike for math mostly stemmed from the fact that I had had terrible math teachers in high school, and that I never did my homework. When I got to college and had good math professors, and studied and did the coursework I found two things out. Math wasn't as hard as I thought it was. Math can be interesting and very satisfying.
Now, that's not to say I didn't have to work at it. I would do a few of the assigned problems from each lesson, but more importantly, starting two or three days before the exam I would do every single assigned problem for the material leading up to that exam. I did this and got As in every single math class I took (aside from the one course I DIDN'T do this for, statistics).
There is also the math that is more directly related to computer science. Discrete Mathematics. This will be unlike any math you have ever taken in many ways, but it also feels more more tangible. This is more akin to the things you actually do in computer science every day.
Also, if math is really not your forte, look into what your university requires. My university only required up to Business Calculus (basically calculus without trigonometry). That's what I took and it was a breeze. I actually wanted to take regular calculus, but due to scheduling constraints it was business calc or not graduating on time for me. I've been going back over the time since college and doing it on Khan Academy and the like though, just to stay on top of it.
In my day to day life I don't do THAT much math. Some statistical stuff, some linear algebra, some trig and that's about it. I guess you can count constructing mathematical formulas into computer code as "math" but it's really not that much. A lot of whether or not you'll heavily use math in your career as a computer scientist really depends on your area of focus. I am in the maritime domain and I do stuff with GIS so I use the trig and linear algebra due to that. A lot of it I hate to teach myself because they don't really teach you non-euclidean geometry much in formal classes.
All in all, it's worth it. Try it out, work hard at math, and you might find you like it.