"If you want to be a pilot, you have to be very good at math." That was the advice of my elders as I worked my way through grade school, and it's an often repeated sentiment today. But it's a lie. You don't have to be a math-whiz to be a pilot.
Most of the private pilot curriculum focuses on physical hands-on skills along with the many regulations governing those skills. Math only creeps into the equation when we talk about the subjects of flight planning, aircraft performance, and weight & balance. But none of those things are very complicated, and anyone who can add, subtract and multiply can do this. If you have trouble with arithmetic (don't we all?), don't worry -- calculators are allowed.
Even some of those calculations are going away thanks to modern technology. With fancy flight planning software, you can plot a course for adventure on an iPad. It's as simple as telling the program where you want to go.
If you are planning to go all the way and take flying to the professional level, there will be a few mathematical hurdles to get through. It mostly has to do with fuel planning in more complicated airplanes, but it isn't much more difficult than the stuff you learn in Private Pilot 101. You can do it.
So why does everybody say that pilots have to be good at math? Back in the bad-old-days, professional pilots had no choice. There were no flight computers, other than the trusty E6-B. Pilots and flight engineers had to interpret a wall of gauges and calculate fuel-burn, ETA, groundspeed and more by hand.
I grew up with a lot of kids who wanted to be pilots, but most of them gave up on that dream citing the excuse, "I'm just not good at math." Don't believe the hype, it's not that tough.