[MikeWaters]

You're not too old...but it may not be the best idea.

5 or 6 years to get your PhD. Then 1 or 2 post-docs. Then a real grind after that as well trying to get your career going and get tenure (you are not going to study poverty and brain development in the private sector).

Brain development of humans? Or animals?

Using tissue? Or scans?

I'm assuming you would be doing humans/scans. In that case, probably the more appropriate field is psychology (PhD). or psychiatry (MD). One advantage of the psychology degree would be the opportunity to be clinical, as an additional career option.

With your math background, maybe you are more interested in epidemiology and health services research....that's another thing to look into, and can be approached from a number of different fields/directions.