You should have plenty of time for him to grow. He will get as big as he is meant to get. You just need to give him the proper nutrition in order for him to do it. Most of my horses grow well until they are 5/6years old. I have had a few BIG geldings mature until 9. I have on mare who is smaller than I would have expected, but it turned out she had a medical reason for that :/.