Each fracture is different. I would consider getting a second opinion from someone who specializes in sports medicine. They'll be better able to tell you how to manage your injury and what, if any, limitations you'd have. I suffered a stress fracture of my L1 but it was very stable and has never caused a problem. It sounds like your injuries are a lot different.
I think you'll be fine if you use caution and common sense. IMO there's no "100% safe horse" out there if they've a pulse. Getting past the jitters and the mental part's half the battle.
I had compression fractures of three cervical vertebrae in an auto accident in 1995. I'm shorter now than I used to be! I was not riding at the time of my accident and did not get back into equines for several years after my accident. I wish I had gotten some PT, I lack range of motion in my shoulders and neck and my back is very crabby now. I ride with no complications though and riding does help me relax.