You look like in need of an organization. Figure out the time with your miss to take care of the kids while the other has a free time to do what he/she likes. Also hire a nanny for one afternoon on the weekend.I used to be a regular at the gym, lifting at least 4 days a week. It was my passion. My life's work. My love along with videogames.
How far is the gym? By walk, public transport or car? Can you go in the evenings or early mornings before work? Can you have a small gym at home or in a garage (if you have one)? When we had a single kid I was at the gym at 6am, finishing at 6:45, jogging 5 minutes back to apartment, taking a shower, waking up the kid.
Your kids are 5 and 11, they are perfectly capable to survive on their own and express their needs. My 3.5 year old daughter is basically on autopilot now, it's the 18-month old that requires constant supervision.
I really don't understand what are you complaining about.