It does seem likely that there are other dimensions that our senses can't really perceive. They may be microscopic dimensions that just affect quarks, or a hypersphere that encases our universe.
We have such limited ability to look and measure at this point that our only real indications are that this theory makes the math work. However, making the math work is a decent indication that there is probably something there, even if we haven't figured it all out yet.
In maths we can have infinite dimensions, we can have negative space and time, we can divide zero by zero etc. All these things don't exist in real life. In real life our universe seems to behave and show properties that are predictable. We have seen the start of the universe and by using black holes we can predict the end. All of this is clearly seen and observable and not hidden in some 4th dimension.
Even the Gods mentioned in our holy books could not go back to time, think about it for a second. Scientists claim that they are myths yet even they didn't dare say that God could go back and change anything they wanted.
So we have to make some fundamental rules about our universe based on true data and things we can observe rather than just using maths otherwise we would end up with more theories like the string theory.