I would say that the voice acting is by far the most important part of acting in terms of animation. Voice conveys everything. Emotion, tone, and all the other things that go into making a scene work. The physical movements are unimportant. Especially since it's not even like live action where a bad actor can be doing physical things that are bad. No animator is going to draw characters looking bored or gazing off-screen.
But animation isn't acting. It's just the physical movements. And I would argue that good writing can stand up to bad acting. Babylon 5 is one of the best written shows of all time, yet it often had actors who weren't exactly world class, and often looked like the poster child for 'low budget'. Doctor Who is another example. Great writing can survive bad acting.
Animation is only one part of that execution, however. If the animation is bad, and the score is bad, and the voice acting is bad, then yes, it's not going to come across. But if you have a great script that simply doesn't have great animation, that will come across just fine. Just as there are plenty of examples of great movies or TV shows with bad sets and shoddy camera work that are still recognize for being great.