It wouldn't have. Please tell you didn't listen to the Shashi Tharoor speech and think this up. I've seen too many having listened to that and buying into what he said.
If it wasn't for the British, it would've been the French and Portuguese who would've ruled India, but the British were just stronger. There was no 'India' when they came, because before the British, it was another group of Persian rulers called the Mughals who had invaded and conquered the land. There was the Sikh Empire, the Mughal Empire, the Maratha Kingdoms and many other monarchic dynasties all throughout. Thinking there would be this national unity country, super rich and wealthy is just flat out false.
Also, when the Japanese invaded Burma, their next goal was to invade India as well. If it wasn't the British, it would've been someone else. At least the Brits unified the majority of the country, tried to settle the sectarian strife that had erupted primarily amongst Muslims who wanted their own country to rule. The biggest lingering effect isn't that material wealth was stolen, or labor used to power the British economy, but that the Partition displaced 15 million people overnight and that almost 500,000 to a million died. Primarily Muslim cities were part of India, most of Sikh Punjab became Pakistani/Muslim, East Pakistan would never be unified with west because they were Bengali whereas western Pakistanis were primarily Punjabi, this in turn leads to civil strife culminating in the liberation of Bangladesh and the Indo-Pak war of 1971.
I think every group of people can acknowledge that their ancestors did terrible things, that almost all empires were built on the backs of others and that you can find anything that can cause you to not be proud. I mean why wouldn't Mongolians be proud of Genghis Khan and his dynasty despite how brutal it was?