Forget considering either version based on extra content, or controls, those are only skin-deep comparisons. It goes further than that.
The DS version messed with the level designs to accommodate the ill-suited controls (some platforms were made larger for example) but everything else added either did nothing for the game or took away from it. For example, the extra characters mess with the carefully designed level structures - one platform in Snowman's land once required Mario to bounce off an enemy and twirl gracefully towards it, now it can easily be circumvented using Luigi. Likewise a lot of the challenge was dumbed down, perhaps patronisingly so. Balloon power-ups were placed in locations where Mario once had to deftly wall jump, among other changes. Basically, the DS version asked less from its players, in a game that's totally about experimentation and mastery of the controls.
And the overall aesthetic of Mario 64 DS lacks the charm of the original. Mario 64 had textures that were designed to be blurred, and DS lacks texture filtering. The new textures change the look of the game a tad too much in some areas. Likewise its character models are a product of a time when Nintendo was finding its feet with 3D rendering and design.
Mario 64 DS is still a great game. But it's just such an awkward adaptation, back when Nintendo was probably unsure of what the DS was actually for.
tl;dr: You should play the N64 original. The DS version fundamentally waters down the level designs and expects less from you, the player.
