Yes, that is expected when taking a photo, because the camera can only see one of the two images that make up the 3D, but your eyes (or at least mine) see both images and combine them into one, in turn eliminating this "drop in color quality" that you talk about. At least that is my experience.
I have my 3DS right next to me at this moment and I tested it turning the 3D on and off many times and paying close attention to the colors. They look slightly deeper in 3D, and I have TN screens.
Again I'm not saying you're wrong, because I've seen others say the same thing as you. All I'm saying is that your experience is not universal to everyone.
Also to me the screen resolution seems to at least double in 3D compared to 2D. It's almost like going from SD to HD. Games look blurry and muddy in 2D on the 3DS to me (only on the 3DS though), which is why it blows my mind when I see people saying they never play with 3D on.