I read the explanation in the OP and it makes sense reading that.... but when I use the color picker in PS it says the dark area is brownish/gold. That's the picture I see.
I can see the dark dress if I turn my phone and look at an extreme angle though.
Right, because of how our eyes can interpret colors, you may see the brownish/gold as darker than it really is.
There is a picture of a multi-colored cube with a shadow somewhere in this thread. The center of each face is brown, but the center in the shadow appears orange/yellow, even though in PS or whatever picks it as brown. Because of the shadow our eyes/brain instantly assume it has to be a different color, even if it's not. Same logic with the checkerboard test and shadow and with this picture.