A car with extreme tires going up against the same exact same car with a tweaked up engine that added a buttload of HP but with adequate tires, the more powerful car will win every time. Sure the tires help, but only if the tires are not adequate for the amount of HP, adding drag radials to a bone stock Corolla won't make you a dragster lol. Not that the PS5 is a Corolla either, it's a great system, but overall Series X is slightly more powerful, the point is that you don't look at a single aspect when it comes to analyzing a final outcome, frequency is part of the measurement when calculating a theoretical max, and the theoretical max is higher for Series X, in addition to having next gen features added to the GPU that are not present in the PS5, like hardware support for VRS tier 2 and hardware accelerated ML.
Remember the OMG PS4 Pro has 64 Render Output Units and X1X only has 32
Fortunately 32 RoPs was adequate for X1X and other "more important factors" like higher CU count, memory bandwidth and overall Teraflops ruled the day, Xbox One X takes the cake for performance all day long even if it only had half the RoPs.