Thanks for your thoughts on this.
The counter argument is if you bring up a ballad and decide to play it as a tango, by changing the style, it sets to an appropriate tempo for the new style; or changing from bossa to fast swing—unless you need to, your tempo is changed to match the new style.
However I know what you mean

As an aside, there are slight differences in the jazz styles as they are defined by speed (medium, fast etc.) which is why there is not just one (as the backing instruments do slightly different things depending on the tempo).