The Tempo Map should get used on all tracks, unless you opt out of it during certain conversions.
I know of no settings that would cause this.
Like I said, it usually just works here.
Sometimes something weird like bit/rate mismatch can cause issues, but your issue sounds nothing like that.
_________________________
Make your sound your own!
.. I do not work here, and the benefits are still awesome