In the centre of the Audio Preferences Panel, check "Output Buffer Milli Secs". This setting applies to MME drivers only and has a range of 30 to 4000 ms. The longer time does result in volume change delays whereas the shorter time can result in segmented sound. Experiment to find the setting that suits your setup. I believe the default setting is 4000ms.