I believe that professional mastering services have (or should have) a specialized room with "perfect" acoustic treatment and high end monitors and a few hardware and software tools to master songs according to sonic requirements of today's music.
What to do if you are like me with the lack of a specialized acoustical treatment and monitors? Mastering for me just becomes an exercise in leveling the volumes across all the songs that would make up a CD. of course today, folks like to think that you can master individual songs. If it is for one song, you are going to use a compressor (or a limiter -- a specialized compressor) to compress the peaks and then boost the overall gain to get closer to what one would consider radio ready volume levels. Perhaps a multi-band compressor with a limiter on top of that is really the most you can reasonably do (
Free multi-band compressors).
Your job, as a home studio guy, is to just really mix really well. If you have an excellent mix, then mastering is not as required -- except for maybe that multiband compressor and limiter combination on the master bus.
Kevin