My first backing tracks were on a Yamaha 4 track cassette player. I had to have the higher level model because it had speed/pitch adjuster knob. I recorded guitar, bass, harmonica and some drum machine, probably a TR-707. Also had Akai, Tascam and a few others. I actually wore out several 4 track cassette players.

About 20 years ago I went to Band In A Box tracks recorded to Sony Minidisc Walkman style. Velcored to a music stand. I used the Walkman minidisc for about 3 years or so. It was not the kind you could transfer files via USB. I had to record each song to it manually and the discs only held about 1 hour of mono music. I still have stacks of mini discs all organized and labeled and a few blanks also. My son, age 30 still used the same Sony mini disc Walkman and the $300 Sony stereo mic option I purchased with it to sample sounds in the field.

Next came the Archos mp3 player. Velcroed to a music stand. That saved a lot of time and all my songs plus a bunch of other stuff fit nicely organized on it's 20 gig hard drive. Now I could transfer songs via USB, rename then, make play lists, etc... This was several years before iPODs and MP3s were popular. Many people were amazed that all that sound could come out of a little box half the size of a deck of cards. The Archos G-Mini died out after about 4 years. I would still be using the Archos G-Mini if it worked.
I went a several years with no backing tracks and now using a inactive $69 Android phone from Walmart. It requires no proprietary software. No iTunes, etc... I simply transfer the tracks via USB and play them with Rocket Player Ap. It really helps to have a stereo volume pedal between the Adroid and whatever sound system.

Does the noise in your head bother me ?