This is why I either use Shadowmere or Arvak. They are completely free. Shadowmere is nearly unstoppable, and Arvak obviously can't be killed.
You can choose either one according to how you like to play. Arvak is for those players that get annoyed easily by horses that try to help you fight and fast travel with you. But keep in mind, when you fast travel, if you start walking away, the horse doesn't follow you. So I don't see the problem here, but anyways. Arvak can be summoned anywhere, and can't die or be lost, so that is handy. And he also looks totally kick ass. But he rarely helps you fight considering he just "dies" and goes back (and he makes this horrifying sound every time he does, it bothers me to hear a horse like that, it reminds me of my own horse Roxy).
Shadowmere is more so for people who don't mind their horse fighting by their side. You don't pay for him, so there is no waste of gold. But I have seen this horse take on TWO dragons at once, with no help from anyone, and he did perfectly fine on his own. You can stand there and beat the plops out of him with legendary enchanted daedric and dragonbone weapons, and he's totally fine. That horse is evil I swear with his cool red eyes. So really the only way I've ever seen him die is by falling off a cliff, but even then he respawns, so no biggy. Personally, I typically prefer Shadowmere. I have a love for horses because I actually own them, so I don't mind him fighting by my side. Plus, he picks up the slack and gets me out of tight spots by distraction. Also, he has a unique saddle blanket with the DB symbol. Lastly, his stats are higher than all the other horses. Oh, and he has red eyes, I thought that was cool....
But, for the normal horses, it is a complete waste of money. Absolutely agree here. They die so easily, so I never wanted to take them anywhere with me. Since I knew they were going to die, I tried not to make them fight, but to no avail. So if you fast travel where something is already trying to kill you, you can bet your horse is gonna die. This is indeed very annoying.
EDIT, I forgot to add. Plus you can fight off of horseback, which is actually handy because you take less damage. And if you see something you want to collect or explore... just get off the horse.