I have usually played as a lone-wolf too - it may go back to those bad memories of M'wind or Oblivvy "escort" missions (where you had to get a suicidal someone with 2 health-points and a pointy stick past a bevy of enemies...).
Lydia drove me nuts, and I happily "sold" to the Blades as soon as posssible - which made my Breezehome much quieter. Never really trusted her; she had her own bed, but always just sat in that chair staring at mine. She kept saying how keen she was to help me; but if I gave her lifesaving gear, she'd moan about it like a sulky teenager.
Then I hired that Marcurio fellow, thinking his magical zapping would complement my whacking nicely...not so much. Sold him to the Blades too. Erik the Slayer, I just kind of liked. At first, he seemed like the nephew I never had... but soon, the Blades had all three wanted recruits.
One bother with followers (for me) is that I feel responsible for them. Maybe it's "hangover" from those escort-missions of old, but if I have a follower, I get distracted worrying they're okay. I took Iona out on one mission... and spent more time worrying about her than whatever it was I ("we") was up to. Terrible things happened, best not to speak of.
THAT SAID, I have been well-served by Jenassa, who has saved my life many times and been a great co-fighter. The irony(?) is that she's a one-handed fighter/bow sort, same as me; and the two of us have worked far better than my attempts at other-type followers. I made a full suit of glass for her, and some mean daggers - which she will dual-wield indoors. I wanted to practise, my own marksmanship (so I hired her to take the heat) - and she proved herself truly fierce. She also once gave me a freebie... agreeing to face death unpaid.
There is a ~combination~ of two of your points about followers I would mention. They do fall behind, and they can get in the way. Once, I went overland toward the goal, and she fell behind. There were several outside baddies there, and I had a job whacking them - and when I turned, I saw a scary looking warrior charging up the road at me, weapon-drawn... needless to say, that was her, Jenassa, finally catching up - and with her weapon out (just like they sneak if you do) - I thought I was toast...!
