As I am a hoarder, I would only use Lydia & co for carrying my gear.
Even in the mini missions were you need to help out someone random outside a cave, and they ask can they come, I always say no, as they are so annoying, and I have killed so many people with the Dark Brotherhood, by...