that's very true, and if you want to dual play with someone, they might have the restrictions that an invader would have, in which case, to them everyone would be immortal and they would have little freedom, and as you say, it may be malicious, aditionally considering the skyrim map is big and i fail to see how the invader would even end up anywhere near you, they could mess everything up in your village or steal the dragon armor you have stocked up in your house and leave before you know they were there. i think it would be impractical if the invader was to just pop up infront of you ready for battle when you are on low health after just fighting a daedric priest and they just chopped you down in one shout.
I think if there was to be dual play, you would possibly have to do a quest chain where there are seperate planes that are like yours and you can create a portal for someone to cross over. Then you learn a spell that creates a blue portal and in the selected friend's world, a blue portal appears nearby and they get a notification that it is there and who sent it, they can then just step through into your world.