It's a toss up for me between three separate species:
1. The Springgan - Honestly, this may sound weird, but no beast in Skyrim frightens me as much as the Spriggan. Their whole presence creeps me out, the way they lurk in old ruins that have been reclaimed by nature, silently standing guard and suddenly appearing to drive away 'intruders'. It's also disturbing how they screw with your mind, making the screen turn green and resurrecting themselves just when you think you've killed them.
2. The Hagraven - Unnatural beasts and all the scarier for the fact that they were one completely normal women. They have artificially turned themselves into these withered, growling, clawed abominations.
I remember one time I was exploring and came across a small hill. I was instantly set upon by bandits and naturally assumed they were just protecting their loot. Upon reaching the top of the hill, I found their leader - a Hagraven! Complete with her own tent and a severed Spriggan head resting on a pike outside.
It's scary enough to consider what they've done to themselves, but it's downright terrifying to think of them wandering through Skyrim and gathering followers to aid them in their rituals. Brrr....
3. Vampires - I don't find them scary at all, but their sheer presence in Skyrim was the reason I purchased the game. And, sure enough, I am now a Vampire myself. Mostly, I feed on Lydia as she sleeps, but I also have access to an old womans house and use her as an alternative source of blood when the sun goes down. Vampires in Skyrim are just awesome, it has to be said.