It goes in phases for me.
First it was trolls .. the ice troll blocking the trail to High Hrothgar was a whole new kind of challenge for me at that level, followed by the one underneath the Thalmor embassy.
Then it was Drauger Deathlords. That mission with the thieves guild dude is memorable for its difficulty. At the time I didn't realize they were just disarming me, not destroying my weapons, so I ended up using losing my good bow and sword, picking up ancient nord weapons, drinking all my potions, actually eating food, and then having to run away, hide, turn into a werewolf and eventually survived with a sliver of health. And that was just the first batch of them.
Then it was the Foresworn, especially that big gang of them outside the Sky Haven Temple. First time I faced so many decent casters.
Lately it's been the random ancient dragons that seem to show up when I'm out in the open, in the middle of a lake or something.
But overall I think the most consistent worst enemy for me are the Foresworn. If I can survive the Ice Storm casting mages and archers, I eventually run into a Foresworn Briarheart, which easily one-shots me, even in heavy armor, if he gets within melee range.