A lot of things:
First of all the world of Oblivion or better said Cyrodiil. I like the medieval atmosphere. I like the cities and castles. I like the terrain in general, mainly temperate climate with little variety at the province edges.
I like the guilds, DB and TG are fantastic Mages guild is very good too, fighters guild while more linear it is still ok.
Magic in Oblivion was better.
Arena is a very nice addition, adoring fan is a plus even if he is a jerk
.
Knights of the nine I liked a LOT because I like the medieval chivalric order mood, and I love the armor.
(I did not like Shivering Isles as much as the rest of the people here - I like classic, temperate climate medieval setting more. I like classic Skyrim better than SI)
Town quests were good.
What else should I say? Many more unique characters, like Glarthir, Janus Hassildor, Thoronir, Waterfront people, etc.
The game engine in Skyrim is much better of course, but content wise only the thieves guild questline and the civil war is on par with oblivion. That by all means does not mean Skyrim is bad, we were simply spoiled by the excellent content of Oblivion. I am sure that such good content could be (easily?) added to settings running on Skyrim engine and I hope modders will get to that.