From my experience, Frost spells are situational, but worth it.
Like you said, they drain stamina and they slow down enemies, which by itself is worth it, especially if, like me, have a pure mage character and you need to make sure whatever is running towards you isn't going to kill you in one swing. So really, just because there are a lot of Nords (which isn't surprising since the game takes place in Skyrim), doesn't mean that Frost spells become less effective; it just means that you'll be killing them slower with Frost, and you want to use Fire for damage anyway.