That's interesting to know. I had been thinking about (when I will be playing on pc and can mod) a plain sword, which is modded to swing at dagger speed. To prevent myself from going overboard, I was thinking of giving it some type of percentage enchantment, like chance to paralyze, so it's mostly just smithed sword damage. Perhaps it would be an interesting mod to set weapon swing by weight, so a heavy dagger swings slower, while a silver greatsword actually swings faster than a dwarven or something.
To balance that, add weight to weapons as you smith them up, so they get heavier, and slower as you increase the damage. Not linearly, there should still be an advantage to smithing, so a factor of 1.5 ought to do it. (If you triple the damage, it only doubles the weight.) Daggers generally have the bet damage/weight ratio already, at about half the weight for a given damage. (With the exception of the Skyforge Dagger, which has Steel weight, and Elven damage.) All others are heavier than their base damage.
If I were doing it, all weapons of a given type would have the same base damage, but higher quality materials would make them lighter. For swords, and daggers, anyway, Axes, Maces, and Hammers deal damage as a factor of their weight, but then we'd have to factor in your speed adjustments. This would, of course, drastically alter the balance of weapons. I would certainly do this with Armor, improvements only slightly increase the protection, but more drastically decrease the weight, though this would eliminate the need for the Weightless perks, and Steed Stone. (For that, I'd just go back to the 3rd era Speed increase.)