I know that if your character is wearing stormcloak armor in an Imperial occupied hold, or vice versa, the guards in the hold will question you about why you are dressed that way. Depending on your answer, they will either attack you, or they will give you fair warning that it's a bad idea to be dressed that way. What happens if you're wearing a mix of stormcloak and Imperial armor? (I.E. Light Imperial armor, light Imperial bracers, light Imperial boots, with a stormcloak helmet.) I have never tried, because personally, I prefer to max out my alchemy, enchanting, and smithing skills, and create, improve, and enchant my own badass Daedric or Glass armor.