So too does Legate Rikke, Hadvar, Hadvar's Uncle Alvor, Kodlak Whitemane, countless others within the Empire. Yet they have not picked up arms in the name of Ulfric Stormcloak. The Ban of Talos has been in effect for twenty six years and there have been no incidents save one recorded instance, a bard who sang about Talos in public for seven days in a row in Markarth, which is the Thalmor Justicar headquarters.
There were a large number of citizens of Skyrim, who didn't pay much attention to the treaty, everyone still had their little shrine to Talos. The Empire did a very poor job enforcing the Talos Ban, even in Empire supporting cities, the Thalmor actually have a very hard time making arrests, without evidence. Such as the quest given by the Thalmor in Markarth Keep, to break into a Nords house to find evidence of Talos worship because the Empire supporting Jarl won't allow the Thalmor to arrest him without evidence.
The Thalmor are very limited in what they can actually do while the Empire is watching them. That is why they caused the Rebellion and are aiding Ulfric Stormcloak and his Militia. The rebellion causes the Empire to waste resources and soldiers, taking many Guards and soldiers off the roads and out of the cities to fight this war, this allows the Thalmor to arrest anyone they like since the Empire isn't around to stop them, they're busy dealing with their asset, Ulfric Stormcloak.
"We didn't pay much attention to it when I was a boy - everyone still had their little shrine to Talos. But then Ulfric and his "Sons of Skyrim" started agitating about it, and sure enough the Emperor had to crack down."
The rebellion isn't strictly about the worship of Talos, nor is it about making Skyrim independent. It is a war of ego, Ulfric wants songs and stories, a feature about him that irritates Galmar during the Battle for Solitude. Skyrim could of been independent and free of the conditions of the treaty without a single drop of blood being spilled. But peaceful negotiations aren't tale worthy of a mead hall. Torygg greatly respected Ulfric, admired him well. It was his mistake, trusting Ulfric Stormcloak to do what is best for the people of Skyrim.