First, you are speaking of different things, Savari is an assasin, the justiciars are not, they are official agents tasked with rooting out Talos worship throughout Skyrim, to do this, they are bound by specific terms dictated in the WGC, to which they have to abide.
And which town are you talking about ? Whiterun ? Heimskr gets thrown in jail (thank you Talos) if you side with The Empire in the Civil War.
That's not gonna happen, Skyrim is very far from the Summerset Isles, the trip alone would cost any state an inmense amount of resources just to keep the soldiers alive; it would have to be a one way trip throughout the dangerous waters of the Sea of Ghosts, since they wouldn't be able dock in Hammerfell or High Rock.
It's insanity, even in the case that the Dominion army somehow makes it to the coasts of Skyrim, they would never be able to hold it, since Skyrim is surrounded on both sides by High Rock and Cyrodiil.
If The Empire did such a thing, then it would come to be after either The Dominion or The Stormcloaks won, it makes no sense to attack both sides at the same time, the best option would be for The Empire to hold it's armies and wait for whoever is left standing.
As for Valenwood and Summerset, highly unlikely as well, they'd have to leave Cyrodiil defenseless if they campaign through those provinces, which would give the Dominion a better chance to retake the imperial capital were they to take a Stormcloak-controlled Skyrim, which they easily could.