It doesn't matter what way you or I see it, a duel wasn't even considered between the two of them which can be seen by the conversation between Jarl B, his steward and Irileth. They are discussing the possibilites of what could happen next, and they agree that after killing Toyrgg that it is more than likely he'll use his army to storm Whiterun rather than a fight between Jarl B and Ulfric, which I'm going to reiterate was a discussion between the three of them, not an option given by Ulfric/Jarl B(Whichever side you choose). Jarl B mentions a duel in the old way because it was only recently that Ulfric killed Toyrgg in the 'old way'. Proven even more by the fact that as soon as he reject's the axe he starts mustering troops, why would he do this if the axe is a sign of a duel?
Accepting the axe = peace, not accepting = war.
Ok, so are we going to just disregard the dialogue I just posted then? That what we're doing?
Obviously, Jarl B took Ulfric's gesture with the axe the same as a challenge to a duel. That was very clearly his interpretation of it. Now whatever Ulfric is doing in the kitchen is something else - fine. However, Jarl B took it as his leadership being challenged according to the "old ways" and his intention was to deal with Ulfric man to man.
Stormcloaks for whatever reason always shy away from this part of the discussion, and you can do the same however, the dialogue directly contradicts what you're saying. A duel was considered and it also sheds some light on the duel with Torygg. There's a relationship here which disagree feel free, but it should not be ignored.
It's like I know, I have always said, what if High King Torygg had answered Ulfric's duel by sending his army instead? And furthermore, you guys talk up this duel between High King Torygg, but if you're willing to turn your back on in game dialogue that indicates a duel, then why should anyone believe what Ulfric said about his conduct during the duel with High King Torygg?
And you know, that goes without saying, you have one side who banned Talos worship, yet, ironically, puts a premium on honoring the old ways, then you have another side, who claims to honor Talos yet uses the old ways to suite themselves. Wouldn't it have been honorable for Ulfric to duel with Jarl B instead of all those people having to die? The Jarl was ready to face Ulfric... why send his army to answer a personal challenge? Because Jarl Ulfric was challenging Jarl B's leadership... that is something very personal in almost every culture I've studied. It's almost like issuing someone a "summons".
However you choose to digest this, remember one thing. While Jarl B had his day in court, Ulfric "the accuser" was not there. And, if Jarl B challenged Ulfric's leadership instead, Ulfric did not show up to defend his honor, he sends others to fight and die in his place.
So yes. If someone challenged Tiber Septim's leadership... think he wouldn't have at least shown up to answer his enemy(s)?