If it eases your conscience, after Eorlund Graymane repairs the weapon, he says that as one who found the pieces you should have the honour of carrying it into battle. Nobody tells you that the weapon has to stay on the statue, only that placing it there will open the way (so that you can do as Eorlund wishes and carry it into battle).
The other members of the circle don't object to you taking it back, so they can't mind.
Also, what would Ysgramor want? Would he like his weapon gathering dust in some out-of-the-way tomb, or would he like it back among the living where it could be used to smite a few more enemies?