Well, if we're discussing goats in Skyrim, I have an excuse to link this comic:
http://browse.deviantart.com/art/Skyrim-Gleda-the-Goatborn-312635852
I think the goat that follows the little boy in Dragonsreach is adorable.
I used to just ignore the goats in Skyrim, but then I got Hearthfire. I like building all the house furniture myself, and goat horns are ALWAYS one of the component I can't find enough of. The entire house will be finished except for the damn lanterns and chandeliers, and the general goods merchants don't have a high rate of having goat horns in stock. So now whenever I see a goat I'm all: "DIE! AND DROP HORNS!"
One time I saw one just outside of Windhelm, killed it with an arrow, walked over... and saw the bell around it's neck. I hurried off pretty quickly. I don't think anyone saw me do it, somehow.
And did you know the people of Falkreath love their goat as much as the people of Riverwood love their chickens? I once watched a friend go: "Cool! I'll get a new pelt!" then sat there laughing as the entire town took up arms to avenge their goat. He didn't try that again.