I think the inference with Joric is that he's possibly inherited the prophetic psychic visions that Jarl Idgrod is known for, and they're not doing his mental health any good. One thing he says ambiently has always caught my ear (paraphrasing as I don't remember it exactly): "Sometimes I get lost. I don't know where I go, but it's not here".
I could be reading too much into that throwaway line, but the wording of it and the way he says it almost makes it sound like he's randomly finding himself in places that aren't even in Mundus - almost like he's being transported, if not physically then mentally, to Oblivion, or somewhere else. If that's what he means, then I could see how that might mess with a 12 year old's brain. It's possible that this is how the visions manifest in their family - that it feels like you've suddenly been transported somewhere else, like the Dreamstride almost. And given some of the things that we've seen in Oblivion... erm... yeah, not good for a 12 year old. Even if he's just being 'transported' to places around Tamriel, there's some stuff going on there that isn't necessarily healthy for a 12 year old to see either.
It does very much depend on what he's been seeing, as to how 'mad' he could potentially be. As we know, there's another child in Skyrim with prophetic dreams, and she seems perfectly well-adjusted. Sissel, in Rorikstead, claims to have had dreams about "a good dragon. He was old and gray, but he wasn't scary". She can potentially say this even before the Dragonborn has met Parthurnax, an old, grey, good dragon who isn't scary, and whom nobody but the Greybeards should even know exists. And despite this, and being abused by her father and sister, she seems pretty happy. But having dreams about friendly dragons, and being transported to places that are "not here" and seeing Azura-knows-what, are potentially two very different things.
