Sure, they take some of the XP away per fight, because every time they damage an enemy that's one less time you've used a skill. That said, however, there are more than enough dungeons, tombs, caves, shrines and other assorted locations with critters for you to swipe at, that I don't think it...