How heavy is the fan you're thinking of? A relatively lightweight fan should not come anywhere near overtaxing a wall--and especailly an exterior wall. It might be good, particularly with an interior wall, to attempt to attach it so that it is supported by a stud rather than just the thin wood paneling.
I haven't moutned a fan, but I did mount towel bars to various walls of my motorhome and it's been just fine. I used normal plastic hollow wall anchors, and added a little thin plywood (1/8" or so) base plate underneath the wall anchors to help spread the load a bit--mostly to ensure it didn't just puncture the paneling due to a point load.
