I cannot recommend fencing and swordplay in general enough for the purpose of improving the body. I practiced a light form of kenjutsu for a few years off-and-on amidst other martial arts, and I noticed it strongly when I stopped.

Another friend of mine studied fencing intensely for a few years and came out of it with a physique something like Bruce Lee's, with no additional effort.