It's not that so many fancy cancel stamps are damaged, it is that these stamps were saved at all. If a stamps is cheap and has nothing special about it and is damaged (thinned, pin hole what ever) it will be tossed. The same stamp with an interesting cancel might be saved - regardless of condition. A 65 with nothing special about it that has a tear or thin is tossed and many millions were over the years. Put a nice Devil's Head or Masonic on it and it has some interest - even before fancy cancels become so popular and pricey.

As a collector and dealer in fancy cancels, I believe the condition of the stamp is important, but I will accept damaged stamps that have a nice fancy cancel. I'll pay less for them, but I won't reject them out of hand.