First, a 20% deposit is ridiculous on a small job like that. 50% should have been the minimum deposit, 100% would have been better.
Second, as soon as you knew there was a problem you should have called. Face to face isn't necessary, prompt attention is. A simple explanation that banner tape is stronger than stitching would have taken care of that part of the problem. Any other problems he had could have been quickly, and politely, handled while he was still in "deal with this banner problem" mode. You getting in the car, driving 40 minutes both ways, just to save a 3x3 banner order is ridiculous.
Now....my biggest problem with this whole thing is TELLING THE CUSTOMER TO DEPOSIT THE CHECK FOR YOU. What the heck is that? If a vendor told me to do that I'd laugh in their face.
Come on man, sack up, call the guy, and take care of the problem.