Hey Dougie, I had similar sounding problem... i.e - the brake calliper blowing a hole in the left pannier.
I discovered, embarrassingly, that I had fitted the rear brake calliper incorrectly. I had not put the front bolt, that secures the calliper to the swing-arm, through the calliper. The result of my error was that the rear brake calliper sat too far to the rear to avoid the inside corner of the pannier... i.e - big hole.
With the rear brake calliper located correctly, it will not hit pannier. The rear brake worked just fine, even though it was 'floating'.... & bashing a hole in my pannier. Cheers, John & Kate.