Driver's seatbelt mechanism rattles

I have an intermittent problem whereby the driver's side seatbelt mechanism will make a rattling noise when I go over bumps. Does anyone else have this problem or know a solution?
 
I have the same problem but on the passenger side, it rattles whilst driving over rough bits of road and when the music basey and is over 18+ vol..

I've had a look and tried sitting in the back whilst I got the mrs to drive and pressure seems to fix it.. But I can't find anyway of permanently solving this :S I would like to remove the plastic panel and put it back on to see if it makes a difference..

Have you managed to sort it?