If you are on rough ground and are boucing, you could be lifting off the seat momentarily causing the power to the seat belt to be interrupted. Most of the time you do not realize you have lifted off the seat. On early units the seat belt power had to go through the seat switch before reaching the seat belt. If this is your problem you can modify the seat harness or just purchase the latest harness where power is supplied independent of the seat swtich.
It is also possible you have a bad switch in the seat belt or seat switch.
