Just another case or the brilliant build quality im afraid. Mine have not come apart YET, but they creak like feck only a matter of time.
To remove the handles I think you have to take the door card off and they should be bolted to door card, not sure though.