If you have full access to your company settings you can remove your co-workers' access to All Ears. Simply click on your name in the lower left corner and then select Users. Click on the waste bin to remove access.
From here you may also invite new users by clicking on Invite co-worker.
If you want to know more about the access levels, click here.