In Display settings, under Manager views we have added an option to allow/prevent manager level users (or those HR who are using manager restrictions) from uploading and/or deleting employee documents

You can access this page by going to 

Admin > Company > Settings > Display settings > Manager Views

To stop managers from being able to to upload and/ or delete employee documents, please untick the relevant boxes. 

This will then mean that managers will no longer see the buttons or icons to upload and/or delete documents within the document folders for their employees.