I would suggest modifying the CustomerRoleController.cs in admin and change every one of these lines:
if (!_permissionService.Authorize(StandardPermissionProvider.ManageCustomers))
to:
if (!_permissionService.Authorize(StandardPermissionProvider.ManageAcl))
Then you can use the Admin area. Manage ACL permission from within the Access control list area to limit user access.  I think the permissions to manage the ACL go hand in hand with setting roles. Allowing anyone with Manage Customers rights to change users roles and therefore change their access privileges does not make sense.
Additionally, the CustomerController.cs would have to be modified to prevent roles from being added or changed when a customer record is added or edited, also using "if (!_permissionService.Authorize(StandardPermissionProvider.ManageAcl))". Allowing only the guest role here would make sense.  If someone needs roles added they would have to have a user with ACL rights do it.