Checkout only for usergroup xyz

Is it somehow possible to only allow a checkout if the customer is in a certain usergroup?

You can only restrict Guest users without customization. For other roles, you need customization.

There is an Access Control List permission for Public store. Enable shopping cart.  That may work for you.

You can disable the Public store. Enable shopping cart  for all the roles except XYZ. This will restrict the user with other roles from adding any products to the cart so the checkout process can not be initiated.
If you want to allow all users to add items to the cart but restrict the checkout flow only for the user with roles XYZ then you would need to make small customization to the checkout controller where you need to check the role and access or deny them accordingly, as the feature is not available out of the box.