Can I let only some people use the VPN, but everyone use the office network?

Yes. This is a separate rule from the one for the office network. On the gateway's card, tick Only allow access from Entra groups and choose the groups. Only members of those groups can connect through that gateway. Everyone else can still use the office network as normal.

Both rules apply. If the client already limits network access to certain groups, someone using the VPN has to pass that rule and this one. So adding a VPN gateway never lets more people onto the network.

With the box unticked, which is the default, the gateway adds no rule of its own. It doesn't block everybody. Group membership comes from Entra ID and is refreshed regularly, so a change applies within minutes, not at the next certificate renewal.

The rule is for people. A device certificate belongs to a machine, which isn't in any of these groups, so it's logged but not checked against them. Use user certificates for people connecting to a VPN.