Can I restrict a network to one group?

Yes. Point a client at an Entra group and only members of it may join. Membership is checked against Microsoft every ten minutes and applied on every connection, so adding somebody to a group lets them on within about fifteen minutes — and removing a group from the list takes effect on the very next connection, even if Microsoft is unreachable at the time.

It needs the optional GroupMember.Read.All permission — without it Passbeam can tell that an account is enabled but not whether that person is supposed to be on this network, and it says so rather than guessing.

If Microsoft cannot be reached, the last answer is kept: up to 8 hours to let somebody on, up to 7 days to keep somebody off. Past that a connection is allowed rather than taking the network offline during somebody else's outage, and every one of those is in the authentication log with the reason.