Limit port access to container running network_mode:host

Is it possible to limit access to a port that a container has opened when the container is set with network_mode: host?
The container needs to be set as network_mode:host because it needs to broadcast some data over UDP on the local network. But it will also open a port, lets say 5000 and we want only other containers that are in the default bridge network to be able to access this port on this container. We do not want any other computer on the local network to be able to connect to port 5000 on the unit.

I have tried in the docker-compose.yml file to set ports: - “127.0.0.1:5000:5000/tcp” but it does not seem to help, I can still access the port from another computer on the network.

Can anyone help me with this?