@codewithcheese Thanks for the pointers - exactly what I was looking for.
So, what happens when I start the service is that it starts up fine and runs for a few minutes happily. Then it realizes there is a network config change that it needs to make:
Sep 07 08:04:49 c6327f8 78f4fd5e9a25[785]: [debug] Replacing container for service dnscache because of config changes:
Sep 07 08:04:49 c6327f8 78f4fd5e9a25[785]: [debug] Network changes detected
After a while we get to this (removed some duplicate lines from resin-supervisor):
Sep 07 08:05:02 c6327f8 balenad[785]: time="2020-09-07T08:05:02.158458734Z" level=info msg="shim reaped" id=6e654f723795a56e9adc5da181ed5fe6b0d84c099db5e0c785d9494d4874f78f
Sep 07 08:05:02 c6327f8 balenad[785]: time="2020-09-07T08:05:02.164912034Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Sep 07 08:05:04 c6327f8 78f4fd5e9a25[785]: [event] Event: Service exit {"service":{"appId":1725022,"serviceId":681292,"serviceName":"dnscache","releaseId":1521939}}
Sep 07 08:05:04 c6327f8 78f4fd5e9a25[785]: [event] Event: Service stop {"service":{"appId":1725022,"serviceId":681292,"serviceName":"dnscache","releaseId":1521939}}
The first two I’m not sure I understand, but the other two make sense.
Then:
Sep 07 08:05:05 c6327f8 78f4fd5e9a25[785]: [event] Event: Service install {"service":{"appId":1725022,"serviceId":681292,"serviceName":"dnscache","releaseId":1521939}}
Sep 07 08:05:06 c6327f8 78f4fd5e9a25[785]: [error] Scheduling another update attempt in 900000ms due to failure: Error: Failed to apply state transition steps. (HTTP code 400) unexpected - container sharing network namespace with another container or host cannot be connected to any other network Steps:["start"]
… and a callstack.
So I think this identifies my problem, although I’m not sure how to handle it.
I’m trying to create two services, one is the dns cache we’ve been talking about, and the other is a local authoratitive dns server (tinydns). Usually these are often installed together, with tinydns listening on 127.0.0.1 and the cache listening to the outside network and relaying any relevant requests to it.
So I’m uncertain how I can set that up here, with two separate containers/services. The cache can certainly sit there, listening for requests and fulfilling them. But I am uncertain how it can communicate with the tinydns container, as it works only with ip addresses, and I don’t know the address of the other service.
I’ve tried to deal with that in this way (just showing relevant lines from the docker_compose.yml):
tinydns:
expose:
- 53/udp
networks:
internal:
ipv4_address: 10.0.0.2
dnscache:
network_mode: host
ports:
- "53:53/udp"
networks:
internal:
ipv4_address: 10.0.0.3
networks:
internal:
driver: bridge
ipam:
config:
- subnet: 10.0.0.0/24
Clearly, that’s a problem. The cache can’t be part of the bridged network and also part of the host network. I can sort of understand that, but then how can I reach the tinydns service, by IP address, and also talk to the outside network? Have I made this over-complicated? I feel I’m missing what should be an obvious bit of understanding about networking between services/containers. But I’ve only seen them referred to by service_name:port, that I can recall, and that won’t work in the dns cache config files.
Any advice most welcome!