Using ResinOS 2.2.0+rev1, supervisor 6.1.2, on an Intel Nuc, I’m unable to do various tasks over the network, such as using apt-get (in the final layer, works in build layers) and communicate with a Docker daemon.
When running apt-get update, the output is:
Could not resolve host: deb.debian.org
When running docker pull {any image from dockerhub}:
Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on [::1]:53: read udp [::1]:60959->[::1]:53: read: connection refused
The exact commands are able to run on a different machine on which I’ve installed Resin OS 2.0.4+rev1 (prod) with supervisor version 4.2.2.
The original resolv.conf looks at nameserver 172.17.0.1.
On my working device, the /etc/resolv.conf file reads:
#we use dnsmasq at 127.0.0.2 so that user containers can run their own dns cache and forwarder and not conflict with dnsmasq on the host nameserver 127.0.0.2
By putting in the start script “echo “8.8.8.8” > /etc/resolv.conf”, I was able to get these commands working again. This is a common fix I’ve seen, pointing to the Google DNS server.
Why is the default nameserver 172.17.0.1 unable to connect to any of these services? Is this a bug in the ResinOS image?
What do you mean by “communicate with a Docker daemon”? Do you run Docker inside your container – because the host Docker is not accessible by your container.
Regarding build steps, just to clarify, what you refer as “build layers” execute on our builders and the “final” layer upon your container startup on the device.
Do you run a dev resinOS version, or prod? If on dev, have you tried logging into the host and see if the network works?
I’m using a docker-in-docker hack to run docker in the resin container, to then run multiple apps inside of that container. It was this docker daemon that I was unable to connect to.
So it seems that the device itself cannot connect to anything but the Resin VPN using hostname 127.17.0.1. The builders are obviously fine doing apt-get commands, but the final image running on the device is unable to do any of these tasks.
This same issue is reproduced on a Raspberry Pi 3 using the ResinOS 2.2.0+rev1 prod, supervisor 6.1.2.
I’m unable to replace /etc/resolv.conf with “8.8.8.8” because the Resin repo is rejecting any git push resin master, even when using --force. I’m re-provisioning the device with a copy of the same OS version to see if I can get around the git issue. If not, I’ll try older versions of ResinOS.
Hi, can you give a bit more detail on what the git push is rejecting? What change are you trying to push exactly? Is there a repo that we could look at what you are trying to push?
Does the device function properly, if you push a simpler project such as idling, just to confirm that the device works fine on the network by itself (decoupled from your application)?
Not sure what or why the git push was rejecting, but a new install of ResinOS on the device fixed that issue. We’ll see if it starts again when I put my production application on the device.
I got the Ras Pi to run a fresh Dockerfile using Ubuntu and the /bin/bash entrypoint, and am able to ping google.com and run apt-get.
My theory is that the docker install I’m doing in this container is battling the default DNS lookup on the device. From googling, it looks like 127.17.0.1 is used by Docker, so why does the Resin image use this as the default DNS lookup? Is there a reason it switched from the old 127.0.0.2? Will changing it to 8.8.8.8 affect anything in Resin or on my end?
It appears that this may be caused by the change to DNSMasq or possibly the update to docker 17.03, and that we haven’t tested dind since this was added.
We run host networking to containers by default, so the wiping iptable rules also has the nasty knock-on effect which stops the supervisor from being able to request update images (and probably more).