Ah, this seems to be the critical bit: iptables /usr/sbin/iptables-legacy
So I tried adding:
RUN update-alternatives --set iptables /usr/sbin/iptables-legacy
At the top of the docker-compose.yml, but I don’ t fully know how this interacts with dockerize:
Any advice?
FROM balenalib/%%BALENA_ARCH%%-python as builder
RUN install_packages dnsmasq wireless-tools git rsync binutils busybox-static iptables
RUN update-alternatives --set iptables /usr/sbin/iptables-legacy
WORKDIR /usr/src/app
RUN curl https://api.github.com/repos/balena-io/wifi-connect/releases/latest -s \
| grep -hoP 'browser_download_url": "\K.*%%BALENA_ARCH%%\.tar\.gz' \
| xargs -n1 curl -Ls \
| tar -xvz -C /usr/src/app/
RUN pip install git+https://github.com/larsks/dockerize
RUN mkdir output
RUN dockerize -n --verbose -o /usr/src/app/output/ -u dnsmasq -a /var/run /var/run --filetools `which busybox` `which balena-idle` `which dnsmasq` /usr/src/app/wifi-connect `which iptables` `which bash` `which sh` /usr/sbin/iptables-legacy
WORKDIR /var/lib/misc
RUN touch dnsmasq.leases
FROM scratch
COPY --from=builder /usr/src/app/output/ ./
COPY --from=builder /usr/src/app/ui /usr/src/app/ui
COPY --from=builder /var/lib/misc/dnsmasq.leases /var/lib/misc/dnsmasq.leases
COPY start.sh ./
CMD ["/bin/busybox","sh", "start.sh"]Oops, forgot to say what the current stumbling block was! In the WiFi Connect container, this is what I get:
# iptables -nL
Fatal: can't open lock file /run/xtables.lock: No such file or directory
Thoughts?
What container does your frontend: app run in? If it’s a Debian balena image there isn’t much value in using the balenablocks version and you could get much greater control and tools for port forwarding by using the original. Including a working iptables.
Yeah, that’s a decent plan B, but I would much prefer to have these tasks decoupled from each other.
I mean run it as a separate container, but on the same base. The way Docker layers work means if you reuse the same base image in both images you don’t use any extra space. What is your base image for the front end?
The BalenaBlocks wifi-connect is a good Balena idea to reduce the image size, but many are wrapping wifi-connect in other tools, apps, or customising its use like you are doing here with IPTables. I for example, allow users of my App to set the SSID, and also stop/start it during certain processes. When used like this, the BalenaBlocks versions is quite limiting, what we really need is to be able to have it in a proper container.
We can do that by reusing the base image from our apps. If our app runs in a balena Debian image, and we make a second container with the same base, the Docker layering system means we don’t actually sacrifice space. It shares all the initial layers with the other running container, and only adds on top the apps/commands we run on top, making it about the same as the BalenaBlocks version.
Where this strategy falls down, is Wifi-Connect doesn’t currently support Alpine builds, meaning for a large part of the community they have to run two different base images. Which of course sucks for space saving/efficiency.
Long-term, the best solution is for Alpine support for wifi-connect: https://github.com/balena-io/wifi-connect/issues/185, and we are waiting patiently. Hopefully this appearance of the BalenaBlocks version isn’t going to slow down that process, as it’s not really an alternative.
In the meantime, if you are already using Debian based images for your other containers, you can reuse that base, get the same gains as with the BalenaBlocks version, but open up a bunch of possibilities for customising (now and later in case you want to make changes in the future), including the port forwarding you are trying to achieve.
1 Like
Hi Adam,
I wonder if a iptables --wait command would help with the lock you’re seeing. You may need something more sophisticated, like a script that runs in that container that waits for iptables to be “free” and then triggers your in-container iptables command.
John
I don’t think it’s an actual lock:
# iptables --wait -L
Fatal: can't open lock file /run/xtables.lock: No such file or directory
I can’t why would have been keeping that lock for hours like that, anyway.
Thanks,
A.
In that case, that’s exactly what I will do, thank you for taking the time to explain.
I was really hoping that I’d be in a position to contribute a PR back so we could have this work for everyone using the stock container, but I’ll just fork it and go my own way, at least for now.
How do you communicate between your application container(s) and the wifi-connect container?
So this isn’t telling me that my images both take up that much space @maggie0002?
[Info] ┌──────────────┬────────────┬────────────┬─────────────────────┐
[Info] │ Service │ Image Size │ Delta Size │ Build Time │
[Info] ├──────────────┼────────────┼────────────┼─────────────────────┤
[Info] │ wifi-connect │ 118.97 MB │ 115.76 MB │ 1 minute, 7 seconds │
[Info] ├──────────────┼────────────┼────────────┼─────────────────────┤
[Info] │ frontend │ 115.30 MB │ 0 bytes │ 4 seconds │
[Info] └──────────────┴────────────┴────────────┴─────────────────────┘For communication., this may help: What firewall/port-blocking options are there on BalenaOS when using host network mode?
Docker layers can be a tricky business to dig into, or get a grip of. Couldn’t find a good explainer I liked. In short, what you are seeing there is correct file sizes, for each of the images individually. So if you exported them and took them somewhere else, that is how big they would be. But as they are running together on the same host, they are not taking up that space twice. When you start a container from that image, the container adds a layer to the read-only image, and docker only records the changes made to that image in a layer. When you start a second container of the same image, it makes its own layer of recorded changes, but shares the read-only base image. Your size then becomes original image size + size of changes recorded in container 1 + size of changes recorded in container 2.
That is why Docker has a weird scenario, where using one big container, and one small container, is actually less efficient than using the big container twice (space wise).
You can also share layers across images. If you do:
docker pull balenalib/orange-pi-zero-alpine
then do:
docker pull balenalib/orange-pi-zero-alpine-python
You will see a bunch of lines like:
b7d3d85a9f8d: Already exists
That is the layers that it shares with the last pull, as they are the same, it doesn’t pull them again, it just reuses the read only layers. It will be followed by things like:
13b1cc9a9ec1: Pull complete
Which is the python layers being pulled down to place on top.
So if you are tempted to use a Balena Alpine base image, which is around 60mbs, and then for another app use the official Alpine images, which are 5mb, you could end up adding 5mb extra to your devices that could be avoided by reusing the Balena Alpine image twice. Of course a better solution would be to use the 5mb Alpine image twice, but that has its own issues.
I am talking Docker here. Balena-engine which is based on Docker does most of the same things, although it also adds in a Delta system, which is pulling content from within layers.
Thanks for the extra explanation.
In any case, the iptables command from within a proper Debian image shows the full tables, as we hoped. So now I just need to remember shell script syntax to tweak the start.sh to my liking and I should be good to go. 
1 Like
Sorry to dredge up an old topic, but I came across this exact issue and read through the replies in this thread. I am hoping that two years later there is a workaround for this?
My app also listens on port 80. I was asked to allow for a Wifi Captive Portal to configure Wifi.
Easy enough, set it up, Captive Portal doesnt work, checked logs, cant start HTTP server on port 80.
Ohhh, yeah, cant have two things on the same port.
Any ideas?