GPSD service fails to start

I’m trying to create a docker service that will pipe data from gpsd to a socket. However i’m getting the following error.

systemctl command not found

version: '2.1'
services:
    gpsd:
        build:
            context: ./gpsd
        privileged: true
        expose:
          - "2947"
        ports:
            - "2947:2947"
        labels:
            io.balena.features.kernel-modules: '1'
FROM balenalib/%%BALENA_MACHINE_NAME%%-debian-python:2.7-build

WORKDIR /usr/src/app

RUN install_packages gpsd python-gps 

COPY Dockerbin/gpsd /etc/default/gpsd

COPY . ./

ENV INITSYSTEM on
ENV UDEV 1

CMD ["bash", "start.sh"]
#!/usr/bin/env bash

systemctl enable gpsd

gpsd /dev/ttyACM0 -F /var/run/gpsd.sock

sleep infinity
START_DAEMON="true"
GPSD_OPTIONS="/dev/ttyACM0"
DEVICES=""
USBAUTO="true"
GPSD_SOCKET="/var/run/gpsd.sock"

When I include systemd in the install packages, the error goes away. However I get connection refused when trying to connect to the port remotely. Also when I ssh into the container i’m able to see outputs for gpsd -V and /dev/ttyACM0

Hi @seesoe , the first thing that comes to mind is that you are using the balenalib base images and trying to enable a systemd service. The set of balenalib images are not 1-to-1 compatible with resin/ base images, one of the more significant changes is that systemd is no longer included in the base image. You can see a list of the major changes here: https://www.balena.io/docs/reference/base-images/base-images/#major-changes .

Installing systemd in the container is not as simple as just doing an apt-get systemd unfortunately, it needs a few additional changes, you can follow the docs here: https://www.balena.io/docs/reference/base-images/base-images/#installing-your-own-initsystem and this is an example to work from https://github.com/balena-io-playground/balenalib-systemd-example

@shaunmulligan, thank you for the info, makes sense, didn’t think it would be as easy as apt-get systemd. I setup the dockerfile like the examples you suggested. I ran into 2 issues. Both were fixed with the following 2 lines being added.

RUN ["chmod", "+x", "/usr/bin/entry.sh"]
RUN ["chmod", "644", "/etc/systemd/system/balena.service"]

Then when the service started I hang at this, not sure what it means or to do from here with it.

[Logs]    [2019-5-16 22:46:37] Started service 'gpsd sha256:49d2dceed38a035cbd8d7122b4e2150209526325a153daa1558d067f4a5e4e2e'
[Logs]    [2019-5-16 22:46:37] [gpsd] Systemd init system enabled.
[Logs]    [2019-5-16 22:46:37] [gpsd] systemd 232 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
[Logs]    [2019-5-16 22:46:37] [gpsd] Created symlink /etc/systemd/system/systemd-udevd.service → /dev/null.
[Logs]    [2019-5-16 22:46:37] [gpsd] Detected virtualization docker.
[Logs]    [2019-5-16 22:46:37] [gpsd] Detected architecture arm.
[Logs]    [2019-5-16 22:46:37] [gpsd] Set hostname to <6d054ef3ddb8>.
[Logs]    [2019-5-16 22:46:38] [gpsd] kmod-static-nodes.service: Cannot add dependency job, ignoring: Unit kmod-static-nodes.service is masked.
[Logs]    [2019-5-16 22:46:37] [gpsd] Failed to install release agent, ignoring: No such file or directory
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd.service: Cannot add dependency job, ignoring: Unit systemd-udevd.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd.service: Cannot add dependency job, ignoring: Unit systemd-udevd.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] sys-fs-fuse-connections.mount: Cannot add dependency job, ignoring: Unit sys-fs-fuse-connections.mount is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-remount-fs.service: Cannot add dependency job, ignoring: Unit systemd-remount-fs.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] sys-kernel-config.mount: Cannot add dependency job, ignoring: Unit sys-kernel-config.mount is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] dev-hugepages.mount: Cannot add dependency job, ignoring: Unit dev-hugepages.mount is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] getty.target: Cannot add dependency job, ignoring: Unit getty.target is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-logind.service: Cannot add dependency job, ignoring: Unit systemd-logind.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] display-manager.service: Cannot add dependency job, ignoring: Unit display-manager.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd-kernel.socket: Socket service systemd-udevd.service not loaded, refusing.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd-control.socket: Socket service systemd-udevd.service not loaded, refusing.

I read in another post that udev could be conflicting with the host, I removed ENV INITSYSTEM on and I believe the service is started now. But still can’t connect to the service at ip:2947.

[Logs]    [2019-5-16 23:00:21] Started service 'gpsd sha256:04fe139d7416c52d01a9d1bedf95bd79aee1bc1401226cdadb1e6cec19f391da'
[Logs]    [2019-5-16 23:00:21] [gpsd] Synchronizing state of gpsd.service with SysV service script with /lib/systemd/systemd-sysv-install.
[Logs]    [2019-5-16 23:00:21] [gpsd] Executing: /lib/systemd/systemd-sysv-install enable gpsd
root@96d95b5:~# lsof -i -P -n | grep 2947
balena-en 20502     root    6u  IPv6 126216      0t0  TCP *:2947 (LISTEN)

I created a simple node script that ran on the same service as gpsd and was able to communicate with the gpsd service with localhost:2947. I moved my code to a test service to see if I can still communicate with the service however the connection keeps getting refused, tried 0.0.0.0, and 127.0.0.1 with no luck.

Do I need to configure some sort of network param on the services to be able to communicate to the gpsd socket?

    test:
        build:
            context: ./test
        privileged: true
        expose:
          - "2947"
        depends_on:
            - gpsd

Hi @seesoe, when you have several services, each of them will have its own IP in a local network created by docker (or in this case, balenaEngine). This is described in this doc https://docs.docker.com/compose/networking/ though the balena supervisor implements a subset of those features (e.g. Links aren’t supported). But in your case, you should be able to talk with the gpsd service from the test service by hitting gpsd:2947 instead of localhost:2947. Note that you only need to have expose in the gpsd service, and you don’t need the ports definition unless you want to access the gpsd service from outside the device (in which case you can have ports without expose). Hope this helps!

@pcarranzav, thank you for the clarity, I tried that however I still get the following issue.

[Logs]    [2019-5-17 18:35:59] [test] socket connection refused
[Logs]    [2019-5-17 18:35:59] [test] Socket disconnected.
version: '2.1'
services:
    gpsd:
        build:
            context: ./gpsd
        privileged: true
        expose:
          - "2947"
        labels:
            io.balena.features.kernel-modules: '1'
    test:
        build:
            context: ./test
        privileged: true
        depends_on:
            - gpsd

@seesoe I’m not familiar with gpsd, but is it possible that it’s only listening on the loopback interface? Judging from these docs http://www.catb.org/gpsd/gpsd.html I think you might need to add a -G flag, since as far as the container knows, the other service is on an external interface

@pcarranzav, That did it! I don’t know how I missed that flag in the docs :confused:. Makes total sense though, gpsd does all it can to keep data on the machine unless -G It even says Socket disconnected.

Final solution
systemd

gpsd /dev/ttyACM0 -F /var/run/gpsd.sock -G

var listener = new gpsd.Listener({
    port: 2947,
    hostname: 'gpsd',
    logger: console,
    parse: true
});

Thanks Balena Team for the awesome help!

That’s great to hear, have a good weekend!

Hi folks,
I struggled with the same issue with GPSD.

Handing off the service to systemd isn’t best practice for a container. The running container process should always be pid 1. That way if the process crashes the engine can restart the container.
Using the method above i.e. “sleep infinity”, in the case where gpsd crashes your container won’t restart. GPSD let’s us start it in non-daemon mode which is super useful.
Also over permissive access can be dangerous.

If it helps here’s what I’ve been using. Admittedly, I’m not using a non-root user. It’s on the todo list.

Dockerfile

FROM balenalib/raspberrypi3-alpine
RUN apk add --no-cache gpsd gpsd-clients
CMD /usr/sbin/gpsd -N -n -G /dev/ttyAMA0

#Docker-compose

services:
gpsd:
hostname: gpsd
restart: always
build: gpsd
devices:
- /dev/ttyAMA0:/dev/ttyAMA0
ports:
- 2947:2947

Yeah, that makes sense, thanks for sharing your experience as well. It’s good to collect best practices!

As a sidenote, that the balenalib base images don’t come with systemd (or openrc in the case of alpine base images as above).