GPSD service fails to start

seesoe · May 16, 2019, 8:56am

I’m trying to create a docker service that will pipe data from gpsd to a socket. However i’m getting the following error.

systemctl command not found

version: '2.1'
services:
    gpsd:
        build:
            context: ./gpsd
        privileged: true
        expose:
          - "2947"
        ports:
            - "2947:2947"
        labels:
            io.balena.features.kernel-modules: '1'

FROM balenalib/%%BALENA_MACHINE_NAME%%-debian-python:2.7-build

WORKDIR /usr/src/app

RUN install_packages gpsd python-gps 

COPY Dockerbin/gpsd /etc/default/gpsd

COPY . ./

ENV INITSYSTEM on
ENV UDEV 1

CMD ["bash", "start.sh"]

#!/usr/bin/env bash

systemctl enable gpsd

gpsd /dev/ttyACM0 -F /var/run/gpsd.sock

sleep infinity

START_DAEMON="true"
GPSD_OPTIONS="/dev/ttyACM0"
DEVICES=""
USBAUTO="true"
GPSD_SOCKET="/var/run/gpsd.sock"

When I include systemd in the install packages, the error goes away. However I get connection refused when trying to connect to the port remotely. Also when I ssh into the container i’m able to see outputs for gpsd -V and /dev/ttyACM0

shaunmulligan · May 16, 2019, 12:00pm

Hi @seesoe , the first thing that comes to mind is that you are using the balenalib base images and trying to enable a systemd service. The set of balenalib images are not 1-to-1 compatible with resin/ base images, one of the more significant changes is that systemd is no longer included in the base image. You can see a list of the major changes here: https://www.balena.io/docs/reference/base-images/base-images/#major-changes .

Installing systemd in the container is not as simple as just doing an apt-get systemd unfortunately, it needs a few additional changes, you can follow the docs here: https://www.balena.io/docs/reference/base-images/base-images/#installing-your-own-initsystem and this is an example to work from https://github.com/balena-io-playground/balenalib-systemd-example

seesoe · May 17, 2019, 2:51am

@shaunmulligan, thank you for the info, makes sense, didn’t think it would be as easy as apt-get systemd. I setup the dockerfile like the examples you suggested. I ran into 2 issues. Both were fixed with the following 2 lines being added.

RUN ["chmod", "+x", "/usr/bin/entry.sh"]
RUN ["chmod", "644", "/etc/systemd/system/balena.service"]

Then when the service started I hang at this, not sure what it means or to do from here with it.

[Logs]    [2019-5-16 22:46:37] Started service 'gpsd sha256:49d2dceed38a035cbd8d7122b4e2150209526325a153daa1558d067f4a5e4e2e'
[Logs]    [2019-5-16 22:46:37] [gpsd] Systemd init system enabled.
[Logs]    [2019-5-16 22:46:37] [gpsd] systemd 232 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
[Logs]    [2019-5-16 22:46:37] [gpsd] Created symlink /etc/systemd/system/systemd-udevd.service → /dev/null.
[Logs]    [2019-5-16 22:46:37] [gpsd] Detected virtualization docker.
[Logs]    [2019-5-16 22:46:37] [gpsd] Detected architecture arm.
[Logs]    [2019-5-16 22:46:37] [gpsd] Set hostname to <6d054ef3ddb8>.
[Logs]    [2019-5-16 22:46:38] [gpsd] kmod-static-nodes.service: Cannot add dependency job, ignoring: Unit kmod-static-nodes.service is masked.
[Logs]    [2019-5-16 22:46:37] [gpsd] Failed to install release agent, ignoring: No such file or directory
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd.service: Cannot add dependency job, ignoring: Unit systemd-udevd.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd.service: Cannot add dependency job, ignoring: Unit systemd-udevd.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] sys-fs-fuse-connections.mount: Cannot add dependency job, ignoring: Unit sys-fs-fuse-connections.mount is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-remount-fs.service: Cannot add dependency job, ignoring: Unit systemd-remount-fs.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] sys-kernel-config.mount: Cannot add dependency job, ignoring: Unit sys-kernel-config.mount is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] dev-hugepages.mount: Cannot add dependency job, ignoring: Unit dev-hugepages.mount is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] getty.target: Cannot add dependency job, ignoring: Unit getty.target is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-logind.service: Cannot add dependency job, ignoring: Unit systemd-logind.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] display-manager.service: Cannot add dependency job, ignoring: Unit display-manager.service is masked.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd-kernel.socket: Socket service systemd-udevd.service not loaded, refusing.
[Logs]    [2019-5-16 22:46:38] [gpsd] systemd-udevd-control.socket: Socket service systemd-udevd.service not loaded, refusing.

seesoe · May 17, 2019, 3:05am

I read in another post that udev could be conflicting with the host, I removed ENV INITSYSTEM on and I believe the service is started now. But still can’t connect to the service at ip:2947.

[Logs]    [2019-5-16 23:00:21] Started service 'gpsd sha256:04fe139d7416c52d01a9d1bedf95bd79aee1bc1401226cdadb1e6cec19f391da'
[Logs]    [2019-5-16 23:00:21] [gpsd] Synchronizing state of gpsd.service with SysV service script with /lib/systemd/systemd-sysv-install.
[Logs]    [2019-5-16 23:00:21] [gpsd] Executing: /lib/systemd/systemd-sysv-install enable gpsd

root@96d95b5:~# lsof -i -P -n | grep 2947
balena-en 20502     root    6u  IPv6 126216      0t0  TCP *:2947 (LISTEN)

seesoe · May 17, 2019, 8:56am

I created a simple node script that ran on the same service as gpsd and was able to communicate with the gpsd service with localhost:2947. I moved my code to a test service to see if I can still communicate with the service however the connection keeps getting refused, tried 0.0.0.0, and 127.0.0.1 with no luck.

Do I need to configure some sort of network param on the services to be able to communicate to the gpsd socket?

    test:
        build:
            context: ./test
        privileged: true
        expose:
          - "2947"
        depends_on:
            - gpsd

pcarranzav · May 17, 2019, 10:18pm

Hi @seesoe, when you have several services, each of them will have its own IP in a local network created by docker (or in this case, balenaEngine). This is described in this doc https://docs.docker.com/compose/networking/ though the balena supervisor implements a subset of those features (e.g. Links aren’t supported). But in your case, you should be able to talk with the gpsd service from the test service by hitting gpsd:2947 instead of localhost:2947. Note that you only need to have expose in the gpsd service, and you don’t need the ports definition unless you want to access the gpsd service from outside the device (in which case you can have ports without expose). Hope this helps!

seesoe · May 17, 2019, 10:38pm

@pcarranzav, thank you for the clarity, I tried that however I still get the following issue.

[Logs]    [2019-5-17 18:35:59] [test] socket connection refused
[Logs]    [2019-5-17 18:35:59] [test] Socket disconnected.

version: '2.1'
services:
    gpsd:
        build:
            context: ./gpsd
        privileged: true
        expose:
          - "2947"
        labels:
            io.balena.features.kernel-modules: '1'
    test:
        build:
            context: ./test
        privileged: true
        depends_on:
            - gpsd

pcarranzav · May 17, 2019, 10:44pm

@seesoe I’m not familiar with gpsd, but is it possible that it’s only listening on the loopback interface? Judging from these docs http://www.catb.org/gpsd/gpsd.html I think you might need to add a -G flag, since as far as the container knows, the other service is on an external interface

seesoe · May 17, 2019, 11:27pm

@pcarranzav, That did it! I don’t know how I missed that flag in the docs . Makes total sense though, gpsd does all it can to keep data on the machine unless -G It even says Socket disconnected.

Final solution
systemd

gpsd /dev/ttyACM0 -F /var/run/gpsd.sock -G

var listener = new gpsd.Listener({
    port: 2947,
    hostname: 'gpsd',
    logger: console,
    parse: true
});

Thanks Balena Team for the awesome help!

pcarranzav · May 18, 2019, 12:34am

That’s great to hear, have a good weekend!

rhysjtevans · May 21, 2019, 8:50am

Hi folks,
I struggled with the same issue with GPSD.

Handing off the service to systemd isn’t best practice for a container. The running container process should always be pid 1. That way if the process crashes the engine can restart the container.
Using the method above i.e. “sleep infinity”, in the case where gpsd crashes your container won’t restart. GPSD let’s us start it in non-daemon mode which is super useful.
Also over permissive access can be dangerous.

If it helps here’s what I’ve been using. Admittedly, I’m not using a non-root user. It’s on the todo list.

Dockerfile

FROM balenalib/raspberrypi3-alpine
RUN apk add --no-cache gpsd gpsd-clients
CMD /usr/sbin/gpsd -N -n -G /dev/ttyAMA0

docker-compose

services:
gpsd:
hostname: gpsd
restart: always
build: gpsd
devices:
- /dev/ttyAMA0:/dev/ttyAMA0
ports:
- 2947:2947

imrehg · May 21, 2019, 6:15pm

Yeah, that makes sense, thanks for sharing your experience as well. It’s good to collect best practices!

As a sidenote, that the balenalib base images don’t come with systemd (or openrc in the case of alpine base images as above).

adgardo · July 31, 2020, 10:45am

Hi there,

It’s being some time since you posted. Hope you can help me.

I am trying to migrate my raspberry pi 4 project to a multicontainer project using BALENA.
I use a USB GPS that works like a charm in my current setup (home assistant installed into raspbian buster OS).I just installed gpsd package and a component in home assistant, with no further configurations, home assistant is able to pull data from the USB GPS sensor via GPSD.

Now, trying to set it up with Balena, I configured a GPSD service in a multicontainer project (with home assistant and MQTT containers too) from:

I tried several setups, ensured to run both projects privileged and ENV UDEV=1 as I noted from https://www.balena.io/docs/reference/base-images/base-images/#working-with-dynamically-plugged-devices

I get the following errors:

gpsd:ERROR: unable to connect to the DBUS system bus
gpsd:ERROR: can’t bind to IPv6 port gpsd, Cannot assign requested address

homeassistant 2020-07-31 10:18:24 ERROR (SyncWorker_4) [homeassistant.components.gpsd.sensor] Not able to connect to GPSD

Do you have a working dockerfile and docker-compose configuration that I could try?

Thank you in advance!

P.S. I tried as well what [rhysjtevans] wrote (dockerfile and docker compose) but it didnt work for me.

Adrián