Logspout is not getting logs from other containers.

Product support balenaEngine
1

I created a sample project for this study.

docker-compose.yml

version: '2'
services:
  frontend:
    build: ./frontend
    depends_on:
      - logspout
  logspout:
    build: ./logspout
    ports:
      - "80:80"
    labels:
      io.balena.features.balena-socket: 1

frontend/index.js

...
setInterval(function() {console.log("WHY!!")}, 1000)

↓ balena console

image
image870×407 43.1 KB

↓ curl logspout logs

image
image1033×161 53.5 KB

The frontend service logs are not output on logspout. Instead, the resin_supervisor logs is output.

Of course, if it is not balena, there is no problem.

docker-compose -f docker-compose.mac.yml up

image

image

How can I get logs from other containers? (even if I don’t use logspout?)
Can I get logs of other containers from docker.sock?

1 Like
2

Hi Kazu,

Here is an example on how you can use the supervisor API to fetch logs and store them either to a file, or use a cloud service to keep logs: https://github.com/balena-io-playground/device-cloud-logging. Here is an example using Datadog: https://github.com/balena-io-playground/balena-datadog. Let me know if those examples help, and if we can help further. Thanks!

4

Hi sradevski,

The device-cloud-logging sample solved my problem. Thank you.

However, I thought that balena.sock could get logs of other containers, isn’t it?
I tried to get logs in Logspout, but it was the same as balena-engine.sock (only resin-supervisor logs).

11

Hi,

I tried the following:
I deployed your application, current master (https://github.com/kazuph/balena-logspout-sample/commit/14a5392206635165dd31f7e1a112aaee4e0a5c26) to a RPI3. After sshing into the logspout container, I ran the following in the console:

/ > env | grep DOCKER_HOST
DOCKER_HOST=unix:///var/run/balena-engine.sock
/ > apk add curl
...
/ > curl --unix-socket /var/run/balena-engine.sock http:/docker/containers/a17330a55c861cbdc372b872a4e4413fed682fe6d908dbe4df945b0
98a23edcc/logs?stdout=true

This curl command printed:

....
WHY!!
WHY!!
WHY!!
WHY!!
WHY!!

(a17330a55c861cbdc372b872a4e4413fed682fe6d908dbe4df945b0
98a23edcc) is the ID of the frontend container on that device.

So I would guess that there is something wrong in logspout somewhere, since the balena socket would be delivering the logs.

I hope this helps.

13

For anyone who finds this, I believe I’ve figured out why logspout isn’t gathering logs from the other containers: Logspout is programmed to ignore containers that have a TTY enabled (see discussion: Missing Logspout logs for containers running with TTY enabled · Issue #231 · gliderlabs/logspout · GitHub).

Apparently all Balena containers have TTY enabled (I don’t know enough to understand why or what that does). Here’s a Balena Supervisor issue that seems to support this theory.

Logspout added the ability to include TTY-enabled containers in version 3.2; unfortunately, emdem’s Raspberry Pi-supporting fork (which I think is popular around here) is based on 3.1.

There are 2 solutions:

  1. Use logspout >= 3.2 and set the ALLOW_TTY environment variable (for the logspout container) to true

OR

  1. set tty: false in docker-compose.yml for any services you want logspout to collect logs for. WARNING: I do not know what effect this has on the container, Balena functionality, etc.