# \[error\] Non-empty stderr stream from journalctl log fetching:

**URL:** <https://forums.balena.io/t/error-non-empty-stderr-stream-from-journalctl-log-fetching/270827>\
**Category:** Product support\
**Created:** [March 12, 2021, 5:51am UTC](https://forums.balena.io/t/error-non-empty-stderr-stream-from-journalctl-log-fetching/270827 "2021-03-12T05:51:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bradsquicciarini](https://avatars.discourse-cdn.com/v4/letter/b/df788c/32.png) [@bradsquicciarini](https://forums.balena.io/u/bradsquicciarini)\
**Post date:** [March 12, 2021, 5:51am UTC](https://forums.balena.io/t/error-non-empty-stderr-stream-from-journalctl-log-fetching/270827/1 "2021-03-12T05:51:17Z")

</div>

Recently started with balena using the most recent jetson-xavier-nx development image, and I’m having some issues. It’s hard to pin point the problem, but here are some symptoms:

# System is Sluggish

Overall the system is very sluggish to perform any actions on it. For example `balena push` or making any requests to the supervisor api., and sometimes outright fails.

## Supervisor Errors

I repeatedly get this error from the supervisor:

```auto
[error] Non-empty stderr stream from journalctl log fetching: Considering root directory '/run/log/journal'.
[error] Root directory /run/log/journal added.
[error] Considering directory '/run/log/journal/72df14572ac545eda3caf972ff8d6baa'.
[error] Directory /run/log/journal/72df14572ac545eda3caf972ff8d6baa added.
[error] Journal effective settings seal=no compress=no compress_threshold_bytes=8B
[error] File /run/log/journal/72df14572ac545eda3caf972ff8d6baa/system.journal added.
[error] File /run/log/journal/72df14572ac545eda3caf972ff8d6baa/system@a79d91c044364210896f3ba1b03e3b3a-000000000007bfaa-0005bd4e05aa067c.journal added.
[error] File /run/log/journal/72df14572ac545eda3caf972ff8d6baa/system@a79d91c044364210896f3ba1b03e3b3a-000000000007b02c-0005bd4e05a0129b.journal added.
[error] File /run/log/journal/72df14572ac545eda3caf972ff8d6baa/system@a79d91c044364210896f3ba1b03e3b3a-0000000000079abe-0005bd4e0597f396.journal added.
[error] File /run/log/journal/72df14572ac545eda3caf972ff8d6baa/system@a79d91c044364210896f3ba1b03e3b3a-00000000000785b0-0005bd4e03388dd4.journal added.
[error] File /run/log/journal/72df14572ac545eda3caf972ff8d6baa/system@a79d91c044364210896f3ba1b03e3b3a-0000000000076fbd-0005bd4e031fb6d9.journal added.

```

There are probably 100+ repeated messages after a minute. In addition, after a while this message will also appear:

```auto
[error] Insufficient watch descriptors available. Reverting to -n.

```

When this happens, trying to grab the logs will return this error

```auto
$ balena logs da55d9637fae -f 

error from daemon in stream: Error grabbing logs: error getting journald fd: Too many open files

```

In addition, this debug message will be outputed about 100 times in a row:

```auto
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service
[debug] Spawning journald with: chroot /mnt/root journalctl -a --follow -o json _SYSTEMD_UNIT=balena.service

```

And this one:

```auto
[error] Attempt to move to uninitialized object: 126232
[error] Failed to iterate through journal: Bad message
[error] Directory /run/log/journal/72df14572ac545eda3caf972ff8d6baa removed.
[error] Root directory /run/log/journal removed.
[error] mmap cache statistics: 28676 hit, 28 miss
[error]

```

Any ideas if these are related?

## Long Response Time from Supervisor Api

Additionally, any calls to the supervisor api will take around 30 seconds to complete. Is this normal?

For example:

`http://192.168.50.10:48484/ping`

Took 30 seconds to respond.

## Running in Local Mode `balena push` sometimes fails

```auto
[Debug] Using build source directory: . 
[Debug] Pushing to local device: fed25f4.local
[Debug] Checking we can access device
[Debug] Sending request to http://192.168.50.10:48484/ping
[Debug] Checking device supervisor version: 12.3.0
[Info] Starting build on device 192.168.50.10
[Debug] Loading project...
[Debug] Resolving project...
[Debug] docker-compose.yaml file found at "."
[Debug] Creating project...
[Debug] Tarring all non-ignored files...
[Debug] Sending request to http://192.168.50.10:48484/v2/local/device-info
ECONNRESET: read ECONNRESET

Error: read ECONNRESET
    at TCP.onStreamRead (internal/stream_base_commons.js:205:27)

For further help or support, visit:
https://www.balena.io/docs/reference/balena-cli/#support-faq-and-troubleshooting

```

## Conslusion(?)

Overall, it seems the system is unstable. Any suggestions would be greatly appreciated.

Thanks,

---

<div class="post-metadata">

**Author:** ![anujdeshpande](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/anujdeshpande/32/3155_2.png) [@anujdeshpande](https://forums.balena.io/u/anujdeshpande)\
**Post date:** [March 18, 2021, 8:52am UTC](https://forums.balena.io/t/error-non-empty-stderr-stream-from-journalctl-log-fetching/270827/3 "2021-03-18T08:52:29Z")

</div>

Hi

The ping numbers, etc feel like a lot! Can you help us recreate this -

- what OS and supervisor version are you using?
- what does your network connection look like? Is the sluggishness even there when you are on local mode?
- If you have a minimum docker file that can help us recreate this, it would greatly help in figuring out what’s happening

---

<div class="post-metadata">

**Author:** ![20k-ultra](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/20k-ultra/32/3435_2.png) [@20k-ultra](https://forums.balena.io/u/20k-ultra)\
**Post date:** [March 22, 2021, 5:58pm UTC](https://forums.balena.io/t/error-non-empty-stderr-stream-from-journalctl-log-fetching/270827/7 "2021-03-22T17:58:41Z")

</div>

We fix this issue in Supervisor `v12.4.3` see [Journalctl process infinite spawn · Issue #1591 · balena-io/balena-supervisor · GitHub](https://github.com/balena-io/balena-supervisor/issues/1591).

---

<div class="post-metadata">

**Author:** ![20k-ultra](https://sea2.discourse-cdn.com/balena/user_avatar/forums.balena.io/20k-ultra/32/3435_2.png) [@20k-ultra](https://forums.balena.io/u/20k-ultra)\
**Post date:** [March 22, 2021, 5:59pm UTC](https://forums.balena.io/t/error-non-empty-stderr-stream-from-journalctl-log-fetching/270827/8 "2021-03-22T17:59:53Z")

</div>

You should be able to upgrade to this version or later via our self-serve upgrades: [Self-service supervisor upgrades - Balena Documentation](https://www.balena.io/docs/reference/supervisor/supervisor-upgrades/)
