API goes in econnreset

openBalena
1

Hi,

suddenly all my balena-cli commands stopped working and are giving me this error:

balena login
CONNRESET: request to https://api.domain.org/login_ failed, reason: socket hang up

balena device uuid --debug
[debug] new argv=[/usr/local/lib/balena-cli/bin/node,/usr/local/lib/balena-cli/bin/run,device,uuid] length=4
ECONNRESET: request to https://api.domain.org/v5/device?$filter=startswith(uuid,'uuid')&$expand=belongs_to__application($select=app_name) failed, reason: socket hang up

FetchError: request to https://api.domain.org/v5/device?$filter=startswith(uuid,'uuid')&$expand=belongs_to__application($select=app_name) failed, reason: socket hang up
    at ClientRequest.<anonymous> (/usr/local/lib/balena-cli/node_modules/node-fetch/index.js:133:11)
    at ClientRequest.emit (events.js:203:15)
    at ClientRequest.EventEmitter.emit (domain.js:448:20)
    at TLSSocket.socketOnEnd (_http_client.js:426:9)
    at TLSSocket.emit (events.js:203:15)
    at TLSSocket.EventEmitter.emit (domain.js:448:20)
    at endReadableNT (_stream_readable.js:1129:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

Any idea of what might cause it?

(I am an openbalena user, and I am able to connect through ssh to the server)

Thanks
Matteo

4

Some more logs that could be useful(?)

docker logs openbalena_haproxy-container

Building certificate from environment variables...
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
[NOTICE] 275/093432 (16) : New worker #1 (18) forked
[WARNING] 275/093432 (18) : Server backend_api/balena_api_1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 275/093432 (18) : backend 'backend_api' has no server available!
[WARNING] 275/093450 (18) : Server backend_api/balena_api_1 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
Building certificate from environment variables...
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
[NOTICE] 342/160633 (15) : New worker #1 (17) forked
[WARNING] 342/160633 (17) : Server backend_api/balena_api_1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 342/160633 (17) : backend 'backend_api' has no server available!
[WARNING] 342/160634 (17) : Server backend_registry/balena_registry_1 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 342/160634 (17) : backend 'backend_registry' has no server available!
[WARNING] 342/160635 (17) : Server vpn-tunnel/balena_vpn is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 342/160635 (17) : proxy 'vpn-tunnel' has no server available!
[WARNING] 342/160638 (17) : Server backend_registry/balena_registry_1 is UP, reason: Layer4 check passed, check duration: 3ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 342/160641 (17) : Server vpn-tunnel/balena_vpn is UP, reason: Layer4 check passed, check duration: 1ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.
[WARNING] 342/160649 (17) : Server backend_api/balena_api_1 is UP, reason: Layer4 check passed, check duration: 0ms. 1 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.



docker logs open-balena-api

Systemd init system enabled.
Systemd init system enabled.


docker logs open-balena-db
2020-10-16 11:32:55.242 UTC [18324] LOG:  could not receive data from client: Connection reset by peer
2020-10-16 11:32:55.247 UTC [18324] LOG:  incomplete startup packet
2020-10-16 11:35:38.414 UTC [18328] LOG:  could not receive data from client: Connection reset by peer
2020-10-16 11:35:38.414 UTC [18328] LOG:  incomplete startup packet
5

Everything restarted after a docker restart open-balena-api

moments of fear :sweat_smile: :joy:

6

Hi,
Can you try whether visiting https://api.domain.org/ping and https://api.domain.org/device-types/v1/ gives you back a response?

Kind regards,
Thodoris

8

Yes, now I get a response, thanks! I still haven’t figured out why the api container stopped working…

9

Hi,
Thanks for letting us know that a restart fixed the issue for you.
This can prove useful to other users as well.

I’ve opened the following an issue for future reference and which you can use to track discussions and progress around this issue:

Kind regards,
Thodoris

1 Like
11

while trying to configure os file
ECONNRESET: request to https://api.mydomain.com/download-config failed, reason: socket hang up

FetchError: request to https://api.mydomain.com/download-config failed, reason: socket hang up
at ClientRequest. (/snapshot/balena-cli/node_modules/node-fetch/lib/index.js:1505:11)
at ClientRequest.emit (node:events:539:35)
at ClientRequest.emit (node:domain:475:12)
at TLSSocket.socketOnEnd (node:_http_client:478:9)
at TLSSocket.emit (node:events:539:35)
at TLSSocket.emit (node:domain:475:12)
at endReadableNT (node:internal/streams/readable:1345:12)
at processTicksAndRejections (node:internal/process/task_queues:83:21)