We found out this was a dns error using nslookup
. Everything returned with the 1.2.3.4
address.
We used configizer to set DNSSERVERS="8.8.8.8"
on the device’s config.json file.
After a reboot the device was again online and updating.
UPDATE
But we’re still having connection problems that we can’t trace to our Internet connection. Here are some common logs that journalctl
spits out::
balenad[1371]: time="2021-10-16T14:59:45.197662037Z" level=warning msg="Error getting v2 registry: Get https://registry2.balena-cloud.com/v2/: dial tcp 54.166.87.228:443: connect: connection refused"
balenad[1371]: time="2021-10-16T14:59:45.197807182Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry2.balena-cloud.com/v2/: dial tcp 54.166.87.228:443: connect: connection refused"
balenad[1371]: time="2021-10-16T14:59:45.197977420Z" level=error msg="Handler for POST /images/create returned error: Get https://registry2.balena-cloud.com/v2/: dial tcp 54.166.87.228:443: connect: connection refused"
3d8d94b4953c[1371]: [event] Event: Docker image download {"image":{"name":"registry2.balena-cloud.com/v2/b3069c84e30c1baa86d9277f76485e9b@sha256:8f8bec2d36c48a93e9ebbfe3fd1f26399f279da4615a23f55517008e7dc2b4fa","appId":1872644,"serviceId":1264730,"serviceName":"nginx-proxy","imageId":4139115,"releaseId":1947971,"dependent":0}}
balena-supervisor[1895]: [event] Event: Docker image download {"image":{"name":"registry2.balena-cloud.com/v2/b3069c84e30c1baa86d9277f76485e9b@sha256:8f8bec2d36c48a93e9ebbfe3fd1f26399f279da4615a23f55517008e7dc2b4fa","appId":1872644,"serviceId":1264730,"serviceName":"nginx-proxy","imageId":4139115,"releaseId":1947971,"dependent":0}}
3d8d94b4953c[1371]: [event] Event: Image download error {"error":{"message":"(HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 54.166.87.228:443: connect: connection refused ","stack":"Error: (HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 54.166.87.228:443: connect: connection refused \n at /usr/src/app/dist/app.js:10:2333796\n at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:10:2333683)\n at IncomingMessage.emit (events.js:322:22)\n at endReadableNT (_stream_readable.js:1187:12)\n at processTicksAndRejections (internal/process/task_queues.js:84:21)"},"image":{"name":"registry2.balena-cloud.com/v2/d886532ea1b99adbb0237c2d6f2f06e8@sha256:17e463649c4708feea31945d866bafde3fb448c6e715b6c6bc9f1b0f865ec7fc","appId":1872644,"serviceId":1262445,"serviceName":"metube","imageId":4139120,"releaseId":1947971,"dependent":0}}
balena-supervisor[1895]: [event] Event: Image download error {"error":{"message":"(HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 54.166.87.228:443: connect: connection refused ","stack":"Error: (HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 54.166.87.228:443: connect: connection refused \n at /usr/src/app/dist/app.js:10:2333796\n at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:10:2333683)\n at IncomingMessage.emit (events.js:322:22)\n at endReadableNT (_stream_readable.js:1187:12)\n at processTicksAndRejections (internal/process/task_queues.js:84:21)"},"image":{"name":"registry2.balena-cloud.com/v2/d886532ea1b99adbb0237c2d6f2f06e8@sha256:17e463649c4708feea31945d866bafde3fb448c6e715b6c6bc9f1b0f865ec7fc","appId":1872644,"serviceId":1262445,"serviceName":"metube","imageId":4139120,"releaseId":1947971,"dependent":0}}
3d8d94b4953c[1371]: [event] Event: Image download error {"error":{"message":""},"image":{"name":"registry2.balena-cloud.com/v2/f4772ae9b83aba9f630b89b4c301258b@sha256:ed4b56a834f736c36c37a82d309a1091ddd693448aa557ed453e849ff88c9bb4","appId":1872644,"serviceId":1261893,"serviceName":"portal","imageId":4139118,"releaseId":1947971,"dependent":0}}
balena-supervisor[1895]: [event] Event: Image download error {"error":{"message":""},"image":{"name":"registry2.balena-cloud.com/v2/f4772ae9b83aba9f630b89b4c301258b@sha256:ed4b56a834f736c36c37a82d309a1091ddd693448aa557ed453e849ff88c9bb4","appId":1872644,"serviceId":1261893,"serviceName":"portal","imageId":4139118,"releaseId":1947971,"dependent":0}}
balenad[1371]: time="2021-10-16T14:59:45.224557409Z" level=warning msg="Error getting v2 registry: Get https://registry2.balena-cloud.com/v2/: dial tcp 3.223.56.254:443: connect: connection refused"
balenad[1371]: time="2021-10-16T14:59:45.224720313Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry2.balena-cloud.com/v2/: dial tcp 3.223.56.254:443: connect: connection refused"
balenad[1371]: time="2021-10-16T14:59:45.224899644Z" level=error msg="Handler for POST /images/create returned error: Get https://registry2.balena-cloud.com/v2/: dial tcp 3.223.56.254:443: connect: connection refused"
3d8d94b4953c[1371]: [event] Event: Image download error {"error":{"message":"(HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 3.223.56.254:443: connect: connection refused ","stack":"Error: (HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 3.223.56.254:443: connect: connection refused \n at /usr/src/app/dist/app.js:10:2333796\n at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:10:2333683)\n at IncomingMessage.emit (events.js:322:22)\n at endReadableNT (_stream_readable.js:1187:12)\n at processTicksAndRejections (internal/process/task_queues.js:84:21)"},"image":{"name":"registry2.balena-cloud.com/v2/b3069c84e30c1baa86d9277f76485e9b@sha256:8f8bec2d36c48a93e9ebbfe3fd1f26399f279da4615a23f55517008e7dc2b4fa","appId":1872644,"serviceId":1264730,"serviceName":"nginx-proxy","imageId":4139115,"releaseId":1947971,"dependent":0}}
balena-supervisor[1895]: [event] Event: Image download error {"error":{"message":"(HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 3.223.56.254:443: connect: connection refused ","stack":"Error: (HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: dial tcp 3.223.56.254:443: connect: connection refused \n at /usr/src/app/dist/app.js:10:2333796\n at IncomingMessage.<anonymous> (/usr/src/app/dist/app.js:10:2333683)\n at IncomingMessage.emit (events.js:322:22)\n at endReadableNT (_stream_readable.js:1187:12)\n at processTicksAndRejections (internal/process/task_queues.js:84:21)"},"image":{"name":"registry2.balena-cloud.com/v2/b3069c84e30c1baa86d9277f76485e9b@sha256:8f8bec2d36c48a93e9ebbfe3fd1f26399f279da4615a23f55517008e7dc2b4fa","appId":1872644,"serviceId":1264730,"serviceName":"nginx-proxy","imageId":4139115,"releaseId":1947971,"dependent":0}}
3d8d94b4953c[1371]: [warn] Ignoring unsupported or unknown compose fields: containerName
balena-supervisor[1895]: [warn] Ignoring unsupported or unknown compose fields: containerName
3d8d94b4953c[1371]: [error] Scheduling another update attempt in 128000ms due to failure: Error: Failed to apply state transition steps. Steps:["fetch","fetch","fetch"]
balena-supervisor[1895]: [error] Scheduling another update attempt in 128000ms due to failure: Error: Failed to apply state transition steps. Steps:["fetch","fetch","fetch"]
balena-supervisor[1895]: [error] at fn (/usr/src/app/dist/app.js:6:8594)
3d8d94b4953c[1371]: [error] at fn (/usr/src/app/dist/app.js:6:8594)
3d8d94b4953c[1371]: [error] Device state apply error Error: Failed to apply state transition steps. Steps:["fetch","fetch","fetch"]
balena-supervisor[1895]: [error] Device state apply error Error: Failed to apply state transition steps. Steps:["fetch","fetch","fetch"]
balena-supervisor[1895]: [error] at fn (/usr/src/app/dist/app.js:6:8594)
3d8d94b4953c[1371]: [error] at fn (/usr/src/app/dist/app.js:6:8594)
openvpn[1383]: Sat
E: the current --script-security setting may allow this configuration to call user-defined scripts
openvpn[1383]: Sat
/UDP: Preserving recently used remote address: [AF_INET6]2600:1f18:6600:7f02:3ef5:d239:9bb4:6036:443
openvpn[1383]: Sat
ket Buffers: R=[131072->131072] S=[16384->16384]
openvpn[1383]: Sat
empting to establish TCP connection with [AF_INET6]2600:1f18:6600:7f02:3ef5:d239:9bb4:6036:443 [nonblock]
openvpn[1383]: Sat
: connect to [AF_INET6]2600:1f18:6600:7f02:3ef5:d239:9bb4:6036:443 failed: Permission denied
openvpn[1383]: Sat
USR1[connection failed(soft),init_instance] received, process restarting
openvpn[1383]: Sat
tart pause, 10 second(s)
balena-supervisor[1895]: [error] LogBackend: unexpected error: Error: connect ECONNREFUSED 3.226.114.49:443
balena-supervisor[1895]: [error] at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)
3d8d94b4953c[1371]: [error] LogBackend: unexpected error: Error: connect ECONNREFUSED 3.226.114.49:443
3d8d94b4953c[1371]: [error] at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)
openvpn[1383]: Sat
E: the current --script-security setting may allow this configuration to call user-defined scripts
openvpn[1383]: Sat
/UDP: Preserving recently used remote address: [AF_INET]35.169.89.252:443
openvpn[1383]: Sat
ket Buffers: R=[131072->131072] S=[16384->16384]
openvpn[1383]: Sat
empting to establish TCP connection with [AF_INET]35.169.89.252:443 [nonblock]
openvpn[1383]: Sat
: connect to [AF_INET]35.169.89.252:443 failed: Connection refused
openvpn[1383]: Sat
USR1[connection failed(soft),init_instance] received, process restarting
openvpn[1383]: Sat
tart pause, 20 second(s)