Why is my service aborting? Crash log

Product support
1

Hello Balenafam,

I was hoping someone might be able to shed some light… Is this a bug?
I have a 32bit armhf fleet, 3 devices. 2 pizero2w and 1 pzero1w.
2 containers running on each, hostname and a debian rpi base image container, running httpd on port 80.
one pizero2w and the pizero1w works, and one pizero2w doesnt.
the debian [cam] service runs, and aborts after a few minutes.

Nov 24 16:58:31 cam2 sshd[36289]: Received disconnect from xx port 3219:11:
Nov 24 16:58:31 cam2 sshd[36289]: Disconnected from user root x port 3219
Nov 24 16:58:33 cam2 balenad[36967]: runtime/cgo: pthread_create failed: Resource temporarily unavailable
Nov 24 16:58:33 cam2 balenad[36967]: SIGABRT: abort
Nov 24 16:58:33 cam2 balenad[36967]: PC=0x7fa059268c m=3 sigcode=18446744073709551610
Nov 24 16:58:33 cam2 balenad[36967]: goroutine 0 [idle]:
Nov 24 16:58:33 cam2 balenad[36967]: runtime: unknown pc 0x7fa059268c
Nov 24 16:58:33 cam2 balenad[36967]: stack: frame={sp:0x7f78f62130, fp:0x0} stack=[0x7f78762b60,0x7f78f62760)
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62030:  00000000003c47b5  0000007fa079a588
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62040:  0000007fa07a0fb0  0000007f78f62068
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62050:  0000007f78f62064  0000007fa0525998
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62060:  0000000078f620c0  0000000000000000
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62070:  0000000000000005  0000000000000000
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62080:  0000007f00000001  0000007fa07a3270
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62090:  0000007f78f62190  0000007fa077abe4
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620a0:  0000000002af8190  0000007f78f63840
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620b0:  0000007f78f624e0  0000007f78f630b0
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620c0:  0000007fc50f2237  000000000080e920 <net/http.(*http2Framer).WriteDataPadded+976>
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620d0:  0000007f78754000  0000007f78f622a0
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620e0:  0000000000000000  0000004000000d80
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620f0:  0000007f00000000  0000007fa07a3608
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62100:  0000000000000000  0000007fa07a3270
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62110:  0000000000000000  0000007f78f62158
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62120:  0000007fa07a2018  0000007fa0526698
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62130: <0000007f78f62150  0000007fa05592d8
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62140:  0000000000000006  0000007fa06876c0
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62150:  0000007f78f62170  0000007fa0548da4
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62160:  0000007fa0687000  0000007fa0548d58
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62170:  0000007f78f622c0  0000000001698180
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62180:  0000007fa0686600  0000000001938fab
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62190:  0000007f78f624e0  0000007fa0780774
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621a0:  0000007fa0686600  0000000000000020
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621b0:  0000007f78f624e0  0000000000000001
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621c0:  0000007fa0532b78  0000007fa06864c3
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621d0:  0000007f78f62200  0000007fa05894d4
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621e0:  0000000000000040  0000007fa063cd44
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621f0:  656372756f736552  61726f706d657420
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62200:  0000007fa06423b0  0000007f78f5fd27
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62210:  00000000fbad2887  0000000000000000
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62220:  000000000000000a  0000007fa06881f0
Nov 24 16:58:33 cam2 balenad[36967]: runtime: unknown pc 0x7fa059268c
Nov 24 16:58:33 cam2 balenad[36967]: stack: frame={sp:0x7f78f62130, fp:0x0} stack=[0x7f78762b60,0x7f78f62760)
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62030:  00000000003c47b5  0000007fa079a588
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62040:  0000007fa07a0fb0  0000007f78f62068
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62050:  0000007f78f62064  0000007fa0525998
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62060:  0000000078f620c0  0000000000000000
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62070:  0000000000000005  0000000000000000
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62080:  0000007f00000001  0000007fa07a3270
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62090:  0000007f78f62190  0000007fa077abe4
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620a0:  0000000002af8190  0000007f78f63840
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620b0:  0000007f78f624e0  0000007f78f630b0
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620c0:  0000007fc50f2237  000000000080e920 <net/http.(*http2Framer).WriteDataPadded+976>
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620d0:  0000007f78754000  0000007f78f622a0
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620e0:  0000000000000000  0000004000000d80
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f620f0:  0000007f00000000  0000007fa07a3608
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62100:  0000000000000000  0000007fa07a3270
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62110:  0000000000000000  0000007f78f62158
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62120:  0000007fa07a2018  0000007fa0526698
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62130: <0000007f78f62150  0000007fa05592d8
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62140:  0000000000000006  0000007fa06876c0
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62150:  0000007f78f62170  0000007fa0548da4
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62160:  0000007fa0687000  0000007fa0548d58
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62170:  0000007f78f622c0  0000000001698180
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62180:  0000007fa0686600  0000000001938fab
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62190:  0000007f78f624e0  0000007fa0780774
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621a0:  0000007fa0686600  0000000000000020
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621b0:  0000007f78f624e0  0000000000000001
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621c0:  0000007fa0532b78  0000007fa06864c3
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621d0:  0000007f78f62200  0000007fa05894d4
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621e0:  0000000000000040  0000007fa063cd44
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f621f0:  656372756f736552  61726f706d657420
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62200:  0000007fa06423b0  0000007f78f5fd27
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62210:  00000000fbad2887  0000000000000000
Nov 24 16:58:33 cam2 balenad[36967]: 0000007f78f62220:  000000000000000a  0000007fa06881f0
Nov 24 16:58:33 cam2 balenad[36967]: goroutine 1 [runnable, locked to thread]:
Nov 24 16:58:33 cam2 balenad[36967]: runtime.gcenable()
Nov 24 16:58:33 cam2 balenad[36967]:         /usr/lib/go/src/runtime/mgc.go:214 +0xb8
Nov 24 16:58:33 cam2 balenad[36967]: runtime.main()
Nov 24 16:58:33 cam2 balenad[36967]:         /usr/lib/go/src/runtime/proc.go:184 +0x164
Nov 24 16:58:33 cam2 balenad[36967]: runtime.goexit()
Nov 24 16:58:33 cam2 balenad[36967]:         /usr/lib/go/src/runtime/asm_arm64.s:1130 +0x4
Nov 24 16:58:33 cam2 balenad[36967]: r0      0x0
Nov 24 16:58:33 cam2 balenad[36967]: r1      0x9069
Nov 24 16:58:33 cam2 balenad[36967]: r2      0x6
Nov 24 16:58:33 cam2 balenad[36967]: r3      0x8
Nov 24 16:58:33 cam2 balenad[36967]: r4      0x20
Nov 24 16:58:33 cam2 balenad[36967]: r5      0x7f78f5fd27
Nov 24 16:58:33 cam2 balenad[36967]: r6      0x656372756f736552
Nov 24 16:58:33 cam2 balenad[36967]: r7      0x61726f706d657420
Nov 24 16:58:33 cam2 balenad[36967]: r8      0x83
Nov 24 16:58:33 cam2 balenad[36967]: r9      0x7fa063cd44
Nov 24 16:58:33 cam2 balenad[36967]: r10     0x7fa0525998
Nov 24 16:58:33 cam2 balenad[36967]: r11     0x7f78f62068
Nov 24 16:58:33 cam2 balenad[36967]: r12     0x7f78f62064
Nov 24 16:58:33 cam2 balenad[36967]: r13     0x656c62616c696176
Nov 24 16:58:33 cam2 balenad[36967]: r14     0x7f6c000960
Nov 24 16:58:33 cam2 balenad[36967]: r15     0x0
Nov 24 16:58:33 cam2 balenad[36967]: r16     0x0
Nov 24 16:58:33 cam2 balenad[36967]: r17     0x0
Nov 24 16:58:33 cam2 balenad[36967]: r18     0x270f
Nov 24 16:58:33 cam2 balenad[36967]: r19     0x9069
Nov 24 16:58:33 cam2 balenad[36967]: r20     0x6
Nov 24 16:58:33 cam2 balenad[36967]: r21     0x7f78f630b0
Nov 24 16:58:33 cam2 balenad[36967]: r22     0x1
Nov 24 16:58:33 cam2 balenad[36967]: r23     0x7fc50f2237
Nov 24 16:58:33 cam2 balenad[36967]: r24     0x80e920
Nov 24 16:58:33 cam2 balenad[36967]: r25     0x7f78754000
Nov 24 16:58:33 cam2 balenad[36967]: r26     0x7f78f622a0
Nov 24 16:58:33 cam2 balenad[36967]: r27     0x0
Nov 24 16:58:33 cam2 balenad[36967]: r28     0x4000000d80
Nov 24 16:58:33 cam2 balenad[36967]: r29     0x7f78f62130
Nov 24 16:58:33 cam2 balenad[36967]: lr      0x7fa0592678
Nov 24 16:58:33 cam2 balenad[36967]: sp      0x7f78f62130
Nov 24 16:58:33 cam2 balenad[36967]: pc      0x7fa059268c
Nov 24 16:58:33 cam2 balenad[36967]: fault   0x0
Nov 24 16:58:33 cam2 balenad[30606]: time="2022-11-24T16:58:33.095523098Z" level=warning msg="Failed to disable IPv6 on all interfaces on network namespace \"/var/run/balena-engine/netns/28ab045b0f23\": reexec to set IPv6 failed: exit status 2"
Nov 24 16:58:33 cam2 balenad[36991]: runtime/cgo: pthread_create failed: Resource temporarily unavailable
Nov 24 16:58:33 cam2 balenad[36991]: SIGABRT: abort
Nov 24 16:58:33 cam2 balenad[36991]: PC=0x7fbe09468c m=3 sigcode=18446744073709551610
Nov 24 16:58:33 cam2 balenad[36991]: goroutine 0 [idle]:
Nov 24 16:58:33 cam2 balenad[36991]: runtime: unknown pc 0x7fbe09468c
Nov 24 16:58:33 cam2 balenad[36991]: stack: frame={sp:0x7f96a64130, fp:0x0} stack=[0x7f96264b60,0x7f96a64760)
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64030:  00000000003c47b5  0000007fbe29c588
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64040:  0000007fbe2a2fb0  0000007f96a64068
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64050:  0000007f96a64064  0000007fbe027998
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64060:  0000000096a640c0  0000000000000000
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64070:  0000000000000005  0000000000000000
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64080:  0000007f00000001  0000007fbe2a5270
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64090:  0000007f96a64190  0000007fbe27cbe4
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640a0:  0000000002af8190  0000007f96a65840
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640b0:  0000007f96a644e0  0000007f96a650b0
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640c0:  0000007fd13b2727  000000000080e920 <net/http.(*http2Framer).WriteDataPadded+976>
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640d0:  0000007f96256000  0000007f96a642a0
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640e0:  0000000000000000  0000004000000d80
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640f0:  0000007f00000000  0000007fbe2a5608
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64100:  0000000000000000  0000007fbe2a5270
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64110:  0000000000000000  0000007f96a64158
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64120:  0000007fbe2a4018  0000007fbe028698
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64130: <0000007f96a64150  0000007fbe05b2d8
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64140:  0000000000000006  0000007fbe1896c0
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64150:  0000007f96a64170  0000007fbe04ada4
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64160:  0000007fbe189000  0000007fbe04ad58
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64170:  0000007f96a642c0  0000000001698180
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64180:  0000007fbe188600  0000000001938fab
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64190:  0000007f96a644e0  0000007fbe282774
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641a0:  0000007fbe188600  0000000000000020
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641b0:  0000007f96a644e0  0000000000000001
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641c0:  0000007fbe034b78  0000007fbe1884c3
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641d0:  0000007f96a64200  0000007fbe08b4d4
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641e0:  0000000000000040  0000007fbe13ed44
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641f0:  656372756f736552  61726f706d657420
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64200:  0000007fbe1443b0  0000007f96a61d27
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64210:  00000000fbad2887  0000000000000000
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64220:  000000000000000a  0000007fbe18a1f0
Nov 24 16:58:33 cam2 balenad[36991]: runtime: unknown pc 0x7fbe09468c
Nov 24 16:58:33 cam2 balenad[36991]: stack: frame={sp:0x7f96a64130, fp:0x0} stack=[0x7f96264b60,0x7f96a64760)
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64030:  00000000003c47b5  0000007fbe29c588
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64040:  0000007fbe2a2fb0  0000007f96a64068
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64050:  0000007f96a64064  0000007fbe027998
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64060:  0000000096a640c0  0000000000000000
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64070:  0000000000000005  0000000000000000
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64080:  0000007f00000001  0000007fbe2a5270
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64090:  0000007f96a64190  0000007fbe27cbe4
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640a0:  0000000002af8190  0000007f96a65840
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640b0:  0000007f96a644e0  0000007f96a650b0
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640c0:  0000007fd13b2727  000000000080e920 <net/http.(*http2Framer).WriteDataPadded+976>
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640d0:  0000007f96256000  0000007f96a642a0
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640e0:  0000000000000000  0000004000000d80
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a640f0:  0000007f00000000  0000007fbe2a5608
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64100:  0000000000000000  0000007fbe2a5270
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64110:  0000000000000000  0000007f96a64158
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64120:  0000007fbe2a4018  0000007fbe028698
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64130: <0000007f96a64150  0000007fbe05b2d8
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64140:  0000000000000006  0000007fbe1896c0
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64150:  0000007f96a64170  0000007fbe04ada4
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64160:  0000007fbe189000  0000007fbe04ad58
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64170:  0000007f96a642c0  0000000001698180
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64180:  0000007fbe188600  0000000001938fab
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64190:  0000007f96a644e0  0000007fbe282774
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641a0:  0000007fbe188600  0000000000000020
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641b0:  0000007f96a644e0  0000000000000001
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641c0:  0000007fbe034b78  0000007fbe1884c3
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641d0:  0000007f96a64200  0000007fbe08b4d4
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641e0:  0000000000000040  0000007fbe13ed44
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a641f0:  656372756f736552  61726f706d657420
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64200:  0000007fbe1443b0  0000007f96a61d27
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64210:  00000000fbad2887  0000000000000000
Nov 24 16:58:33 cam2 balenad[36991]: 0000007f96a64220:  000000000000000a  0000007fbe18a1f0
Nov 24 16:58:33 cam2 balenad[36991]: goroutine 1 [runnable, locked to thread]:
Nov 24 16:58:33 cam2 balenad[36991]: runtime.gcenable()
Nov 24 16:58:33 cam2 balenad[36991]:         /usr/lib/go/src/runtime/mgc.go:214 +0xb8
Nov 24 16:58:33 cam2 balenad[36991]: runtime.main()
Nov 24 16:58:33 cam2 balenad[36991]:         /usr/lib/go/src/runtime/proc.go:184 +0x164
Nov 24 16:58:33 cam2 balenad[36991]: runtime.goexit()
Nov 24 16:58:33 cam2 balenad[36991]:         /usr/lib/go/src/runtime/asm_arm64.s:1130 +0x4
Nov 24 16:58:33 cam2 balenad[36991]: r0      0x0
Nov 24 16:58:33 cam2 balenad[36991]: r1      0x9081
Nov 24 16:58:33 cam2 balenad[36991]: r2      0x6
Nov 24 16:58:33 cam2 balenad[36991]: r3      0x8
Nov 24 16:58:33 cam2 balenad[36991]: r4      0x20
Nov 24 16:58:33 cam2 balenad[36991]: r5      0x7f96a61d27
Nov 24 16:58:33 cam2 balenad[36991]: r6      0x656372756f736552
Nov 24 16:58:33 cam2 balenad[36991]: r7      0x61726f706d657420
Nov 24 16:58:33 cam2 balenad[36991]: r8      0x83
Nov 24 16:58:33 cam2 balenad[36991]: r9      0x7fbe13ed44
Nov 24 16:58:33 cam2 balenad[36991]: r10     0x7fbe027998
Nov 24 16:58:33 cam2 balenad[36991]: r11     0x7f96a64068
Nov 24 16:58:33 cam2 balenad[36991]: r12     0x7f96a64064
Nov 24 16:58:33 cam2 balenad[36991]: r13     0x656c62616c696176
Nov 24 16:58:33 cam2 balenad[36991]: r14     0x7f88000960
Nov 24 16:58:33 cam2 balenad[36991]: r15     0x0
Nov 24 16:58:33 cam2 balenad[36991]: r16     0x0
Nov 24 16:58:33 cam2 balenad[36991]: r17     0x0
Nov 24 16:58:33 cam2 balenad[36991]: r18     0x270f
Nov 24 16:58:33 cam2 balenad[36991]: r19     0x9081
Nov 24 16:58:33 cam2 balenad[36991]: r20     0x6
Nov 24 16:58:33 cam2 balenad[36991]: r21     0x7f96a650b0
Nov 24 16:58:33 cam2 balenad[36991]: r22     0x1
Nov 24 16:58:33 cam2 balenad[36991]: r23     0x7fd13b2727
Nov 24 16:58:33 cam2 balenad[36991]: r24     0x80e920
Nov 24 16:58:33 cam2 balenad[36991]: r25     0x7f96256000
Nov 24 16:58:33 cam2 balenad[36991]: r26     0x7f96a642a0
Nov 24 16:58:33 cam2 balenad[36991]: r27     0x0
Nov 24 16:58:33 cam2 balenad[36991]: r28     0x4000000d80
Nov 24 16:58:33 cam2 balenad[36991]: r29     0x7f96a64130
Nov 24 16:58:33 cam2 balenad[36991]: lr      0x7fbe094678
Nov 24 16:58:33 cam2 balenad[36991]: sp      0x7f96a64130
Nov 24 16:58:33 cam2 balenad[36991]: pc      0x7fbe09468c
Nov 24 16:58:33 cam2 balenad[36991]: fault   0x0
Nov 24 16:58:33 cam2 balenad[30606]: time="2022-11-24T16:58:33.416984195Z" level=error msg="Resolver Start failed for container c48282864355987ce19003e86355739e19bc732411b2bc98f75c4e8a62090e8a, \"setting up IP table rules failed: reexec failed: exit status 2\""
Nov 24 16:58:33 cam2 kernel: eth0: renamed from veth8464ebd
Nov 24 16:58:33 cam2 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Nov 24 16:58:33 cam2 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth110a80e: link becomes ready
Nov 24 16:58:33 cam2 kernel: br-a1d67e3bfa20: port 1(veth110a80e) entered blocking state
Nov 24 16:58:33 cam2 kernel: br-a1d67e3bfa20: port 1(veth110a80e) entered forwarding state
Nov 24 16:58:33 cam2 NetworkManager[1524]: <info>  [1669309113.5555] device (veth110a80e): carrier: link connected
Nov 24 16:58:33 cam2 NetworkManager[1524]: <info>  [1669309113.5838] device (br-a1d67e3bfa20): carrier: link connected
Nov 24 16:58:33 cam2 kernel: eth1: renamed from veth96a4484
Nov 24 16:58:33 cam2 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
Nov 24 16:58:33 cam2 kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethcb93b13: link becomes ready
Nov 24 16:58:33 cam2 kernel: supervisor0: port 1(vethcb93b13) entered blocking state
Nov 24 16:58:33 cam2 kernel: supervisor0: port 1(vethcb93b13) entered forwarding state
Nov 24 16:58:33 cam2 NetworkManager[1524]: <info>  [1669309113.6429] device (vethcb93b13): carrier: link connected
Nov 24 16:58:33 cam2 NetworkManager[1524]: <info>  [1669309113.6459] device (supervisor0): carrier: link connected
Nov 24 16:58:34 cam2 c48282864355[30606]: + export DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket
Nov 24 16:58:34 cam2 c48282864355[30606]: + DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket
Nov 24 16:58:34 cam2 c48282864355[30606]: + sleep 10
Nov 24 16:58:34 cam2 6c6b2d684453[30606]: [event]   Event: Service started {"service":{"appId":1976283,"serviceId":1858215,"serviceName":"cam","commit":"fb39de8dc95e470590caf9fb93513ef6","releaseId":2385647}}
Nov 24 16:58:34 cam2 balena-supervisor[31213]: [event]   Event: Service started {"service":{"appId":1976283,"serviceId":1858215,"serviceName":"cam","commit":"fb39de8dc95e470590caf9fb93513ef6","releaseId":2385647}}
Nov 24 16:58:34 cam2 balena-supervisor[31213]: [api]     POST /v2/applications/1976283/start-service 200 - 11742.297 ms
Nov 24 16:58:34 cam2 6c6b2d684453[30606]: [api]     POST /v2/applications/1976283/start-service 200 - 11742.297 ms
Nov 24 16:58:35 cam2 avahi-daemon[29614]: Joining mDNS multicast group on interface vethcb93b13.IPv6 with address fe80::c897:bbff:fe68:2e41.
Nov 24 16:58:35 cam2 avahi-daemon[29614]: New relevant interface vethcb93b13.IPv6 for mDNS.
Nov 24 16:58:35 cam2 avahi-daemon[29614]: Registering new address record for fe80::c897:bbff:fe68:2e41 on vethcb93b13.*.
Nov 24 16:58:35 cam2 balena-supervisor[31213]: [debug]   Spawning journald with: chroot  /mnt/root journalctl -a -S 2022-11-24 15:19:18 -o json CONTAINER_ID_FULL=xx
Nov 24 16:58:35 cam2 6c6b2d684453[30606]: [debug]   Spawning journald with: chroot  /mnt/root journalctl -a -S 2022-11-24 15:19:18 -o json CONTAINER_ID_FULL=xx
Nov 24 16:58:35 cam2 avahi-daemon[29614]: Joining mDNS multicast group on interface veth110a80e.IPv6 with address fe80::1c4a:b5ff:fe13:ddaf.
Nov 24 16:58:35 cam2 avahi-daemon[29614]: New relevant interface veth110a80e.IPv6 for mDNS.
Nov 24 16:58:35 cam2 avahi-daemon[29614]: Registering new address record for fe80::1c4a:b5ff:fe13:ddaf on veth110a80e.*.
Nov 24 16:58:37 cam2 6c6b2d684453[30606]: [info]    Reported current state to the cloud
Nov 24 16:58:37 cam2 balena-supervisor[31213]: [info]    Reported current state to the cloud

I python well , but i don’t comprehend go as well. what is happening with the dump above?

for some reason, when the device is in this state - the public url doesnt redirect to post 80 but to somewhere else.

any help would be great!

kindly,

puc.

2

Hello @puccaso could you please give more details on how are you connecting your device to the Internet and the balenaOS and supervisor versions?

Thanks!

3

Hello Marc,
So - for the 3 devices, they are all connecting via 2.4g wifi.
for the 2 pizero2w devices, they have usb asic ethernet adapters also.
the details are as below.

  "os_version": "balenaOS 2.94.4",
  "supervisor_version": "14.4.1",
4

Somehow, the issue was related to my dockerfile and entrypoint script. However, even if i pushed the new build with fixed code, it wouldn’t deploy on the device due to the error above.
issue is resolved now,
however i did have to :
1- create new empty fleet
2- move devices to empty fleet and allow device to restart fully and become idle
3- balena system prune
4- move device back to original fleet.

thanks for everyones help!

kindly,
puc

5

Hi Puc, thanks for sharing how you fixed this issue here for everyone :+1:

1 Like