ResinOS is dumping stack traces while system is idle

Product support
1

Unmanaged Beaglebone Greens running ResinOS are dumping stack traces while idle. The stack traces are followed by a number of broadcast messages from systemd.

Find appended an example of two events from a node idling in our dev environment this past weekend.

—cut—

[77165.728891] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
[77165.734336] Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo br_netfilter pru_rproc pruss_intc pruss omap_aes_driver crypto_engine omap_sham pruss_soc_bus nls_ascii nls_cp437 omap_rng rng_core evdev tps65217_charger uio_pdrv_genirq uio sch_fq_codel
[77165.759043] CPU: 0 PID: 733 Comm: docker-containe Not tainted 4.9.34+ #2
[77165.765774] Hardware name: Generic AM33XX (Flattened Device Tree)
[77165.771895] task: db3090c0 task.stack: db6f6000
[77165.776461] PC is at do_select+0x88/0x6e0
[77165.780489] LR is at do_select+0x6c/0x6e0
[77165.784518] pc : [<c02f3f08>]    lr : [<c02f3eec>]    psr: 600f0013
[77165.784518] sp : db6f7ae0  ip : db6f7ae0  fp : db6f7dec
[77165.796047] r10: 00000000  r9 : 00000000  r8 : db6f7e1c
[77165.801294] r7 : 00000000  r6 : db6f7e04  r5 : 00000000  r4 : 00000000
[77165.807849] r3 : 00000000  r2 : 00000000  r1 : db6f7e04  r0 : 00000000
[77165.814407] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[77165.821573] Control: 10c5387d  Table: 9c51c019  DAC: 00000051
[77165.827343] Process docker-containe (pid: 733, stack limit = 0xdb6f6218)
[77165.834074] Stack: (0xdb6f7ae0 to 0xdb6f8000)
[77165.838455] 7ae0: 00000000 3b9aca00 0000c350 00000001 00000000 c150518c c15073fc db6f6000
[77165.846673] 7b00: db6f7b58 db6f7f60 db6f7e04 00000000 00004433 00000000 db6f7b64 db6f7b28
[77165.854890] 7b20: c017a7a8 c0175ad4 00004433 db6f7e1c db6f7e1c db6f7e1c db6f7e1c db6f7e1c
[77165.863107] 7b40: db6f7e1c 00000000 df933178 c017aa3c 00000400 00000000 57066bdc 0000462e
[77165.871325] 7b60: 00000000 ffffffff 00000000 db3090c0 00000000 00000000 00000000 db6f7b88
[77165.879542] 7b80: c017c3c4 c017a990 db6f6000 00000100 db6f7bac db6f7ba0 c1503088 c01446a0
[77165.887760] 7ba0: db6f7bc4 db6f7bb0 c01446a0 c016b0f4 00031d5e 00000000 db309140 db6f7bc8
[77165.895977] 7bc0: 00000800 c011d414 c150518c 00000001 00000000 c017e210 dc4761c0 dc476248
[77165.904195] 7be0: dc476240 db744d00 db6f7c6c db6f7bf8 c01795b0 c0693898 00000001 c017db40
[77165.912412] 7c00: df933178 00000000 003f62e7 c017db64 db6f7cbc db6f7c20 c017db64 c0177198
[77165.920629] 7c20: 00000400 00000000 00000000 c0177478 8bb61fa2 0000462e 00000000 00000000
[77165.928846] 7c40: c0177478 dc4761c0 dc476240 db744d00 db309140 db3090c0 dc1f6600 df933140
[77165.937063] 7c60: db6f7cec db6f7c70 c0184394 c0179570 afb50401 00000000 df933140 dc4761c0
[77165.945281] 7c80: db5ee000 c0becd3c db6f7ca4 db6f7c98 db6f7cbc c011d414 db305ae0 01ad5000
[77165.953497] 7ca0: 00000207 c0bec950 db5ee034 db307000 00000207 dc51c068 db6f7cdc c011ada0
[77165.961715] 7cc0: dfc7da64 00001000 01ad5000 00000207 db6f6000 db3090c0 db6f7d0c db6f7ce8
[77165.969931] 7ce0: c011ada0 c02a67d4 dfc7da64 00000000 01ad5000 00000207 db6f6000 db3090c0
[77165.978148] 7d00: db6f7d24 db6f7d10 c011aef0 c011ad60 db305ae0 c150518c db6f7d8c db6f7d28
[77165.986366] 7d20: c02a5d9c c011ae50 c150518c ffffe000 00000000 00000002 dfc7da64 c16a1bc4
[77165.994583] 7d40: fffff000 db6f7e10 00000000 db5ee000 00000000 db6f7db0 00000000 00040922
[77166.002799] 7d60: c01d6574 db6f7e10 00000001 01ad5000 00000001 00000000 db3090c0 c02a6388
[77166.011016] 7d80: db6f7de4 db6f7d90 c02a6388 c0194ac0 00000207 db6f7e10 00000000 db6f7db0
[77166.019234] 7da0: c150518c db5ee038 db6f7e94 00000001 00000001 00040922 db6f7dd4 00040922
[77166.027451] 7dc0: db6f7e50 00000000 db6f7e1c c150518c 00000000 db6f7e1c 00000000 db6f7e1c
[77166.035668] 7de0: db6f7f4c db6f7df0 c02f4778 c02f3e8c c01d6338 00000000 db6f7f60 00000000
[77166.043885] 7e00: c1538c57 db6f7e1c db6f7e1c db6f7e1c db6f7e1c db6f7e1c db6f7e1c c150518c
[77166.052101] 7e20: db6f7e50 c150518c ffffffff 00000001 c150518c c01d6f6c db6f7e94 db6f7e48
[77166.060318] 7e40: c01d6f6c c01d6c98 c017a7a8 c0177478 00000001 db6f7e50 01ad5000 db5ee000
[77166.068536] 7e60: 00000cca 00040922 db744d00 00040922 00000001 c150518c 00000001 c01c8194
[77166.076753] 7e80: db6f7ed4 db6f7e90 c01c8194 c01c8194 db6f7ee4 db6f7ea0 c01c8194 c0126a10
[77166.084970] 7ea0: 29ff4a66 00d17a3c a60314b4 c02f3e6c 00012d6c 00000000 db6f7f00 c150518c
[77166.093187] 7ec0: db6f7f60 00000000 00000000 00000000 db6f7f4c db6f7ee8 c02f3e6c c01bcc48
[77166.101404] 7ee0: 00012d6c 00000000 323886e2 0000008e 00000000 00000000 00004e20 00000107
[77166.109621] 7f00: 00012d6c 00000000 3238d502 00d17a3c 00000000 00000000 00004e20 00040922
[77166.117839] 7f20: c0109124 b6a8dd40 c150518c 00000000 00000000 00000000 00000000 db6f7f60
[77166.126056] 7f40: db6f7fa4 db6f7f50 c02f4b48 c02f456c db6f7f60 db3090c0 00000000 00000014
[77166.134272] 7f60: 00012d6c 00000000 3238d502 00d17a3c db6f7fa4 00040922 c010ba84 b6a8dd40
[77166.142489] 7f80: 0035cfa1 0000462e 0000008e c0109124 db6f6000 00000020 00000000 db6f7fa8
[77166.150705] 7fa0: c0109100 c02f4a4c b6a8dd40 0035cfa1 00000000 00000000 00000000 00000000
[77166.158923] 7fc0: b6a8dd40 0035cfa1 0000462e 0000008e 00000014 00000000 11e005a0 00002710
[77166.167141] 7fe0: 0000462e b6a8dd3c 00084324 00084348 800f0010 00000000 9fde6861 9fde6c61
[77166.175376] [<c02f3f08>] (do_select) from [<c02f4778>] (core_sys_select+0x218/0x4e0)
[77166.183159] [<c02f4778>] (core_sys_select) from [<c02f4b48>] (SyS_select+0x108/0x178)
[77166.191038] [<c02f4b48>] (SyS_select) from [<c0109100>] (__sys_trace_return+0x0/0x10)
[77166.198910] Code: e1a0200d e3c22d7f e3c2203f e592200c (e59224b4)
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.728891] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.827343] Process d[77166.248844] ---[ end trace 592206016ddd67d7 ]---
ocker-containe (pid: 733, stack limit = 0xdb6f6218)
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.834074] Stack: (0xdb6f7ae0 to 0xdb6f8000)
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.838455] 7ae0: 00000000 3b9aca00 0000c350 00000001 00000000 c150518c c15073fc db6f6000
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.846673] 7b00: db6f7b58 db6f7f60 db6f7e04 00000000 00004433 00000000 db6f7b64 db6f7b28
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.854890] 7b20: c017a7a8 c0175ad4 00004433 db6f7e1c db6f7e1c db6f7e1c db6f7e1c db6f7e1c
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.863107] 7b40: db6f7e1c 00000000 df933178 c017aa3c 00000400 00000000 57066bdc 0000462e
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.871325] 7b60: 00000000 ffffffff 00000000 db3090c0 00000000 00000000 00000000 db6f7b88
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.879542] 7b80: c017c3c4 c017a990 db6f6000 00000100 db6f7bac db6f7ba0 c1503088 c01446a0
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.887760] 7ba0: db6f7bc4 db6f7bb0 c01446a0 c016b0f4 00031d5e 00000000 db309140 db6f7bc8
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.895977] 7bc0: 00000800 c011d414 c150518c 00000001 00000000 c017e210 dc4761c0 dc476248
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.904195] 7be0: dc476240 db744d00 db6f7c6c db6f7bf8 c01795b0 c0693898 00000001 c017db40
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.912412] 7c00: df933178 00000000 003f62e7 c017db64 db6f7cbc db6f7c20 c017db64 c0177198
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.920629] 7c20: 00000400 00000000 00000000 c0177478 8bb61fa2 0000462e 00000000 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.928846] 7c40: c0177478 dc4761c0 dc476240 db744d00 db309140 db3090c0 dc1f6600 df933140
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.937063] 7c60: db6f7cec db6f7c70 c0184394 c0179570 afb50401 00000000 df933140 dc4761c0
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.945281] 7c80: db5ee000 c0becd3c db6f7ca4 db6f7c98 db6f7cbc c011d414 db305ae0 01ad5000
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.953497] 7ca0: 00000207 c0bec950 db5ee034 db307000 00000207 dc51c068 db6f7cdc c011ada0
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.961715] 7cc0: dfc7da64 00001000 01ad5000 00000207 db6f6000 db3090c0 db6f7d0c db6f7ce8
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.969931] 7ce0: c011ada0 c02a67d4 dfc7da64 00000000 01ad5000 00000207 db6f6000 db3090c0
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.978148] 7d00: db6f7d24 db6f7d10 c011aef0 c011ad60 db305ae0 c150518c db6f7d8c db6f7d28
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.986366] 7d20: c02a5d9c c011ae50 c150518c ffffe000 00000000 00000002 dfc7da64 c16a1bc4
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77165.994583] 7d40: fffff000 db6f7e10 00000000 db5ee000 00000000 db6f7db0 00000000 00040922
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.002799] 7d60: c01d6574 db6f7e10 00000001 01ad5000 00000001 00000000 db3090c0 c02a6388
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.011016] 7d80: db6f7de4 db6f7d90 c02a6388 c0194ac0 00000207 db6f7e10 00000000 db6f7db0
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.019234] 7da0: c150518c db5ee038 db6f7e94 00000001 00000001 00040922 db6f7dd4 00040922
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.027451] 7dc0: db6f7e50 00000000 db6f7e1c c150518c 00000000 db6f7e1c 00000000 db6f7e1c
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.035668] 7de0: db6f7f4c db6f7df0 c02f4778 c02f3e8c c01d6338 00000000 db6f7f60 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.043885] 7e00: c1538c57 db6f7e1c db6f7e1c db6f7e1c db6f7e1c db6f7e1c db6f7e1c c150518c
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.052101] 7e20: db6f7e50 c150518c ffffffff 00000001 c150518c c01d6f6c db6f7e94 db6f7e48
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.060318] 7e40: c01d6f6c c01d6c98 c017a7a8 c0177478 00000001 db6f7e50 01ad5000 db5ee000
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.068536] 7e60: 00000cca 00040922 db744d00 00040922 00000001 c150518c 00000001 c01c8194
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.076753] 7e80: db6f7ed4 db6f7e90 c01c8194 c01c8194 db6f7ee4 db6f7ea0 c01c8194 c0126a10
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.084970] 7ea0: 29ff4a66 00d17a3c a60314b4 c02f3e6c 00012d6c 00000000 db6f7f00 c150518c
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.093187] 7ec0: db6f7f60 00000000 00000000 00000000 db6f7f4c db6f7ee8 c02f3e6c c01bcc48
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.101404] 7ee0: 00012d6c 00000000 323886e2 0000008e 00000000 00000000 00004e20 00000107
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.109621] 7f00: 00012d6c 00000000 3238d502 00d17a3c 00000000 00000000 00004e20 00040922
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:56 UTC):
 
kernel[623]: [77166.117839] 7f20: c0109124 b6a8dd40 c150518c 00000000 00000000 00000000 00000000 db6f7f60
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:57 UTC):
 
kernel[623]: [77166.126056] 7f40: db6f7fa4 db6f7f50 c02f4b48 c02f456c db6f7f60 db3090c0 00000000 00000014
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:57 UTC):
 
kernel[623]: [77166.134272] 7f60: 00012d6c 00000000 3238d502 00d17a3c db6f7fa4 00040922 c010ba84 b6a8dd40
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:57 UTC):
 
kernel[623]: [77166.142489] 7f80: 0035cfa1 0000462e 0000008e c0109124 db6f6000 00000020 00000000 db6f7fa8
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:57 UTC):
 
kernel[623]: [77166.150705] 7fa0: c0109100 c02f4a4c b6a8dd40 0035cfa1 00000000 00000000 00000000 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:57 UTC):
 
kernel[623]: [77166.158923] 7fc0: b6a8dd40 0035cfa1 0000462e 0000008e 00000014 00000000 11e005a0 00002710
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:57 UTC):
 
kernel[623]: [77166.167141] 7fe0: 0000462e b6a8dd3c 00084324 00084348 800f0010 00000000 9fde6861 9fde6c61
 
 
Broadcast message from systemd-journald@beaglebone (Sat 2017-11-04 19:44:57 UTC):
 
kernel[623]: [77166.198910] Code: e1a0200d e3c22d7f e3c2203f e592200c (e59224b4)
 
[194778.007074] audit_printk_skb: 249 callbacks suppressed
[194778.014876] audit: type=1325 audit(1509942306.899:33): table=nat family=2 entries=13
[194778.032796] audit: type=1300 audit(1509942306.899:33): arch=40000028 syscall=294 per=800000 success=yes exit=0 a0=4 a1=0 a2=40 a3=1dfdb28 items=0 ppid=1269 pid=1290 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" key=(null)
[194778.061950] audit: type=1327 audit(1509942306.899:33): proctitle=2F7573722F7362696E2F69707461626C6573002D2D77616974002D74006E6174002D4400505245524F5554494E47002D6D006164647274797065002D2D6473742D74797065004C4F43414C002D6A00444F434B4552
[194778.105862] audit: type=1325 audit(1509942306.995:34): table=nat family=2 entries=12
[194778.115026] audit: type=1300 audit(1509942306.995:34): arch=40000028 syscall=294 per=800000 success=yes exit=0 a0=4 a1=0 a2=40 a3=8f5978 items=0 ppid=1269 pid=1291 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" key=(null)
[194778.156871] audit: type=1327 audit(1509942306.995:34): proctitle=2F7573722F7362696E2F69707461626C6573002D2D77616974002D74006E6174002D44004F5554505554002D6D006164647274797065002D2D6473742D74797065004C4F43414C0000002D2D647374003132372E302E302E302F38002D6A00444F434B4552
[194778.245310] audit: type=1325 audit(1509942307.135:35): table=nat family=2 entries=11
[194778.254081] audit: type=1300 audit(1509942307.135:35): arch=40000028 syscall=294 per=800000 success=yes exit=0 a0=4 a1=0 a2=40 a3=dd3e20 items=0 ppid=1269 pid=1295 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-multi" key=(null)
[194778.286673] audit: type=1327 audit(1509942307.135:35): proctitle=2F7573722F7362696E2F69707461626C6573002D2D77616974002D74006E6174002D4600444F434B4552
[194778.317600] audit: type=1325 audit(1509942307.207:36): table=nat family=2 entries=8
[194779.702287] Unable to handle kernel NULL pointer dereference at virtual address 0000000c
[194779.717659] pgd = dc6bc000
[194779.720493] [0000000c] *pgd=9c689831, *pte=00000000, *ppte=00000000
[194779.736166] Internal error: Oops: 17 [#2] PREEMPT SMP ARM
[194779.741684] Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 xfrm_user xfrm_algo br_netfilter pru_rproc pruss_intc pruss omap_aes_driver crypto_engine omap_sham pruss_soc_bus nls_ascii nls_cp437 omap_rng rng_core evdev tps65217_charger uio_pdrv_genirq uio sch_fq_codel
[194779.766479] CPU: 0 PID: 1270 Comm: dockerd Tainted: G      D         4.9.34+ #2
[194779.773908] Hardware name: Generic AM33XX (Flattened Device Tree)
[194779.780117] task: db74d040 task.stack: db6b8000
[194779.784771] PC is at do_select+0x84/0x6e0
[194779.788887] LR is at do_select+0x6c/0x6e0
[194779.793003] pc : [<c02f3f04>]    lr : [<c02f3eec>]    psr: 600f0013
[194779.793003] sp : db6b9ae0  ip : db6b9ae0  fp : db6b9dec
[194779.804706] r10: db6b9e1c  r9 : 00000000  r8 : db6b9e1c
[194779.810039] r7 : 00000000  r6 : db6b9e04  r5 : 00000000  r4 : 00000000
[194779.816682] r3 : 00000000  r2 : 00000000  r1 : db6b9e04  r0 : 00000000
[194779.823327] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[194779.830581] Control: 10c5387d  Table: 9c6bc019  DAC: 00000051
[194779.836439] Process dockerd (pid: 1270, stack limit = 0xdb6b8218)
[194779.842646] Stack: (0xdb6b9ae0 to 0xdb6ba000)
[194779.847116] 9ae0: c15051bc 3b9aca00 0000c350 00000001 00000000 c150518c c15073fc db6b8000
[194779.855421] 9b00: db6b9b58 db6b9f60 db6b9e04 00000000 00004408 00000000 db6b9b64 db6b9b28
[194779.863726] 9b20: c017a7a8 c0175ad4 00004408 db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e1c
[194779.872030] 9b40: db6b9e1c 00000000 df933178 c017aa3c 00000400 00000000 2f1b55c8 0000b126
[194779.880334] 9b60: 00000000 ffffffff 00000000 db74d040 00000000 00000000 00000000 db6b9b88
[194779.888639] 9b80: c017c3c4 c017a990 00000000 c068aa9c c14b6a4c 00000001 db6b8000 00000102
[194779.896945] 9ba0: db6b9bbc db6b9bb0 c14af498 c152d520 02fb649a 00000000 db74d0c0 c0193788
[194779.905249] 9bc0: 00000800 c011d414 c150518c c01632a0 c163bbb0 c150518c c14af498 c152d520
[194779.913553] 9be0: 00007d2a c017a7a8 dc688200 df933178 00007d2a 00000000 00007d2a 00000000
[194779.921858] 9c00: db6b9c4c db6b9c10 c017a7a8 c0175ad4 00007d2a 00000000 00007d2a 00000000
[194779.930162] 9c20: db6b9c6c 00000400 dc688200 ffd23940 ffffffff 6d3af8cd db6b9c6c db6b9c48
[194779.938467] 9c40: c01770a8 c06936f8 dc6b4600 00000000 03983a68 00000000 0000001f 00000001
[194779.946772] 9c60: db6b9cfc db6b9c70 c017f140 c0177028 dc688200 00000000 02a19025 00000000
[194779.955076] 9c80: 00000000 00000000 aea787ef 0000b126 00000001 00000000 db74d0c0 00000000
[194779.963380] 9ca0: 00000207 00000000 00000000 00000000 aecc9d19 0000b126 00000001 db74d0c0
[194779.971684] 9cc0: 00000800 0000b126 0000001f 00000000 db6b8000 00000001 dc6b4600 c0180a50
[194779.979990] 9ce0: df933178 00000000 02a3f825 c0180224 db6b9d9c db6b9d00 c0180224 c0177198
[194779.988294] 9d00: db6b9d24 db6b9d10 c011aef0 c011ad60 df933140 c017ad60 db6b9d8c db6b9d28
[194779.996598] 9d20: db6b9d6c db6b9d30 c01d37ec db309840 db74d0c0 db74d040 df933140 c017ad74
[194780.004902] 9d40: db309840 db74d0c0 db6b9d84 db6b9d58 c017ad74 c0175ee4 c017ac94 db74d040
[194780.013207] 9d60: df933140 df933140 a00f0013 00000000 00000000 df933140 db6b9d9c db6b9d88
[194780.021511] 9d80: c016cffc c017aca0 db3097c0 db3097c0 db6b9dc4 db6b9da0 c016d024 c016cf70
[194780.029816] 9da0: db3097c0 df933140 00000000 a00f0013 00000000 00000000 db6b9de4 00040922
[194780.038121] 9dc0: 00000001 00000000 db6b9e1c c150518c 00000000 db6b9e1c 00000000 db6b9e1c
[194780.046425] 9de0: db6b9f4c db6b9df0 c02f4778 c02f3e8c db6b9e14 00000000 db6b9f60 00000000
[194780.054730] 9e00: 00000001 db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e28
[194780.063035] 9e20: c016f35c c016ef50 c150518c db6b9e50 00000001 c01d6f6c db6b9e94 db6b9e48
[194780.071340] 9e40: c01d6f6c c01d6c98 dc688200 dc1f6e44 db309ce0 db309ce0 11e21000 db5ee8c0
[194780.079645] 9e60: 0000022e 00040922 c0177478 00040922 00000001 c150518c 00000001 c01c8194
[194780.087949] 9e80: db6b9ed4 db6b9e90 c01c8194 c01c8194 db6b9ee4 db6b9ea0 c01c8194 c0126a10
[194780.096255] 9ea0: 89113c17 008df4f8 a60314b4 c02f3e6c 0002f8d9 00000000 db6b9f00 c150518c
[194780.104560] 9ec0: db6b9f60 00000000 00000000 00000000 db6b9f4c db6b9ee8 c02f3e6c c01bcc48
[194780.112864] 9ee0: 0002f8d9 00000000 21377bfe 0000008e 00000000 00000000 00004e20 00000107
[194780.121168] 9f00: 0002f8d9 00000000 2137ca1e 008df4f8 00000000 00000000 00004e20 00040922
[194780.129472] 9f20: c0109124 b6a11d40 c150518c 00000000 00000000 00000000 00000000 db6b9f60
[194780.137778] 9f40: db6b9fa4 db6b9f50 c02f4b48 c02f456c db6b9f60 db74d040 00000000 00000014
[194780.146083] 9f60: 0002f8d9 00000000 2137ca1e 008df4f8 db6b9fa4 00040922 c010ba84 b6a11d40
[194780.154387] 9f80: 0035cfa1 0000b126 0000008e c0109124 db6b8000 00000020 00000000 db6b9fa8
[194780.162692] 9fa0: c0109100 c02f4a4c b6a11d40 0035cfa1 00000000 00000000 00000000 00000000
[194780.170996] 9fc0: b6a11d40 0035cfa1 0000b126 0000008e 00000014 00000000 11e005a0 00002710
[194780.179301] 9fe0: 0000b126 b6a11d3c 00084324 00084348 800f0010 00000000 00000000 00000000
[194780.187623] [<c02f3f04>] (do_select) from [<c02f4778>] (core_sys_select+0x218/0x4e0)
[194780.195494] [<c02f4778>] (core_sys_select) from [<c02f4b48>] (SyS_select+0x108/0x178)
[194780.203459] [<c02f4b48>] (SyS_select) from [<c0109100>] (__sys_trace_return+0x0/0x10)
[194780.211419] Code: e1f03513 e1a0200d e3c22d7f e3c2203f (e592200c)
[194780.228190] ---[ end trace 592206016ddd67d8 ]---
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.736166] Internal error: Oops: 17 [#2] PREEMPT SMP ARM
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.836439] Process dockerd (pid: 1270, stack limit = 0xdb6b8218)
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.842646] Stack: (0xdb6b9ae0 to 0xdb6ba000)
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.847116] 9ae0: c15051bc 3b9aca00 0000c350 00000001 00000000 c150518c c15073fc db6b8000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.855421] 9b00: db6b9b58 db6b9f60 db6b9e04 00000000 00004408 00000000 db6b9b64 db6b9b28
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.863726] 9b20: c017a7a8 c0175ad4 00004408 db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e1c
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.872030] 9b40: db6b9e1c 00000000 df933178 c017aa3c 00000400 00000000 2f1b55c8 0000b126
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.880334] 9b60: 00000000 ffffffff 00000000 db74d040 00000000 00000000 00000000 db6b9b88
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.888639] 9b80: c017c3c4 c017a990 00000000 c068aa9c c14b6a4c 00000001 db6b8000 00000102
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.896945] 9ba0: db6b9bbc db6b9bb0 c14af498 c152d520 02fb649a 00000000 db74d0c0 c0193788
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.905249] 9bc0: 00000800 c011d414 c150518c c01632a0 c163bbb0 c150518c c14af498 c152d520
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.913553] 9be0: 00007d2a c017a7a8 dc688200 df933178 00007d2a 00000000 00007d2a 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.921858] 9c00: db6b9c4c db6b9c10 c017a7a8 c0175ad4 00007d2a 00000000 00007d2a 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.930162] 9c20: db6b9c6c 00000400 dc688200 ffd23940 ffffffff 6d3af8cd db6b9c6c db6b9c48
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.938467] 9c40: c01770a8 c06936f8 dc6b4600 00000000 03983a68 00000000 0000001f 00000001
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.946772] 9c60: db6b9cfc db6b9c70 c017f140 c0177028 dc688200 00000000 02a19025 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.955076] 9c80: 00000000 00000000 aea787ef 0000b126 00000001 00000000 db74d0c0 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.963380] 9ca0: 00000207 00000000 00000000 00000000 aecc9d19 0000b126 00000001 db74d0c0
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.971684] 9cc0: 00000800 0000b126 0000001f 00000000 db6b8000 00000001 dc6b4600 c0180a50
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.979990] 9ce0: df933178 00000000 02a3f825 c0180224 db6b9d9c db6b9d00 c0180224 c0177198
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.988294] 9d00: db6b9d24 db6b9d10 c011aef0 c011ad60 df933140 c017ad60 db6b9d8c db6b9d28
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194779.996598] 9d20: db6b9d6c db6b9d30 c01d37ec db309840 db74d0c0 db74d040 df933140 c017ad74
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.004902] 9d40: db309840 db74d0c0 db6b9d84 db6b9d58 c017ad74 c0175ee4 c017ac94 db74d040
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.013207] 9d60: df933140 df933140 a00f0013 00000000 00000000 df933140 db6b9d9c db6b9d88
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.021511] 9d80: c016cffc c017aca0 db3097c0 db3097c0 db6b9dc4 db6b9da0 c016d024 c016cf70
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.029816] 9da0: db3097c0 df933140 00000000 a00f0013 00000000 00000000 db6b9de4 00040922
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.038121] 9dc0: 00000001 00000000 db6b9e1c c150518c 00000000 db6b9e1c 00000000 db6b9e1c
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.046425] 9de0: db6b9f4c db6b9df0 c02f4778 c02f3e8c db6b9e14 00000000 db6b9f60 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.054730] 9e00: 00000001 db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e1c db6b9e28
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.063035] 9e20: c016f35c c016ef50 c150518c db6b9e50 00000001 c01d6f6c db6b9e94 db6b9e48
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.071340] 9e40: c01d6f6c c01d6c98 dc688200 dc1f6e44 db309ce0 db309ce0 11e21000 db5ee8c0
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.079645] 9e60: 0000022e 00040922 c0177478 00040922 00000001 c150518c 00000001 c01c8194
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.087949] 9e80: db6b9ed4 db6b9e90 c01c8194 c01c8194 db6b9ee4 db6b9ea0 c01c8194 c0126a10
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.096255] 9ea0: 89113c17 008df4f8 a60314b4 c02f3e6c 0002f8d9 00000000 db6b9f00 c150518c
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.104560] 9ec0: db6b9f60 00000000 00000000 00000000 db6b9f4c db6b9ee8 c02f3e6c c01bcc48
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.112864] 9ee0: 0002f8d9 00000000 21377bfe 0000008e 00000000 00000000 00004e20 00000107
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.121168] 9f00: 0002f8d9 00000000 2137ca1e 008df4f8 00000000 00000000 00004e20 00040922
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.129472] 9f20: c0109124 b6a11d40 c150518c 00000000 00000000 00000000 00000000 db6b9f60
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.137778] 9f40: db6b9fa4 db6b9f50 c02f4b48 c02f456c db6b9f60 db74d040 00000000 00000014
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.146083] 9f60: 0002f8d9 00000000 2137ca1e 008df4f8 db6b9fa4 00040922 c010ba84 b6a11d40
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.154387] 9f80: 0035cfa1 0000b126 0000008e c0109124 db6b8000 00000020 00000000 db6b9fa8
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.162692] 9fa0: c0109100 c02f4a4c b6a11d40 0035cfa1 00000000 00000000 00000000 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.170996] 9fc0: b6a11d40 0035cfa1 0000b126 0000008e 00000014 00000000 11e005a0 00002710
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.179301] 9fe0: 0000b126 b6a11d3c 00084324 00084348 800f0010 00000000 00000000 00000000
 
 
Broadcast message from systemd-journald@beaglebone (Mon 2017-11-06 04:25:09 UTC):
 
kernel[623]: [194780.211419] Code: e1f03513 e1a0200d e3c22d7f e3c2203f (e592200c)
2

hey @SuicidalLabRat,

What version of resinOS (unmanaged) are you running on the BBG? also does this kernel panic happen when no containers are running on the device?

3

Sorry, my bad.

cat issue
Resin OS 2.2.0

Linux version 4.9.34+ (builder@ce44078a32a1) (gcc version 6.2.0 (GCC) ) #2 SMP PREEMPT Mon Jul 31 16:01:09 UTC 2017

There are no containers running.

The panic can happen spontaneously, but it seems I can also trigger it by running something as simple as docker info.

Thanks!
SLR-

4

@shaunmulligan I believe I have also see a stack dump to dmesg as well; where the system recovers gracefully and goes on to finish booting.

5

Hey @SuicidalLabRat that is pretty strange. I have a BBG here, so I will try reproduce the issue and let you know what I find.

6

Hey @SuicidalLabRat I set up unmanaged resinOS 2.2.0 on my BBG and haven’t been able to get it to panic yet. One thing that I might have done differently to you is that I downloaded the BeagleBone Black image rather than the BBG specific one.The reason I did this is because we have unified all our beaglebone images into this one image, so it has everything needed to boot on Beaglebone Green/black/blue, etc. It might be the case that there is something messed up on the BBG specific image (we are actually in the process of deprecating them) so I would suggest reflashing the device with the Beaglebone Black image and seeing if the panic persists there.

7

Thanks for the prompt response @shaunmulligan!
Im not working from a downloaded image. I am not using an image from the Beaglebone download page, which are all bbb images, despite the link names. Im doing a recursive of https://github.com/resin-os/resin-beaglebone.git and building resin-image-flasher.

Are you suggesting I change the MACHINE from “beaglebone-green” to “beaglebone”?

SLR-

8

@SuicidalLabRat oh, if you are building from source, I would definitely recommend switching the MACHINE name to “beaglebone”

9

Running a clean build now…