Raspberry PI resets every ~40 seconds if offline

Product support
1

I decided to test how my raspberry pi behaves when offline, so I unplugged ethernet cable from router (leaving wifi network on) and was frustrated by the fact that it reboots every 40 seconds or so. And being operational offline is crucial to my application. Here’s a part of journal that is hopefully informative (I’ve stripped private data):

Aug 02 06:47:47 76138b8 dockerd[968]:     at processImmediate [as _immediateCallback] (timers.js:533:5)
Aug 02 06:47:48 76138b8 dockerd[968]: Wed, 02 Aug 2017 06:47:48 GMT body-parser deprecated bodyParser: use individual json/urlencoded middlewares at usr/src/app/src/proxyvisor.js:45:14
Aug 02 06:47:48 76138b8 start-resin-supervisor[1252]: Wed, 02 Aug 2017 06:47:48 GMT body-parser deprecated bodyParser: use individual json/urlencoded middlewares at usr/src/app/src/proxyvisor.js:45:14
Aug 02 06:47:48 76138b8 start-resin-supervisor[1252]: Wed, 02 Aug 2017 06:47:48 GMT body-parser deprecated undefined extended: provide extended option at usr/src/app/node_modules/body-parser/index.js:105:29
Aug 02 06:47:48 76138b8 dockerd[968]: Wed, 02 Aug 2017 06:47:48 GMT body-parser deprecated undefined extended: provide extended option at usr/src/app/node_modules/body-parser/index.js:105:29
Aug 02 06:47:48 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:47:48.843Z] Starting API server..
Aug 02 06:47:48 76138b8 dockerd[968]: [2017-08-02T06:47:48.843Z] Starting API server..
Aug 02 06:47:48 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:47:48.849Z] Starting Apps..
Aug 02 06:47:48 76138b8 dockerd[968]: [2017-08-02T06:47:48.849Z] Starting Apps..
Aug 02 06:47:49 76138b8 dockerd[968]: [2017-08-02T06:47:49.169Z] Starting periodic check for IP addresses..
Aug 02 06:47:49 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:47:49.169Z] Starting periodic check for IP addresses..
Aug 02 06:47:49 76138b8 dockerd[968]: [2017-08-02T06:47:49.194Z] Updating supervisor version and api info
Aug 02 06:47:49 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:47:49.194Z] Updating supervisor version and api info
Aug 02 06:47:49 76138b8 start-resin-supervisor[1252]: Wed, 02 Aug 2017 06:47:49 GMT body-parser deprecated bodyParser: use individual json/urlencoded middlewares at usr/src/app/src/api.js:28:22
Aug 02 06:47:49 76138b8 dockerd[968]: Wed, 02 Aug 2017 06:47:49 GMT body-parser deprecated bodyParser: use individual json/urlencoded middlewares at usr/src/app/src/api.js:28:22
Aug 02 06:47:57 76138b8 dockerd[968]: [2017-08-02T06:47:57.493Z] Waiting for connectivity...
Aug 02 06:47:57 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:47:57.493Z] Waiting for connectivity...
Aug 02 06:48:04 76138b8 dockerd[968]: [58B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [121B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [120B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [67B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [68B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [48B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [37B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [39B blob data]
Aug 02 06:48:08 76138b8 dockerd[968]: [2017-08-02T06:48:08.737Z] Event: Application start {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:48:08 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:48:08.737Z] Event: Application start {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:48:08 76138b8 dockerd[968]: time="2017-08-02T06:48:08.801689766Z" level=error msg="Handler for POST /containers/<truncated>/start returned error: Container already started"
Aug 02 06:48:13 76138b8 dockerd[968]: [12B blob data]
Aug 02 06:48:13 76138b8 systemd-udevd[29]: starting version 215
Aug 02 06:48:13 76138b8 kernel[663]: [   76.271580] systemd-udevd[29]: starting version 215
Aug 02 06:48:13 76138b8 systemd-udevd[1593]: Process '/usr/sbin/crda' failed with exit code 234.
Aug 02 06:48:16 76138b8 dockerd[968]: [1B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [43B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [54B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [54B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [38B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [56B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [299B blob data]
Aug 02 06:48:16 76138b8 dockerd[968]: [58B blob data]
Aug 02 06:48:17 76138b8 openvpn[926]: RESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname
Aug 02 06:48:17 76138b8 ModemManager[608]: <info>  Couldn't check support for device at '/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.1': not supported by any plugin
Aug 02 06:48:27 76138b8 dockerd[968]: [2017-08-02T06:48:27.644Z] Event: Device info update failure {"error":{"message":"getaddrinfo EAI_AGAIN api.resin.io:443","stack":"Error: getaddrinfo EAI_AGAIN api.resin.io:443\n    at Object.exports._errnoException (util.js:1026:11)\n    at errnoException (dns.js:33:15)\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:76:26)"},"stateDiff":{"api_port":48484,"api_secret":"c7f379ba1e9b9050004fcf038614ef58e9ce1827b6cd2d09d79d3e9491389278","os_version":"Resin OS 2.0.8+rev1","os_variant":"prod","supervisor_version":"5.1.0","provisioning_progress":null,"provisioning_state":"","download_progress":null,"logs_channel":"d2966122fd57f21c4677a84c43a984c381dad17b2f59c3835c5085b422052307"}}
Aug 02 06:48:27 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:48:27.644Z] Event: Device info update failure {"error":{"message":"getaddrinfo EAI_AGAIN api.resin.io:443","stack":"Error: getaddrinfo EAI_AGAIN api.resin.io:443\n    at Object.exports._errnoException (util.js:1026:11)\n    at errnoException (dns.js:33:15)\n    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:76:26)"},"stateDiff":{"api_port":48484,"api_secret":"c7f379ba1e9b9050004fcf038614ef58e9ce1827b6cd2d09d79d3e9491389278","os_version":"Resin OS 2.0.8+rev1","os_variant":"prod","supervisor_version":"5.1.0","provisioning_progress":null,"provisioning_state":"","download_progress":null,"logs_channel":"d2966122fd57f21c4677a84c43a984c381dad17b2f59c3835c5085b422052307"}}
Aug 02 06:48:28 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:48:28.863Z] Event: Application already running {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:48:28 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:48:28.870Z] Event: Start application update poll {"interval":60000}
Aug 02 06:48:28 76138b8 dockerd[968]: [2017-08-02T06:48:28.863Z] Event: Application already running {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:48:28 76138b8 dockerd[968]: [2017-08-02T06:48:28.870Z] Event: Start application update poll {"interval":60000}
Aug 02 06:48:37 76138b8 dockerd[968]: [58B blob data]
Aug 02 06:48:41 76138b8 dockerd[968]: [121B blob data]
Aug 02 06:48:41 76138b8 dockerd[968]: [120B blob data]
Aug 02 06:48:41 76138b8 dockerd[968]: [67B blob data]
Aug 02 06:48:41 76138b8 dockerd[968]: [68B blob data]
Aug 02 06:48:41 76138b8 dockerd[968]: [48B blob data]
Aug 02 06:48:41 76138b8 dockerd[968]: [37B blob data]
Aug 02 06:48:41 76138b8 dockerd[968]: [39B blob data]
Aug 02 06:48:44 76138b8 dockerd[968]: [12B blob data]
Aug 02 06:48:44 76138b8 systemd-udevd[29]: starting version 215
Aug 02 06:48:44 76138b8 kernel[663]: [  106.703963] systemd-udevd[29]: starting version 215
Aug 02 06:48:44 76138b8 systemd-udevd[1761]: Process '/usr/sbin/crda' failed with exit code 234.
Aug 02 06:48:44 76138b8 ModemManager[608]: <info>  (net/eth0): released by modem /sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.1
Aug 02 06:48:47 76138b8 dockerd[968]: [1B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [43B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [54B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [54B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [38B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [56B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [299B blob data]
Aug 02 06:48:47 76138b8 dockerd[968]: [58B blob data]
Aug 02 06:48:47 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:48:47.665Z] Event: Application exit {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:48:47 76138b8 dockerd[968]: [2017-08-02T06:48:47.665Z] Event: Application exit {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:48:47 76138b8 ModemManager[608]: <info>  Couldn't check support for device at '/sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.1': not supported by any plugin
Aug 02 06:49:02 76138b8 openvpn[926]: RESOLVE: Cannot resolve host address: vpn.resin.io: No address associated with hostname
Aug 02 06:49:07 76138b8 start-resin-supervisor[1252]: [2017-08-02T06:49:07.611Z] Event: Application restart {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:49:07 76138b8 dockerd[968]: [2017-08-02T06:49:07.611Z] Event: Application restart {"app":{"id":1,"name":"<truncated>","containerId":"<truncated>","commit":"<truncated>","imageId":"registry2.resin.io/<truncated>/<truncated>","appId":"<truncated>","privileged":null,"env":"{\"RESIN_APP_ID\":\"<truncated>\",\"RESIN_APP_NAME\":\"<truncated>\",\"RESIN_APP_RELEASE\":\"<truncated>\",\"RESIN_DEVICE_UUID\":\"<truncated>\",\"RESIN_DEVICE_NAME_AT_INIT\":\"<truncated>\",\"RESIN_DEVICE_TYPE\":\"raspberrypi3\",\"RESIN_HOST_OS_VERSION\":\"Resin OS 2.0.8+rev1\",\"RESIN_SUPERVISOR_ADDRESS\":\"http://127.0.0.1:48484\",\"RESIN_SUPERVISOR_HOST\":\"127.0.0.1\",\"RESIN_SUPERVISOR_PORT\":48484,\"RESIN_SUPERVISOR_VERSION\":\"5.1.0\",\"RESIN\":\"1\",\"USER\":\"root\",\"DATA_DIR\":\"/data\",\"LOG_THRESHOLD\":\"INFO\",\"MASTER_URL\":\"<truncated>\"}","config":"{\"RESIN_HOST_CONFIG_gpu_mem\":\"75\",\"RESIN_DEVICE_RESTART\":\"7409775079389485\"}"}}
Aug 02 06:49:08 76138b8 dockerd[968]: [58B blob data]
Aug 02 06:49:12 76138b8 dockerd[968]: [121B blob data]
Aug 02 06:49:12 76138b8 dockerd[968]: [120B blob data]
Aug 02 06:49:12 76138b8 dockerd[968]: [67B blob data]
Aug 02 06:49:12 76138b8 dockerd[968]: [68B blob data]
Aug 02 06:49:12 76138b8 dockerd[968]: [48B blob data]
Aug 02 06:49:12 76138b8 dockerd[968]: [37B blob data]
Aug 02 06:49:12 76138b8 dockerd[968]: [39B blob data]
Aug 02 06:49:14 76138b8 dockerd[968]: [12B blob data]
Aug 02 06:49:14 76138b8 systemd-udevd[29]: starting version 215
Aug 02 06:49:14 76138b8 kernel[663]: [  137.172719] systemd-udevd[29]: starting version 215
Aug 02 06:49:14 76138b8 systemd-udevd[1934]: Process '/usr/sbin/crda' failed with exit code 234.
Aug 02 06:49:15 76138b8 ModemManager[608]: <info>  (net/eth0): released by modem /sys/devices/platform/soc/3f980000.usb/usb1/1-1/1-1.1
Aug 02 06:49:17 76138b8 dockerd[968]: [1B blob data]
Aug 02 06:49:17 76138b8 dockerd[968]: [43B blob data]
Aug 02 06:49:17 76138b8 dockerd[968]: [54B blob data]
Aug 02 06:49:17 76138b8 dockerd[968]: [54B blob data]
Aug 02 06:49:17 76138b8 dockerd[968]: [1B blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [38B blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [56B blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
Aug 02 06:49:18 76138b8 dockerd[968]: [1.0K blob data]
2

That’s very strange, really shouldn’t happen, what resinOS version are you running on the device?

3

I am running 2.0.8 prod. Will it update to 2.2 automatically or do I have to burn the card again?

4

The devices won’t automatically update their host OS. You can burn the SD card again and reprovision the device, or we have just started beta-testing remote host OS updates (see this forum post), and can update it for you.

On the other hand, if this is a new issue and an actual bug, it’s uncertain whether upgrading the hostOS will improve on it, will dig in to see whether we can reproduce and triage what the issue is.

What kind of application are you deploying onto the device, does it interact with the network settings at all? (just trying to find out as much as possible about the context, to be able to narrow things down). Any other info that you might think is relevant?

Will try to replicate and keep you posted.

5

Thanks for caring, @imrehg. I’m quite frustrated because I came to rely on resin.io and can’t deliver the project I’ve been working on for the whole year because of this bug.

Unfortunately, upgrading to v2.2.0+rev1.prod did not solve the issue. I tried with another router, and with ethernet instead of WiFi connection. The sequence is the same:

  • connect to a router (using no matter what, wifi or ethernet)
  • unplug internet provider cable from the router
  • enjoy reseting Raspberry Pi 3 every 30 seconds.

I did not interfere with any network settings on resinOS image. My app downloads files over the net, but it doesn’t touch any settings.

When I tried 2.2prod, I didn’t even enter wifi credentials, just downloaded vanilla 2.2prod image and only put "persistentLogging": true to config.json to persist systemd logging.

I’ll try to reproduce the problem with a blank project and will keep you updated.

6

I was not able to reproduce the rebooting behaviour with a minimal “hello world” project. Now I’ll try to distill what provokes rebooting in my project.

1 Like
7

Very strange, just been trying to reproduce this, and so far cannot…

  • Flash an RPi3 with 2.2.0+rev1.dev, connect over Ethernet
  • Connect with a USB serial cable so can see the logs even when the device is unplugged
  • Start up, and follow the logs through in the console through USB serial

For me the device stays up, and just continues working, the application runs in the background (confirmed with docker logs <containerid> on the device, and continues to try to connect. It does not restart for me at all…

What exactly does it mean by “enjoy reseting Raspberry Pi 3 every 30 seconds.” - the whole device restarts every 30 seconds, or you are actually need to reset anything?

8

I see we were both checking it out. :slight_smile:

Let us know if there’s any repository we can look at to see if there’s anything standing out. It might be that if there’s no network connection, the application you deploy exits, and then it would restart the application. But it’s still not clear to me if the whole device resets, or just your application? (I’d guess the latter).

9

Can you please instruct on how to do this? I need a USB male ↔ USB male cable, and then what?

10

Not a make<->male cable, but an USB serial cable. See the details at http://elinux.org/RPi_Serial_Connection

The hardware setup is shown there. On the software side you only need screen installed on your development machine, plug in the USB serial cable and e.g. screen /dev/ttyUSB0 115200 should bring up the console for a .dev device.

11

Ahh, I didn’t have the idea in mind that any uncaught exception leads to restart of the container. While it’s a perfectly reasonable docker setting and I should have assumed that.

It was hard to figure out, because I didn’t see any of my app exceptions in the logs while offline: what I was doing to see logs was examining resin-state/root-overlay/var/log/journal folder (I invented this one-liner docker run -v $PWD:/journal -ti ubuntu journalctl -D /journal --no-pager > log.txt), and for some reason they don’t include logs my app produces from inside container. So I had no clue.

From now on, I’ll try to handle most of the exceptions (it’s not too hard). Sorry for the false alarm on resinOS bug.

BTW, don’t you know why systemd logs left on SD card don’t include container logs? There are some system* files and some user* files, but journalctl -D shows no entries regarding my app, just system stuff.

And thanks for clarification on how to connect to raspberry via USB, I have ordered USB TTL cable.

12

That’s a good question, will be checking with the team, thanks for the feedback!

In the meantime, you can get logs by running docker logs <containerid> for example, that’s what I used on a .dev device when debugging your application.

For the application restart, there’s actually a setting that governs that, and implement’s Docker’s restart policies. You can set a configurations variable RESIN_SUPERVISOR_RESTART_POLICY.

Here are the options for the policies and their effects (copied from Docker, but also added some notes)

  • no: Do not automatically restart the container when it exits.
  • on-failure[:max-retries]: Restart only if the container exits with a non-zero exit status. Optionally, limit the number of restart retries the Docker daemon attempts (the max retries can be set with the RESIN_APP_RESTART_RETRIES config variable)
  • always: Always restart the container regardless of the exit status. When you specify always, the Docker daemon will try to restart the container indefinitely. The container will also always start on daemon startup, regardless of the current state of the container. (as opposed to docker, this is resin’s default!)
  • unless-stopped: Always restart the container regardless of the exit status, but do not start it on daemon startup if the container has been put to a stopped state before.

(this we need to document better, taking care of that too)

14

Container runtime page claims that

One option to keep your containers running is to enable the INITSYSTEM in your container. This can easily be done by creating a device environment variable called INITSYSTEM and setting its value to on.

Does it mean that uncaught exceptions should not trigger container reboot when systemd is enabled? I have had INITSYSTEM=on in my Dockerfile, and assumed that kind of a behaviour.

15

A question, are you using the resin-made base images? Because the INITSYSTEM variable only has effect on our own base images, obviously not for the external ones. From the code you shared with us seems you are using an upstream image, not a resin-provided one.

I’ve tried it out, and you are correct, that with INITSYSTEM=on, application exiting won’t stop the container. :+1: So based on that and what you mentioned, I’d probably set up your application’s start with a systemd system file, and then you can set your own restart policy as well easily, and so on.

One side note, there’s an open issue for Debian Stretch using the INITSYSTEM settings, so that won’t work at the momemnt, but we’re fixing it. Jessie works correctly, though, in case.

16

In my code I have FROM resin/%%RESIN_MACHINE_NAME%%-debian in Dockerfile.template and FROM debian:9.1 in Dockerfile. I can guess, Dockerfile.template takes precedence when I git push resin. I use Dockerfile to try my code on non-arm systems.

So, with FROM resin/%%RESIN_MACHINE_NAME%%-debian in Dockerfile.template containing INITSYSTEM=on, container shouldn’t restart on every exception, right? But it does. It’s something I don’t understand.

17

We’ve checked it out, and the logs include the application logs. You have to run journalctl -a on the log file, because of some quirks regarding how docker saves logs (without -a it just shows [10B blob data] or similar). Your application logs are under the header of docker. So if you get out the persistent log’s systemd journal, then run journalctl --file=system.journal -a on it, and should see your application’s messages too.

As for your last comment, just double-checking it…

1 Like
18

Oh, I got it, just docker logs <container id>.

19

Yeah, both mentioned in this thread and works, one is mixing in the total logs, and the other’s using docker directly. Hope this will be helpful :slight_smile:

1 Like