Incude script in initial image

mexekanez · September 30, 2019, 10:27am

We need to be able to build an image which could be flashed on the USB. This image would contain a file called " auto_run.py " or " auto_run.sh ".

The idea is: as soon as the USB is connected to the machine (Edge device), the script should run automatically. This script would perform an operation like making an API call, storing some logs, repairing an API and etc

dansku · September 30, 2019, 11:32am

Hey @mexekanez,
you must first provision your device with balenaOS, then the auto_run script should run inside a container.

You can also check the SDK, which you can use to control the API from the device. https://github.com/balena-io/balena-sdk

Let me know if you have ay questions.

Cheers

mexekanez · September 30, 2019, 1:05pm

@dansku Hi!
No I don’t want to wait while device will pull updates. I want that script is in image before sd burn. and will run after usb connected

dansku · September 30, 2019, 2:01pm

Hey @mexekanez, one thing that you can test is the balena-preload project => https://github.com/balena-io/balena-preload

Take a look at the repository and see if that would help with your requirements.

Cheers

mexekanez · September 30, 2019, 2:11pm

yes I understood that I can use only preload OR build yocto image.

with preload I have an issue (using openBalena):

 balena preload balena.img --app 12 --commit 9bde88b0e12e4b34228ace17a71d45ab
Building Docker preloader image. [========================] 100%



/ Creating preloader container
- Starting preloader container
\ Fetching application 12
| Reading image information
/ Resizing partitions and waiting for dockerd to start
| Cleaning up temporary files
(HTTP code 500) server error - Get https://registry.DOMAIN.xyz/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 

Additional information may be available by setting a DEBUG=1 environment
variable: "set DEBUG=1" on a Windows command prompt, or "export DEBUG=1"
on Linux or macOS.

If you need help, don't hesitate in contacting our support forums at
https://forums.balena.io

For bug reports or feature requests, have a look at the GitHub issues or
create a new one at: https://github.com/balena-io/balena-cli/issues/

jviotti · September 30, 2019, 2:16pm

Hey there! Is the error message actually printing “https://registry.DOMAIN.xyz/v2/”? If not, what is the actual domain?

mexekanez · September 30, 2019, 2:28pm

hi!
actual domain:
https://registry.visoagent.xyz/v2/
as I said I using openBalena

richbayliss · September 30, 2019, 3:10pm

@mexekanez I tried to look at the domain you listed, and I only so the registry exposed on HTTP and not HTTPS. This would explain the issue, since the preload script it trying to use HTTPS and it reporting a timeout before receiving headers; which would happen if the TLS layer couldn’t be established correctly.

For openBalena I think you should have HTTPS going to your registry container, so I am not sure what setup you have but it looks like something isn’t configured correctly. By default openBalena will expose port 443 to the haproxy container, which reverse proxies the request to the registry container. I would start by looking there to see if you have something misconfigured

mexekanez · October 1, 2019, 8:10am

hi @richbayliss
not sure what you mean
bot domain working well http and https
http://registry.visoagent.xyz/v2/
https://registry.visoagent.xyz/v2/

mexekanez · October 1, 2019, 8:25am

actually I have the same issue with balena-preload

mexekanez@mexekanezlaptop:~$ balena-preload --app 12 --commit 123455667  --api-token 24234234 --img balena.img
Building Docker preloader image. [========================] 100%



/ Creating preloader container
- Starting preloader container
\ Fetching application 12
/ Reading image information
/ Resizing partitions and waiting for dockerd to start
/ Cleaning up temporary files

{ Error: (HTTP code 500) server error - Get https://registry.visoagent.xyz/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 
    at /usr/local/lib/node_modules/balena-preload/node_modules/docker-modem/lib/modem.js:257:17
    at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/balena-preload/node_modules/docker-modem/lib/modem.js:284:9)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9) reason: 'server error', statusCode: 500, json: null }

Looks like this might be an issue with balena-preload. Please report it at https://github.com/balena-io/balena-preload/issues

mexekanez · October 1, 2019, 9:11am

/ Creating preloader container

Starting preloader container
\ Fetching application 12
| Reading image information
/ Resizing partitions and waiting for dockerd to start
/ Cleaning up temporary files

{ Error: (HTTP code 500) server error - Get https://registry.visoagent.xyz/v2/v2/44acbabc9d364cf2263b1840f099ca43/manifests/sha256:81247b1f61a6bce4a3b2dca3a2fa6ac4fd78144d5421787fb32216bbfab52bc2: dial tcp: lookup registry.visoagent.xyz on 192.168.0.1:53: read udp 192.168.0.103:35298->192.168.0.1:53: i/o timeout
at /usr/local/lib/node_modules/balena-preload/node_modules/docker-modem/lib/modem.js:257:17
at IncomingMessage. (/usr/local/lib/node_modules/balena-preload/node_modules/docker-modem/lib/modem.js:284:9)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickCallback (internal/process/next_tick.js:180:9) reason: ‘server error’, statusCode: 500, json: null }

Looks like this might be an issue with balena-preload. Please report it at Issues · balena-io-modules/balena-preload · GitHub
mexekanez@mexekanezlaptop:~$ sudo ufw status
[sudo] password for mexekanez:
Status: inactive

My laptop IP: 192.168.0.103

running preload command on it

where is an issue?

I have all ports opened on openBalenaOS

mexekanez · October 1, 2019, 11:51am

@here any thoughts?

afitzek · October 1, 2019, 12:16pm

Hi,
Could you try to run balena-preload from your laptop with NODE_EXTRA_CA_CERTS=~/open-balena/config/certs/root/ca.crt. Maybe the problem is that balena-preload cannot verify the self signed CA certificate.

mexekanez · October 1, 2019, 4:47pm

nope, doesn’t works @afitzek

mexekanez · October 1, 2019, 5:09pm

where I can check logs in OpenBalena?

robertgzr · October 1, 2019, 5:19pm

How are you running it? Did you follow the instructions here: https://www.balena.io/open/docs/getting-started/?

mexekanez · October 1, 2019, 5:21pm

yes.

also I have the same issue with balena cloud:

mexekanez@mexekanezlaptop:~$ DEBUG=1 balena-preload --commit 189631023ad438125eab7e6f6f25ab41096fad9e  --app 1504381  --img ./balena.img  --api-token 123456
Building Docker preloader image. [===                     ] 12%
Step 1/7 : FROM docker:17.10.0-ce-dind
Building Docker preloader image. [======                  ] 25%
Step 2/7 : RUN apk update && apk add --no-cache python3 parted btrfs-progs docker util-linux sfdisk file coreutils sgdisk
 ---> Using cache
Building Docker preloader image. [=========               ] 37%
Step 3/7 : COPY ./requirements.txt /tmp/
 ---> Using cache
Building Docker preloader image. [============            ] 50%
Step 4/7 : RUN pip3 install -r /tmp/requirements.txt
 ---> Using cache
Building Docker preloader image. [===============         ] 62%
Step 5/7 : COPY ./src /usr/src/app
 ---> Using cache
Building Docker preloader image. [==================      ] 75%
Step 6/7 : WORKDIR /usr/src/app
 ---> Using cache
Building Docker preloader image. [=====================   ] 87%
Step 7/7 : CMD ["python3", "/usr/src/app/preload.py"]
 ---> Using cache
 ---> 3200b63da1b5
Successfully built 3200b63da1b5
Building Docker preloader image. [========================] 100%



/ Creating preloader container
- Starting preloader container
\ Fetching application 1504381
| Reading image informationWaiting for Docker to start...
/ Reading image informationDocker started
\ Reading image information
Leaving splash image alone
| Resizing partitions and waiting for dockerd to startWaiting for Docker to start...
\ Resizing partitions and waiting for dockerd to startDocker started

/ Cleaning up temporary files

{ Error: (HTTP code 500) server error - Get https://registry2.balena-cloud.com/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 
    at /usr/local/lib/node_modules/balena-preload/node_modules/docker-modem/lib/modem.js:257:17
    at IncomingMessage.<anonymous> (/usr/local/lib/node_modules/balena-preload/node_modules/docker-modem/lib/modem.js:284:9)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9) reason: 'server error', statusCode: 500, json: null }

Looks like this might be an issue with balena-preload. Please report it at https://github.com/balena-io/balena-preload/issues

really confused…

mexekanez@mexekanezlaptop:~$ balena -v

(node:10669) Error Plugin: balena-cli: files attribute must be specified in /usr/local/lib/node_modules/balena-cli/package.json

module: @oclif/config@1.13.0

plugin: balena-cli

root: /usr/local/lib/node_modules/balena-cli

See more details with DEBUG=*

11.13.0

Docker version 19.03.2, build 6a30dfc

what else I can check to get point where it doesn’t works?

chrisys · October 1, 2019, 11:02pm

@mexekanez it’s strange that you would have the same/similar error for both different methods.

Looking at the first log you posted I note the section dial tcp: lookup registry.visoagent.xyz on 192.168.0.1:53: read udp 192.168.0.103:35298->192.168.0.1:53: i/o timeout. This looks like it’s trying to contact a DNS server on your local network, and failing. It would make sense that both methods fail in the same fashion if the container is unable to resolve DNS for either of them. It’s hard to speculate any further without knowing your network setup but perhaps this is something to look into?

mexekanez · October 2, 2019, 7:38am

I checked /etc/resolv.conf
and put

nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 127.0.0.53

roman-mazur · October 2, 2019, 9:16am

@mexekanez So did the change help with the issue?