Why Balena push created new container for every changes?

Product support balenaOS
1

I have a doubt related to the working of balena push.

I am trying the sample provided by the Balena.

My Docker file

# base-image for python on any machine using a template variable,
# see more about dockerfile templates here: https://www.balena.io/docs/learn/develop/dockerfile/
FROM balenalib/%%BALENA_MACHINE_NAME%%-python:3-stretch-run

# use `install_packages` if you need to install dependencies,
# for instance if you need git, just uncomment the line below.
# RUN install_packages git

# Set our working directory
WORKDIR /usr/src/app

# Copy requirements.txt first for better cache on later pushes
COPY requirements.txt requirements.txt

# pip install python deps from requirements.txt on the resin.io build server
RUN pip install -r requirements.txt

# This will copy all files in our root to the working  directory in the container
COPY . ./

# Enable udevd so that plugged dynamic hardware devices show up in our container.
ENV UDEV=1

# main.py will run when container starts up on the device
CMD ["python","-u","src/main.py"]

Now When I run balena push then it creates a container and everything works properly.

The second time when I make a change in code and push the code using balena push <ip-addres> it creates a new container.

Also, it doesn’t delete the previous container. It creates a new container and downloads the packages and everything.

Question:

Is there any way I can push on the existing container so that only the changes that I have made will be updated.

3

Hello, could you please capture and paste here the logs of the build? (Please ensure to mask any sensitive details before pasting)

5

I have noticed this happening only in the case of failure.

For example, in python, if I try to install a package network manager and it fails due to some issue. Next time it will create a new container from an existing image.

Example:

balena push <ip-address>                  
[Info]    Starting build on device <ip-address>
[Info]    Creating default composition with source: .
[Build]   [main] Step 1/8 : FROM balenalib/armv7hf-debian-python:3-stretch-run
[Build]   [main]  ---> <Tag>
[Build]   [main] Step 2/8 : WORKDIR /usr/src/app
[Build]   [main] Removing intermediate container 040129ef15a4
[Build]   [main]  ---> <Tag>
[Build]   [main] Step 3/8 : COPY requirements.txt requirements.txt
[Build]   [main]  ---> <Tag>
[Build]   [main] Step 4/8 : RUN pip install -r requirements.txt
[Build]   [main]  ---> Running in bb3db0ab3de8
[Build]   [main] Collecting python-networkmanager (from -r requirements.txt (line 1))
[Build]   [main]   Downloading https://files.pythonhosted.org/packages/73/ec/2cc46290cedc99f59df3633d428dfa91d95946ca49af96fdb53514ee195f/python-networkmanager-2.1.tar.gz
[Build]   [main] Collecting dbus-python (from python-networkmanager->-r requirements.txt (line 1))
[Build]   [main]   Downloading https://files.pythonhosted.org/packages/3f/e7/4edb582d1ffd5ac3c84188deea32e960b5c8c0fe1da56ce70224f85ce542/dbus-python-1.2.8.tar.gz (787kB)
[Build]   [main] Collecting six (from python-networkmanager->-r requirements.txt (line 1))
[Build]   [main]   Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
[Build]   [main] Installing collected packages: dbus-python, six, python-networkmanager
[Build]   [main]   Running setup.py install for dbus-python: started
[Build]   [main]     Running setup.py install for dbus-python: finished with status 'error'
[Build]   [main]     Complete output from command /usr/local/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-0pmhno41/dbus-python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-2i7hweqi/install-record.txt --single-version-externally-managed --compile:
[Build]   [main]     running install
[Build]       running build
[Build]       creating /tmp/pip-install-0pmhno41/dbus-python/build
[Build]       creating /tmp/pip-install-0pmhno41/dbus-python/build/temp.linux-armv7l-3.5
[Build]       checking whether make supports nested variables... no
[Build]       checking whether to enable maintainer-specific portions of Makefiles... no
[Build]       checking for gcc... no
[Build]       checking for cc... no
[Build]       checking for cl.exe... no
[Build]       configure: error: in `/tmp/pip-install-0pmhno41/dbus-python/build/temp.linux-armv7l-3.5':
[Build]       configure: error: no acceptable C compiler found in $PATH
[Build]       See `config.log' for more details
[Build]       Traceback (most recent call last):
[Build]         File "<string>", line 1, in <module>
[Build]         File "/tmp/pip-install-0pmhno41/dbus-python/setup.py", line 109, in <module>
[Build]           tests_require=['tap.py'],
[Build]         File "/usr/local/lib/python3.5/site-packages/setuptools/__init__.py", line 143, in setup
[Build]           return distutils.core.setup(**attrs)
[Build]         File "/usr/local/lib/python3.5/distutils/core.py", line 148, in setup
[Build]           dist.run_commands()
[Build]         File "/usr/local/lib/python3.5/distutils/dist.py", line 955, in run_commands
[Build]           self.run_command(cmd)
[Build]         File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
[Build]           cmd_obj.run()
[Build]         File "/usr/local/lib/python3.5/site-packages/setuptools/command/install.py", line 61, in run
[Build]           return orig.install.run(self)
[Build]         File "/usr/local/lib/python3.5/distutils/command/install.py", line 539, in run
[Build]           self.run_command('build')
[Build]         File "/usr/local/lib/python3.5/distutils/cmd.py", line 313, in run_command
[Build]           self.distribution.run_command(command)
[Build]         File "/usr/local/lib/python3.5/distutils/dist.py", line 974, in run_command
[Build]           cmd_obj.run()
[Build]         File "/tmp/pip-install-0pmhno41/dbus-python/setup.py", line 62, in run
[Build]           cwd=builddir)
[Build]         File "/usr/local/lib/python3.5/subprocess.py", line 271, in check_call
[Build]           raise CalledProcessError(retcode, cmd)
[Build]       subprocess.CalledProcessError: Command '['/tmp/pip-install-0pmhno41/dbus-python/configure', '--disable-maintainer-mode', 'PYTHON=/usr/local/bin/python3.5', '--prefix=/tmp/pip-install-0pmhno41/dbus-python/build/temp.linux-armv7l-3.5/prefix']' returned non-zero exit status 1
[Build]       
[Build]       ----------------------------------------
[Build]   [main] Command "/usr/local/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-0pmhno41/dbus-python/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-2i7hweqi/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-0pmhno41/dbus-python/
[Build]   
[Build]   [main] You are using pip version 19.0.1, however version 19.1.1 is available.
[Build]   You should consider upgrading via the 'pip install --upgrade pip' command.
[Build]   
Some services failed to build:
        main: The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1


Additional information may be available by setting a DEBUG=1 environment
variable: "set DEBUG=1" on a Windows command prompt, or "export DEBUG=1"
on Linux or macOS.

If you need help, don't hesitate in contacting our support forums at
https://forums.balena.io

For bug reports or feature requests, have a look at the GitHub issues or
create a new one at: https://github.com/balena-io/balena-cli/issues/

Now I’ll make some changes and try to install it again:

9318fd0d0f68        83ad743c2d48                       "/bin/sh -c 'apt-get…"   15 seconds ago      Up 5 seconds                                       frosty_saha
bb3db0ab3de8        83ad743c2d48                       "/bin/sh -c 'pip ins…"   8 minutes ago       Exited (1) 4 minutes ago                           angry_poitras

As you can see it’s using the same image But created a new container.

Thus it consumes the space on the device. Is there any way I can push on same container?

6

Hi,

You might be looking for the live switch (https://www.balena.io/docs/reference/cli/#--live--l)

This will monitor your files and live update the running container. It should be very handy wrt development. It is still a Beta feature, so if you encounter any problems or you have proposals on how to improve it, please let us know.

Cheers,

7

Yes I tried that But the problem is that on my image my supervisor version is old How can I update that?

8

You can just update the OS system for that device. This can be found in the dashboard, in the actions for each device.

9

I downloaded the latest OS from the BalenaOS website.

Right Now I haven’t signed up yet for BalenaCloud. Can I do same from openbalena?

10

Ah, we don’t support hostOS updates in openbalena yet, but it is in our roadmap for open source as well. What device type are you using? The live feature is very new, so we might not have released a new enough image for that device type yet.

11

I have downloaded the image for Beaglebone

12

We didn’t have upgraded the Beaglebone yet to a new enough version. The functionality for the supervisor was introduced in v2.31.0, but Beaglebone is at the moment still at v2.29.0. I’ll ask the OS team, when we plan to release the Beaglebone images.

13

Okay Thanks.

15

We don’t have an ETA for releasing the new Beaglebone images yet, but I added it to our internal issue tracker and we will update this thread, once we have it.

16

Okay Thanks. How can I update the host os if I don’t sign up for Balena cloud? Is there any other way?

18

At the moment, you can’t update the host os, with devices in openBalena, but this feature is on our roadmap, so it will reach openBalena.
Wrt the Beaglebone: even in Balena cloud, we don’t yet have released a newer BalenaOS image, so that it can be used with the live option during balena push. So unfortunately, you cannot use the live option with Beaglebones at the moment.

19

Okay and So I need to manually remove the previous container when making changes?

22

what version of the OS are you running at the moment?

Also, extra containers are not using much space at all (images do, but containers are usually very small). Would be good if we can check it out with the similar flow as you are describing and see how things are working. The older containers should not be there, as far as we understand, but maybe we miss something (due to OS, supervisor version, or some other thing)

24

When I did balena push <ip-address> at that time I already had everything in the cache with a minor change in my application. I pushed the code on my device.
After that I checked the number of images available on the device:

root@mydevice:~# balena images
REPOSITORY                        TAG                 IMAGE ID            CREATED              SIZE
local_image_test2               latest              a1aadd0110ed        27 seconds ago       259MB
<none>                            <none>              5f01a6571900        About a minute ago   259MB
<none>                            <none>              7b69fff588a6        3 hours ago          259MB
local_image_test       latest              61989cc08e35        19 hours ago         326MB

Changes were in the Dockerfile of local_image_test2 and in that case, it creates multiple images as you can see from the above log

root@mydevice:~# balena ps -a
928813632cce        a1aadd0110ed                       "/usr/bin/entry.sh e…"   4 minutes ago       Up 10 seconds                                       local_image_test_2
1abcdd8f67cd        61989cc08e35                       "/usr/bin/entry.sh e…"   4 minutes ago       Restarting (0) 28 seconds ago                       local_image_test
cdc5bd12f974        5f01a6571900                       "chmod +x /usr/src/a…"   5 minutes ago       Exited (1) 5 minutes ago                            gracious_jepsen
be431e59fa1f        2343aac9fbf7                       "sh"                     10 minutes ago      Up 10 minutes                                       frosty_mcclintock
2a1bc84deba9        7b69fff588a6                       "/usr/bin/entry.sh /…"   About an hour ago   Exited (130) 19 minutes ago                         pensive_yonath
8aaf02bffee0        2343aac9fbf7                       "sh"                     3 hours ago         Up 3 hours                                          pensive_galileo

As you can see it has created multiple containers which I haven’t been told in docker file. According to my understanding, it should create only local_image_test2 and local_image_test Image and container.

I have downloaded the Beaglebone black image from BalenaOS and my supervisor version is v9.0.1

25

Hi @sharvin26 , I believe the containers with names like gracious_jepsen etc are containers which are started in the build phase. What is weird is that some of them are just started with the sh shell command, which is incredibly strange. What version of the CLI are you using?

29

Version:-

 ➜ balena version
10.10.3

I have a Mac OS on my local machine,

Also, it’s strange that it created a new Image for local_image_test2 but didn’t destroy the previous image. According to my understanding, it should destroy the previous images if it’s creating a new Image. Right now I have to manually delete the Unused Images and container.

34

Hi @sharvin26. I was able to reproduce and can confirm that images and containers of failing local-push builds aren’t getting properly cleared up from the device. Thanks for reporting! We 'll be looking into this.
I couldn’t reproduce containers running “sh” though, using the simple-server-python example. Could those be manually created (maybe using balena run commands from the hostOS ) or somehow defined on your side?