deoqc
June 26, 2018, 1:22pm
1
Hello,
When I update a application, usually all images that should be cached were cached and the build process and the actual update of application is fast.
But sometimes it doesn’t use the caches - and here I’m talking about the cases when they should because nothing changed in to justify not using - and so the build and update is much longer.
I believe it is a (somewhat minor but annoying) bug.
Thanks
Hi @deoqc ,
That’s unusual. Can you provide any more information?
Are you pushing via git push
?
If so, are you pushing with --force
?
Can you share your source code, or at least your Dockerfile?
What exactly has changed in the cases where you push and see an unexpected uncached step?
deoqc
June 26, 2018, 2:14pm
4
Thanks for prompt response.
Yes, using git push
I usually run w/ --force
, though not always needed
These are the first 2 steps in Dockerfile
:
FROM resin/raspberry-pi-buildpack-deps:jessie-20180119
RUN \
apt-get update && \
apt-get install -yq \
build-essential \
libssl-dev && \
apt-get clean && rm -rf /var/lib/apt/lists/*
I would expect only 11th step to not being cached and changed, this is the step of copying files.
This is the output when everything works as it should:
Counting objects: 34, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (33/33), done.
Writing objects: 100% (34/34), 2.02 MiB | 0 bytes/s, done.
Total 34 (delta 20), reused 0 (delta 0)
[Info] Starting build for testerpizero, user deo_queiroz_carlos
[Info] Dashboard link: https://dashboard.resin.io/apps/1003045/devices
[Info] Building on arm01
[Info] Pulling previous images for caching purposes...
[Success] Successfully pulled cache images
[main] Step 1/12 : FROM resin/raspberry-pi-buildpack-deps:jessie-20180119
[main] ---> 8f49d3f8563b
[main] Step 2/12 : RUN apt-get update && apt-get install -yq build-essential libssl-dev && apt-get clean && rm -rf /var/lib/apt/lists/*
[main] Using cache
[main] ---> fe6bac3f8817
[main] Step 3/12 : WORKDIR /usr
[main] Using cache
[main] ---> 30a3c88c7650
[main] Step 4/12 : ARG PYTHON_VERSION=3.6.3
[main] Using cache
[main] ---> 54bb5d91f18d
[main] Step 5/12 : RUN mkdir tmp && cd tmp && wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz && tar -xvf Python-${PYTHON_VERSION}.tar.xz && cd Python-${PYTHON_VERSION} && ./configure && make && make altinstall && rm -rf /usr/tmp
[main] Using cache
[main] ---> 37521f6bda18
[main] Step 6/12 : ARG NODE_VERSION=8.9.4
[main] Using cache
[main] ---> 600ac7fbdd5d
[main] Step 7/12 : RUN mkdir tmp && cd tmp && wget https://nodejs.org/dist/v${NODE_VERSION}/node-v${NODE_VERSION}-linux-armv6l.tar.gz && tar -xvf node-v${NODE_VERSION}-linux-armv6l.tar.gz && cd node-v${NODE_VERSION}-linux-armv6l && rm CHANGELOG.md LICENSE README.md && cp -R * / && rm -rf /usr/tmp
[main] Using cache
[main] ---> 9578104bc47e
[main] Step 8/12 : RUN mkdir tmp && cd tmp && wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key && apt-key add mosquitto-repo.gpg.key && cd /etc/apt/sources.list.d/ && wget http://repo.mosquitto.org/debian/mosquitto-jessie.list && apt-get update && apt-get install -yq mosquitto mosquitto-clients && apt-get clean && rm -rf /var/lib/apt/lists/*
[main] Using cache
[main] ---> 6da28d7cb729
[main] Step 9/12 : RUN apt-get update && apt-get install -yq libffi-dev gettext-base && apt-get clean && rm -rf /var/lib/apt/lists/*
[main] Using cache
[main] ---> 61a7e0235f7e
[main] Step 10/12 : RUN systemctl disable mosquitto && npm install pm2 -g
[main] Using cache
[main] ---> a28c93cc21b0
[main] Step 11/12 : COPY build/* ./
[main] ---> 00ae3e5b74be
[main] Removing intermediate container d9652d1eead0
[main] Step 12/12 : CMD ./startup && modprobe i2c-dev && pm2-runtime start processes.json
[main] ---> Running in 97ff401ec739
[main] ---> 06b926e29b09
[main] Removing intermediate container 97ff401ec739
[main] Successfully built 06b926e29b09
[Info] Uploading images
[Success] Successfully uploaded images
[Success] Release successfully created!
[Info] ┌─────────┬────────────┬────────────┐
[Info] │ Service │ Image Size │ Build Time │
[Info] ├─────────┼────────────┼────────────┤
[Info] │ main │ 684.86 MB │ 3 seconds │
[Info] └─────────┴────────────┴────────────┘
[Info] Build finished in 35 seconds
And this is the output, util step 2, when not using cache:
Counting objects: 10, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (10/10), done.
Writing objects: 100% (10/10), 264.57 KiB | 0 bytes/s, done.
Total 10 (delta 8), reused 0 (delta 0)
[Info] Starting build for testerpizero, user deo_queiroz_carlos
[Info] Dashboard link: https://dashboard.resin.io/apps/1003045/devices
[Info] Building on arm03
[Info] Pulling previous images for caching purposes...
[Success] Successfully pulled cache images
[main] Step 1/12 : FROM resin/raspberry-pi-buildpack-deps:jessie-20180119
[main] ---> 8f49d3f8563b
[main] Step 2/12 : RUN apt-get update && apt-get install -yq build-essential libssl-dev && apt-get clean && rm -rf /var/lib/apt/lists/*
[main] ---> Running in 895568b4b728
[main] Get:1 http://archive.raspbian.org jessie InRelease [14.9 kB]
[main] Get:2 http://archive.raspberrypi.org jessie InRelease [22.9 kB]
[main] Get:3 http://archive.raspbian.org jessie/main armhf Packages [13.3 MB]
[main] Get:4 http://archive.raspberrypi.org jessie/main armhf Packages [171 kB]
[main] Get:5 http://archive.raspbian.org jessie/contrib armhf Packages [49.5 kB]
[main] Get:6 http://archive.raspbian.org jessie/non-free armhf Packages [101 kB]
[main] Get:7 http://archive.raspbian.org jessie/rpi armhf Packages [1297 B]
[main] Get:8 http://archive.raspbian.org jessie/firmware armhf Packages [1208 B]
[main] Fetched 13.7 MB in 7s (1751 kB/s)
[main] Reading package lists...
[main] Reading package lists...
[main] Building dependency tree...
[main] Reading state information...
[main] build-essential is already the newest version.
[main] Recommended packages:
[main] libssl-doc
[main] The following packages will be upgraded:
[main] libssl-dev libssl1.0.0
[main] 2 upgraded, 0 newly installed, 0 to remove and 71 not upgraded.
[main] Need to get 1948 kB of archives.
[main] After this operation, 92.2 kB of additional disk space will be used.
[main] Get:1 http://archive.raspbian.org/raspbian/ jessie/main libssl-dev armhf 1.0.1t-1+deb8u8 [1096 kB]
[main] Get:2 http://archive.raspbian.org/raspbian/ jessie/main libssl1.0.0 armhf 1.0.1t-1+deb8u8 [852 kB]
[main] debconf: delaying package configuration, since apt-utils is not installed
[main]
[main] Fetched 1948 kB in 0s (2758 kB/s)
[main] (Reading database ...
(Reading database ... 30% ... 5%
(Reading database ... 65% ... 35%
[main] (Reading database ... 70%
[main] (Reading database ... 75%
[main] (Reading database ... 80%
[main] (Reading database ... 85%
[main] (Reading database ... 90%
[main] (Reading database ... 95%
(Reading database ... 21957 files and directories currently installed.)
[main] Preparing to unpack .../libssl-dev_1.0.1t-1+deb8u8_armhf.deb ...
[main] Unpacking libssl-dev:armhf (1.0.1t-1+deb8u8) over (1.0.1t-1+deb8u7) ...
[main] Preparing to unpack .../libssl1.0.0_1.0.1t-1+deb8u8_armhf.deb ...
[main] Unpacking libssl1.0.0:armhf (1.0.1t-1+deb8u8) over (1.0.1t-1+deb8u7) ...
[main] Setting up libssl1.0.0:armhf (1.0.1t-1+deb8u8) ...
[main] Setting up libssl-dev:armhf (1.0.1t-1+deb8u8) ...
[main] Processing triggers for libc-bin (2.19-18+deb8u10) ...
[main] ---> 92e0755292bd
[main] Removing intermediate container 895568b4b728
ps: Posted info until step 2, since the cache stop working there already. If necessary, I can expand the info.
lekkas
July 11, 2018, 5:19pm
9
Hi,
Thank you for the extra context. This does look like a bug from our side, we have it in our radar and will update the thread as soon as we have any updates.
Best, Kostas