Multiple version of container running following ENV VAR redef

Product support
1

Hi All,

Just encountered a very interesting problem: after redefining a couple of environment variables for a particular application, the app restarted, which is fine, and normal. Unfortunately it looks like 3 instances of our container spawned - that’s not normal!

I quickly logged into the host, and did a docker ps, to show that I’m not going crazy:

root@da2194a:~# docker ps
CONTAINER ID        IMAGE                                                                      COMMAND                  CREATED             STATUS              PORTS               NAMES
872d9d0e5e50        registry.resin.io/velimopretest/bcd1f95c37bb8efc1117edff2fd265c94f3b2d3d   "dockerd-entrypoint.s"   22 minutes ago      Up 11 minutes                           backstabbing_einstein
59fd7513916c        registry.resin.io/velimopretest/bcd1f95c37bb8efc1117edff2fd265c94f3b2d3d   "dockerd-entrypoint.s"   22 minutes ago      Up 11 minutes                           hungry_bartik
4a855947b0e7        registry.resin.io/velimopretest/bcd1f95c37bb8efc1117edff2fd265c94f3b2d3d   "dockerd-entrypoint.s"   22 minutes ago      Up 11 minutes                           hopeful_booth
7ce63346094b        resin/amd64-supervisor                                                     "/sbin/init"             2 weeks ago         Up 11 minutes                           resin_supervisor

As you can see, three versions of the same container are running there. What’s really weird, is that this status survives a device restart. Has anyone seen anything like that before? I should say now - we’re running a docker-in-docker image here, so it’s possible that it’s part of the problem, however we’ve been running this image a while now, and I don’t think I’ve seen this behaviour before.

Thanks,
Corin

5

Hi, are you running the image in local mode or the unmanaged resinOS version? What resinOS version is that? I’m guessing it’s a NUC?

Definitely does not look good, would be great to gather some more info about what’s happening. :question:
Would also wonder if there’s a simple way of reproducing this.

6

Hiya,

This is our own resinOS version, based on 2.08 NUC. It’s not actually a NUC, but hardware with a similar profile. We’ve been using it for around 2 months now, and I’ve never seen this happen - I happened to update a couple of environment variables at the same time, and then it happened. I’ve been unable to reproduce it since, but I’ll keep trying.

We’re in the process of updating to match the most recent resinOS release, as soon as we’ve got that finished, we’ll be doing some more in-depth testing, including on this edge case, so I’ll let you know if it pops up again.

Cheers,
Corin

1 Like