Environment variables missing and /dev/tty* device list not updating

I can see BALENA_* variables with printenv in the balenaCloud terminal when connecting to main. But I can’t see them when connected to host or container via ssh.

I have added UDEV=1 in my Dockerfile to enable update of /dev/tty* devices, but my /dev/tty* device list is not updated in container, only on host as it should. I can see the UDEV variable in the balenaCloud terminal as I wrote above but nowhere else.
I have read this thread but it didn’t help me: Container can’t see new /dev/tty* device on host

My hardware is a STM32MP1 running BalenaOs 2.58. I don’t know if the problem lay in my Yocto image or how I created my container. I had to choose BeagleBone Black in balenaCloude to add it and it report balenaOS 2.38.3+rev5 to balenaCloude even though I know I bitbaked with meta-balena 2.58.0 which seems strange.

My Dockerfile looks like this:

# our local base image
FROM arm32v7/ubuntu 

LABEL description="Container for use with Visual Studio" 

# install build dependencies 
RUN apt-get update && apt-get install -y g++-8 rsync zip openssh-server make sudo cmake ninja-build  && \
    apt-get update && apt-get install -y git gdb

# set gcc v8 as default gcc
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 8

# set g++ v8 as default g++
RUN update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-8 8

# configure SSH for communication with Visual Studio 
RUN mkdir -p /var/run/sshd

RUN echo 'PasswordAuthentication yes' >> /etc/ssh/sshd_config && \ 
   ssh-keygen -A 

RUN useradd -rm -d /home/dev -s /bin/bash -g root -G sudo -u 1000 dev 
RUN  echo 'dev:dev' | chpasswd

WORKDIR /pistache

# Clone the pistache source code
RUN git clone https://github.com/oktal/pistache.git . && \
    git submodule update --init

#Compile pistache 
RUN mkdir build && \
    cd build  && \
    cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release .. && \
    make && \
    make install

ENTRYPOINT service ssh restart && bash

# Enable udevd so that plugged dynamic hardware devices show up in our container.
ENV UDEV=1

# RUN service ssh start

# expose port 22 
EXPOSE 22 9080

Hi it is normal to not see the BALENA_* variables in the hostOS, but you should be seeing them when SSHing to the container, regardless whether it’s from the webterminal or the through the hostOS.
For example if you SSH into the hostOS and then run balena exec -it <container_name_here> sh, you should be able to see the env vars when running env in there.
Can you confirm how you SSH into the container and whether the above works for you?

Are you also using a docker-compose.yml file ? If so please provide that as well.

If I get this right, you are saying that you are using a BeagleBone Black on different hardware.
Can you provide further info about how you created your custom balenaOS image and the device repository that you used a starting point?
I expect that there is still some reference of 2.38.3+rev5 at some place in your image, since that piece of information is only updated in our backend by the devices.

Kind regards,
Thodoris

With regards to enabling UDEV, I would suggest you to also give this a try when using a balenalib base image to confirm whether the base image has anything to do with it.
Moreover, as you probably already saw in the other forum thread that you linked, in case that you have a docker-compose.yml in your project, you also need to mark your container as privileged: true.
Let me point you to the respective documentation page:

Kind regards,
Thodoris

I can confirm that it worked by SSH into hostOS and then run balena exec -it <container_name_here> sh.

So I guess my problem is how I SSH into my container. I need SSH connection into my development container to debug my Linux application. This because I need access to my USB-hardware via /dev/tty*.

So I tried to solve this by installing openssh-server into the container with the Dockerfile and then expose port 22. In that way I can SSH direct into my container, but maybe I have missed something. I have also added a user called dev as you can see in Dockerfile. Dev is member of sudo and root group so I guess it’s not a permission issue.

I only set BeagleBone Black as devicetype in balenaCloud as there is no STM32MP1 to choose. The image is based on this port: https://github.com/kbumsik/balena-st-stm32mp

I solved that version issue. When compiling the image there is a VERSION file that I forgot to update. I haven’t tried recompiling the image but I guess that was the reason.

Hi, see here an example for ssh-ing to the container: https://github.com/balena-io-playground/balena-openssh

As for the missing tty, as my colleague asked, do you have a docker-compose.yml file in your project?

Thank you for your fast answare. I will check that ssh fix.

When it comes to not updating tty*, I don’t have any docker-composer.yml file in my development project.

Ok. Then can you try testing with a balenalib base image as Thodoris suggested, just to try and rule out problems with the base image?

So I followed that balena-openssh but with no different. I can still ssh into my container as I could earlier and I still can’t get those BALENA_* or RESIN_* environment variables.

I changed base to a balenalib base and the /dev/tty* device list now updates correctly after adding UDEV=on in Dockerfile.

So how do I access those environment variables?
I can access them from balenaCloude terminal when connecting to my container and I can access them when ssh into host and access the container via balena exec -it <container_name_here> sh.
I just can’t access them when ssh direct into the container.

Here is my Dockerfile:

# our local base image
FROM balenalib/armv7hf-ubuntu:focal-build

LABEL description="Container for use with Visual Studio" 

ENV UDEV on

# install build dependencies 
RUN apt-get update && apt-get install -y g++-8 rsync zip openssh-server make sudo cmake ninja-build  && \
    apt-get update && apt-get install -y git gdb nano

# set gcc v8 as default gcc
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-8 8

# set g++ v8 as default g++
RUN update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-8 8

# configure SSH for communication with Visual Studio
RUN mkdir -p /var/run/sshd

RUN echo 'PasswordAuthentication yes' >> /etc/ssh/sshd_config && \
   ssh-keygen -A

RUN useradd -rm -d /home/dev -s /bin/bash -g root -G sudo -u 1000 dev
RUN  echo 'dev:dev' | chpasswd

WORKDIR /pistache

# Clone the pistache source code
RUN git clone https://github.com/oktal/pistache.git . && \
    git submodule update --init

#Compile pistache 
RUN mkdir build && \
    cd build  && \
    cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release .. && \
    make && \
    make install

COPY start.sh /usr/bin/start.sh

# expose port 22 and 9080
EXPOSE 22 9080

CMD [ "/usr/bin/start.sh" ]

Here is my start.sh as from the ssh example:

#!/bin/bash

# Start sshd if we don't use the init system
if [ "$INITSYSTEM" != "on" ]; then
  /usr/sbin/sshd -p 22
fi

echo "This is where your application would start..."
while : ; do
  echo "waiting"
  sleep 60
done

Hi, unusual problem you have on your hands. Something that comes to mind is perhaps your ssh configuration results in you being logged in as a different user and thus a different environment. Also reading through the ticket looks like you did not mention why you need access to those env vars when sshing. Is it for debugging purposes? The reason I ask is perhaps we can suggest a work around to meet you requirements.

Hi,

In above Dockerfile you can see that I created a dev user, but I have also tried using root user with no difference.
I need access to environment variables to decide if the code in the container is running on BalenaOS or on a Docker server under Windows. As now I try to check that BALENA variable to determine that.

In the future it’s possible that we will add other variables to control the running code with settings. But of cause it’s possible to do this by let the code connect to another server and get information.

Hello,

This looks normal.
You are running sshd in the container.
When you connect to it, it starts a new shell for you with a clean environment.
Check man sshd.

If you want to make the environment variables available in shells created by sshd, one solution would be to write them to a file that would be sourced. Something like env > /etc/environment before starting the sshd server in your start.sh script.

Works like a charm. Thanks!

Had to tweak that command little bit. But now it’s working.

env | grep '^BALENA_\|^RESIN_\|^PATH=' > /etc/environment