Rsync over balena ssh tunnel?

Product support
4

@jason10, I will take the liberty of answering a slightly different question:

“How can I use rsync to transfer files between my laptop and my devices?”

Well, glad you asked. :slight_smile: In the current state of play, it may require a few setting up steps, depending on exactly what needs to be achieved.

The standard ssh, scp and rsync tools can be used with balenaOS devices. It works with both development and production balenaOS images. In the case of a production image, a ssh key should be added to the config.json file – see sshKeys section of the meta-balena README file. In the case of a development image, no keys are required. (Development images allow root login without a password or ssh keys, so should never be directly exposed to the public internet.)

The ssh server on a device (host OS) listens on TCP port number 22222. This port is not blocked by the device host OS, not even in production. If this port is blocked by a firewall or router on the device’s local network, or if the device has a private IP address, the balena tunnel command can be used as in the examples below.

Example 1 (laptop and device are on the same local network, OR the device has a public / globally routable IPv4/v6 address and port 22222 is not blocked in the device’s local network)

# All these commands to be executed on your laptop/desktop
$ ssh -p 22222 root@<device_ip_address>
$ scp -P 22222 my_local_file root@<device_ip_address>:/mnt/data/
$ export RSYNC_RSH='ssh -p 22222'
$ rsync my_local_file root@<device_ip_address>:/mnt/data/

Example 2 (remote device with a private IP address, or port 22222 is blocked in the device’s local network)

# All these commands to be executed on your laptop/desktop
$ balena tunnel <deviceUUID> -p 22222:4321
$ ssh -p 4321 root@127.0.0.1
$ scp -P 4321 my_local_file root@127.0.0.1:/mnt/data/
$ export RSYNC_RSH='ssh -p 4321'
$ rsync my_local_file root@127.0.0.1:/mnt/data/

Note: use CLI v10.4.0 or later to benefit from fixes to the tunnel command.

The examples above all transfer files between your laptop/desktop and a folder accessible via the device’s host OS – e.g. /mnt/data/. You you may be able to rely on named volumes to transfer files to app containers in this way. But with a bit more setting up, it’s possible to use rsync to transfer files directly into the app container:

# All these commands to be executed on your laptop/desktop
$ balena tunnel <deviceUUID> -p 22222:4321
$ export RSYNC_RSH="/home/user/rsync-shell.sh" && chmod +x "$RSYNC_RSH"
$ rsync my_local_file main_1_1:destination_folder/

Where 'main_1_1' is the name of the app container (as listed by balena-engine ps on the host OS).

The export line above mentions an rsync-shell.sh script. Here’s a basic version of it, with hardcoded values for port numbers and container names:

rsync-shell.sh

#!/bin/bash
CONTAINER_NAME='main_1_1'
TUNNEL_OPTS='-p 4321 root@127.0.0.1'
new_args=()
for arg in "$@"; do
  if [ "$arg" = "$CONTAINER_NAME" ]; then
    new_args+=("$TUNNEL_OPTS" balena-engine exec -i "$CONTAINER_NAME")
  else
    new_args+=("$arg")
  fi
done
echo original command: ssh "$@" >&2
echo modified command: ssh ${new_args[@]} >&2
ssh ${new_args[@]}

Notes:

  • rsync needs to installed in the app container too. Typically it is a case of apt-get update; apt-get install rsync.
  • The rsync-shell.sh script can be improved to suit your use case. As given above, it prints its original and modified input arguments to stderr, so it’s easier to see what is going on when you run rsync.
5 Likes