How to set sysctl on HostOS for entire fleet permanently?

Product support
1

Hi,
we are using a docker-compose.yml setup.
We need to set the following settings on the HostOS:

sysctl -w net.core.rmem_max=67108864
sysctl -w net.core.rmem_default=67108864

If I do this on the HostOS, everything is fine, our application works as intended.
Unfortunately, after a reboot, this setting is lost.

How can I set this fleet-wide so that it persists across reboots and any updates?

I have tried the following already:

RUN sysctl -w net.core.rmem_max=67108864
RUN sysctl -w net.core.rmem_default=67108864

in my Dockerfile
alongside both commands also in the CMD section, right before starting my main application.
In the docker-compose.yml the container has: privileged: true.
Even tried with: sysctls entries in docker-compose.yml … No luck :-/

Any ideas?
Thanks
Fritz

2

I’m setting these in a start-up script inside my container (not in the Dockerfile) and it works fine. I don’t think you can set kernel parameters (like these) in the Dockerfile as they aren’t stored in the file system where docker detects them and they become part of the overlay.

3

Hi @HakanL
thanks, I also tried that but I’m getting:
sysctl: cannot stat /proc/sys/net/core/rmem_max: No such file or directory
As stated above: the container has privileged: true in the docker-compose.yml.
I only see these settings:

$ sysctl net.core
net.core.somaxconn = 4096
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1

Any further hints? Did you add any additonal settings to your container?
Are you running single container or multi-service?
Thanks!

4

I’m running single container, but that shouldn’t matter, these kernel settings should be available in any of your containers. Here’s my output from inside the container, but I get the same result if I do it on the host OS:

root@b1de590:~# sysctl net.core
net.core.bpf_jit_enable = 1
net.core.bpf_jit_harden = 0
net.core.bpf_jit_kallsyms = 1
net.core.bpf_jit_limit = 132942659584
net.core.busy_poll = 0
net.core.busy_read = 0
net.core.default_qdisc = fq_codel
net.core.dev_weight = 64
net.core.dev_weight_rx_bias = 1
net.core.dev_weight_tx_bias = 1
net.core.devconf_inherit_init_net = 0
net.core.fb_tunnels_only_for_init_net = 0
net.core.flow_limit_cpu_bitmap = 0
net.core.flow_limit_table_len = 4096
net.core.gro_normal_batch = 8
net.core.high_order_alloc_disable = 0
net.core.max_skb_frags = 17
net.core.message_burst = 10
net.core.message_cost = 5
net.core.netdev_budget = 300
net.core.netdev_budget_usecs = 8000
net.core.netdev_max_backlog = 1000
net.core.netdev_rss_key = 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
net.core.netdev_tstamp_prequeue = 1
net.core.netdev_unregister_timeout_secs = 10
net.core.optmem_max = 20480
net.core.rmem_default = 212992
net.core.rmem_max = 3000000
net.core.rps_sock_flow_entries = 0
net.core.skb_defer_max = 64
net.core.somaxconn = 4096
net.core.tstamp_allow_data = 1
net.core.txrehash = 1
net.core.warnings = 0
net.core.wmem_default = 212992
net.core.wmem_max = 3000000
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1

I’m also running privileged. Here’s my docker-compose.yml:

version: '2.1'
networks: {}
volumes:
  resin-data: {}
  udevdata:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /run/udev/data
  logs:
    driver_opts:
      type: tmpfs
      device: tmpfs
  tmpstorage:
    driver_opts:
      type: tmpfs
      device: tmpfs
services:
  main:
    build:
      context: .
      dockerfile: Dockerfile.arm64
    privileged: true
    tty: true
    environment:
      - 'DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket'
    ports:
      - "22:22"
    expose: [22]
    restart: always
    network_mode: host
    volumes:
      - 'resin-data:/data'
      - 'udevdata:/run/udev/data'
      - 'logs:/tmp/DMXCore100Logs'
      - 'tmpstorage:/tmp/DMXCore100Temp'
    labels:
      io.resin.features.kernel-modules: '1'
      io.resin.features.firmware: '1'
      io.resin.features.dbus: '1'
      io.resin.features.supervisor-api: '1'
      io.resin.features.resin-api: '1'
1 Like
5

I think the issue is, that you are running in network_mode: host which we can’t.
Maybe there’s another option?
Thanks

6

Found the issue, I’ve now also added the label io.resin.features.procfs: '1' and also was able to enable network_mode: host on our environment. Now I can set it.
Thanks!

1 Like
7

Cool that you figured it out. I can’t remember why/when I set the procfs, I may have just copied it from a template. The network mode makes sense for my application, but I’m a little surprised it was required, but maybe it is so you’re manipulating the actual network interface, and not a virtual interface.

8

I’m experiencing problems with the latest balenaOS v6 running Docker v20 as it base engine. I’ve found that v19 (in previous versions of balenaOS, especially version v2.83.18) allow for the manipulation of net.core params through using host-mode, but this is stricter on Docker v20/latest balenaOS versions.

@fritz : What version of balenaOS are you running here? I’m trying to accomplish the same as you, but from my experience this is dependant on what docker version you’re running in the base-os.