How to set sysctl on HostOS for entire fleet permanently?

Product support
1

Hi,
we are using a docker-compose.yml setup.
We need to set the following settings on the HostOS:

sysctl -w net.core.rmem_max=67108864
sysctl -w net.core.rmem_default=67108864

If I do this on the HostOS, everything is fine, our application works as intended.
Unfortunately, after a reboot, this setting is lost.

How can I set this fleet-wide so that it persists across reboots and any updates?

I have tried the following already:

RUN sysctl -w net.core.rmem_max=67108864
RUN sysctl -w net.core.rmem_default=67108864

in my Dockerfile
alongside both commands also in the CMD section, right before starting my main application.
In the docker-compose.yml the container has: privileged: true.
Even tried with: sysctls entries in docker-compose.yml … No luck :-/

Any ideas?
Thanks
Fritz

2

I’m setting these in a start-up script inside my container (not in the Dockerfile) and it works fine. I don’t think you can set kernel parameters (like these) in the Dockerfile as they aren’t stored in the file system where docker detects them and they become part of the overlay.

3

Hi @HakanL
thanks, I also tried that but I’m getting:
sysctl: cannot stat /proc/sys/net/core/rmem_max: No such file or directory
As stated above: the container has privileged: true in the docker-compose.yml.
I only see these settings:

$ sysctl net.core
net.core.somaxconn = 4096
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1

Any further hints? Did you add any additonal settings to your container?
Are you running single container or multi-service?
Thanks!

4

I’m running single container, but that shouldn’t matter, these kernel settings should be available in any of your containers. Here’s my output from inside the container, but I get the same result if I do it on the host OS:

root@b1de590:~# sysctl net.core
net.core.bpf_jit_enable = 1
net.core.bpf_jit_harden = 0
net.core.bpf_jit_kallsyms = 1
net.core.bpf_jit_limit = 132942659584
net.core.busy_poll = 0
net.core.busy_read = 0
net.core.default_qdisc = fq_codel
net.core.dev_weight = 64
net.core.dev_weight_rx_bias = 1
net.core.dev_weight_tx_bias = 1
net.core.devconf_inherit_init_net = 0
net.core.fb_tunnels_only_for_init_net = 0
net.core.flow_limit_cpu_bitmap = 0
net.core.flow_limit_table_len = 4096
net.core.gro_normal_batch = 8
net.core.high_order_alloc_disable = 0
net.core.max_skb_frags = 17
net.core.message_burst = 10
net.core.message_cost = 5
net.core.netdev_budget = 300
net.core.netdev_budget_usecs = 8000
net.core.netdev_max_backlog = 1000
net.core.netdev_rss_key = 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00
net.core.netdev_tstamp_prequeue = 1
net.core.netdev_unregister_timeout_secs = 10
net.core.optmem_max = 20480
net.core.rmem_default = 212992
net.core.rmem_max = 3000000
net.core.rps_sock_flow_entries = 0
net.core.skb_defer_max = 64
net.core.somaxconn = 4096
net.core.tstamp_allow_data = 1
net.core.txrehash = 1
net.core.warnings = 0
net.core.wmem_default = 212992
net.core.wmem_max = 3000000
net.core.xfrm_acq_expires = 30
net.core.xfrm_aevent_etime = 10
net.core.xfrm_aevent_rseqth = 2
net.core.xfrm_larval_drop = 1

I’m also running privileged. Here’s my docker-compose.yml:

version: '2.1'
networks: {}
volumes:
  resin-data: {}
  udevdata:
    driver: local
    driver_opts:
      o: bind
      type: none
      device: /run/udev/data
  logs:
    driver_opts:
      type: tmpfs
      device: tmpfs
  tmpstorage:
    driver_opts:
      type: tmpfs
      device: tmpfs
services:
  main:
    build:
      context: .
      dockerfile: Dockerfile.arm64
    privileged: true
    tty: true
    environment:
      - 'DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket'
    ports:
      - "22:22"
    expose: [22]
    restart: always
    network_mode: host
    volumes:
      - 'resin-data:/data'
      - 'udevdata:/run/udev/data'
      - 'logs:/tmp/DMXCore100Logs'
      - 'tmpstorage:/tmp/DMXCore100Temp'
    labels:
      io.resin.features.kernel-modules: '1'
      io.resin.features.firmware: '1'
      io.resin.features.dbus: '1'
      io.resin.features.supervisor-api: '1'
      io.resin.features.resin-api: '1'
1 Like
5

I think the issue is, that you are running in network_mode: host which we can’t.
Maybe there’s another option?
Thanks

6

Found the issue, I’ve now also added the label io.resin.features.procfs: '1' and also was able to enable network_mode: host on our environment. Now I can set it.
Thanks!

1 Like
7

Cool that you figured it out. I can’t remember why/when I set the procfs, I may have just copied it from a template. The network mode makes sense for my application, but I’m a little surprised it was required, but maybe it is so you’re manipulating the actual network interface, and not a virtual interface.