Hi, thanks for the detailed explanation. I agree that reducing the applications DNS timeout would probably help workaround the bug in mosquitto. Unfortunately the BalenaOS default settings are designed to cover a wide range of applications. In this particular case, an increased local timeout helps reducing the bandwidth on slow cellular networks while in general having no adverse effect in other use cases, obviously this is an exception in your case.
Also, configurability is an architectural topic that has been often discussed inside Balena. The consensus is that we cannot provide settings for everything as that would fragment the product making it more difficult to test, maintain and support and also increase its complexity for our target user profile.
As I see it there are three options:
- Keep avoiding the use of DNS by using the IP address as you have described
- Build your own custom BalenaOS and modify it with your custom changes. Disadvantages with this approach include having to maintain your changes as BalenaOS versions are released, as well as not being able to update BalenaOS from the dashboard. Manual fleet update mechanisms exist for custom OSes but they are cumbersome to use.
- In the near future, BalenaOS will introduce the concept of host extensions. These are basically containers that get overlayed over the filesystem at boot. Such a container would allow you to configure the DNS timeout to a specific value. The responsibility to test and maintain changes introduced in this way will be with you, the user, and BalenaOS will not be tested with the change. Support agents may also ask you to reproduce any reported problem without the host extension.
I have opened a product ticket in https://github.com/balena-io/balena-io/issues/2305 and we will update it once a solution via host extensions is released.