As many of you may know, here at resin we work very hard to build the features that you, as users, request. One of the biggest, most requested and definitely most exciting projects that we are working on right now is the multicontainer update.
Currently to update any part of the user software on-device, one must update all of the components together. The multicontainer update is a way for device owners to deploy software in logical chunks, just like in any other modern software system. The advantages of this update will be massive, for example;
- Security - Why allow a networked component access to everything on-device? Give a container access to only the resources it needs, and worry less about exposing potentially private data.
- Stability - If one of your components is misbehaving, why does the rest of your application need to stop? Easily recover from errors by simple restarting the offending service.
- Uptime - Why stop your entire app to update your UI? Only update the components needed, and keep the others running.
But this isn’t an advertisement, this is a progress report! We’ve been working extremely hard to get this out for our users, and here is where we’re at:
The API
The API is the central nervous system of the entire resin infrastructure. It sits between our data and our components and provides structured and secure access to this data. The database represents our users, components and devices. It manages state and provides information, which is intrinsically linked with the resin workflow. This means that to go forward with multicontainer, we must first update the database models and data access protocols to support devices running multiple containers, having non-atomic install procedures and builds which produce several artifacts, whilst ensuring backwards compatibility with all existing devices and applications.
The first step is to update the database models. We use a language named SBVR and a custom parser known as pine. In SBVR, we define resources and the relationships between them, and use this information to define the database schema.
We’ve decided on the new format for the models to support multicontainer, and the SBVR has been updated. This will live update the pine models, but since the database schema itself will remain the same, we need to write migrations to modify the existing schema and data to agree with pine’s internal model. These migrations have been written and tested heavily on real data, and are looking good.
This is only half of the API story, though. We now need to take the incoming requests from older endpoints, and edit them slightly so that they query the data in its new location. This is called a translation, and is run on queries which do not hit the latest API endpoint path (which will be /v4 on release). I’m currently in the process of writing these translations and the new device state endpoint (which your devices use to find out what state that they should be in).
Supervisor
The supervisor is the agent that runs on devices and takes care of ensuring your application is running properly, fetching and applying updates as needed. It has grown organically so it didn’t have an architecture that would easily support having multiple containers for each application. We’ve been rearranging most of its code so that it has a cleaner architecture with a better separation between modules. This makes the supervisor more maintainable and allows us to add the multicontainer changes where they need to go.
Having multicontainer in the supervisor involves not only modelling several containers per app, but also creating and updating containers in the correct order, much like what Docker Compose does but with the extra dimension of update strategies and locks, where each container can have different ways of being updated.
All of this has been coded in a work-in-progress PR. It still needs debugging and testing but the code is there but for a few details in the docker-compose options available for each container.
The next step, once that code is working properly, is to switch to using the new endpoints on the API to get a multicontainer target state for the device, and correctly report the current state when the containers have been updated.
Builder
The builder currently takes a single Dockerfile and does some processing before sending it off to the docker daemon. It then takes the image, tags it correctly and uploads it to the resin registry, ready for your devices to download.
The builder will still do this, but instead of a single Dockerfile, it will take a single docker-compose file and several Dockerfiles (or Dockerfile.templates etc). It must then send the individual projects to the daemon, and then tag each in the correct format and push it to the correct location. The builder must also update the API with all of the information about the new images, their logs, statuses and registry locations.
In addition, the output from several different concurrent builds will definitely not be readable, much less helpful, so the builder will receive a UX overhaul before this feature is released, something I’m definitely looking forward to.
Task completion
Note: this is not an exhaustive list, and also does not detail the amount of work that goes into each task.
Complexity in this case is a measure of how much effort goes into a certain task, which can be indicative to the amount of time that each task will take, but tasks with higher complexities also have a habit of changing requirements slightly once the problem domain is understood in more detail.
[ ] API
[X] Define SBVR/model upgrades
- Complexity: medium
[X] Define migrations needed to move to new model - Updated: 26-09-2017
- Complexity: medium
[X] Define translation which convert old queries to new format - Updated: 26-09-2017
- Complexity: high
[-] Define new device state endpoint to support multiple containers - Updated: 26-09-2017
[X] Current state endpoint changed
[X] Multicontainer endpoint created
- Complexity: medium
[ ] Add tests for new functionality
[ ] Builder
[X] Accept a docker-compose file and build multiple different projects - Updated: 26-09-2017
- Complexity: medium
[X] Gather logs together and insert them to the database using the API - Updated: 26-09-2017
- Complexity: low
[X] Push multiple images to the correct location on the registry
- Complexity: low
[X] Inform the API that updates are available for a given application - Updated: 26-09-2017
- Complexity: low
[ ] UX overhaul
- Complexity: medium
[ ] Supervisor
[X] Refactor the supervisor to allow the multicontainer code to be added - Updated: 26-09-2017
- Complexity: high
[X] Implement the multicontainer models and update procedure - Updated: 26-09-2017
- Complexity: high
[-] Test and debug - Updated: 26-09-2017
- Complexity: medium
[X] Switch to the new API state endpoint
- Complexity: low
[ ] Dashboard
[X] Define mockups and new interfaces
- Complexity: medium
[X] Use new endpoints to query build data
- Complexity: medium
[-] Implement mockups, working with new data
- Complexity: high
[ ] CLI
[ ] Update build command to support several containers
- Complexity: medium (can use builder interface)
[ ] Update deploy to support several images
- Complexity: medium (again can use builder interfaces)
[ ] Update API “glue” to support the new format
- Complexity: high
[ ] Docs
.
I’m ready to beta test.
You can read more on our blog here: 
- thanks for the update, will have quite the fun this evening x’D. No rest for the wicked! ='D And cheers and thanks a lot to the whole team for that awesome work!