China - Great Firewall of China

So my old company deployed Raspberry Pi’s with ResinOS running on them to China. Ultimately we had a bunch of issues that we had to work around:

Internet quality is highly variable depending on your location. We tested devices in a city and they worked pretty well. We moved these devices to a university in a more rural area and we had a lot of issues. We had to work with the school’s IT staff to figure out that they were fiddling with the network in very weird ways.

The VPN was spotty at best and mostly unusable. There is not a lot of rhyme or reason to the way the “great firewall” blocks traffic. Some communication will get through but you will see massive packet loss and high latency. Don’t count on it working. Essentially don’t count on the VPN working.

HTTPS communication gets the same special treatment that the VPN traffic gets. Again no real pattern to the interference. This effects Resin OTA updates. It would take hours to update sometimes and days other times. Sometimes updates wouldn’t work at all.

We ended up using a mix of strategies to make things work. This was not an ideal setup by any means but it was the only way for us to create a somewhat stable platform.

Device communication:

  • We updated our software to only use HTTP communication. We had to use our own encryption scheme to encrypt secret data. All other data was sent encrypted.
  • We used a utility called sshuttle (https://github.com/sshuttle/sshuttle) to allow the devices to open up an on demand encrypted tunnel when we absolutely needed HTTPS. This worked well but I think it only worked because our endpoints were unknown to the PRC firewall. We used it as sparingly as possible in order to lower the chances of getting flagged. I have no idea if that strategy would have worked long term and I suspect that it would not have done as the PRC firewall becomes smarter.

Content upload/download:

  • Our devices were dependent on files that we generated in our backend and distributed to S3. Turning on edge acceleration and/or placing buckets in the asia region dramatically increased download speeds for HTTP and HTTPS (to a lesser extent).

Things that did not work for us:

  • AWS China: AWS China is not AWS. It’s a completely separate company. Beyond that there was at the time no way to have Resin route our VPN traffic to different endpoints so even if we had an internal ISP it would not have halpd the command and control situation.
  • VPN services: These services would work for a little while and then fail. Also we could not use them at the schools where our devices were being used as they are illegal and the school are owned and run by the government.
  • Partnering with Chinese ISPs: This is probably the only solution for a long term business that uses devices in PRC. We started down this road but as a very small company we did not have the time or the resources to figure it out. Three was no obvious information on how to do this. It seemed like you would need a person on the ground with considerable knowledge and experience in that arena to get it done.

Overall Advice:
If I were to do it all again…I wouldn’t. This is more to do with the failure of our business model in China. Ultimately we spent a decent amount of time and money to set up our devices in China just to have them not work well (due to connectivity issues). We were chasing contracts that did not materialize and ultimately cut our losses. Unless you have a very compelling business reason and very good local knowledge and connections in China I would avoid doing business there.

That said should you decide to go that route.

  • Make your system as flexible as possible especially in terms network communication. You will need to be able to configure the entire system.
  • Resin might not be your best choice in this regard. It’s not the fault of Resin but we ended up circumventing all of the features that Resin provides in order to make our devices work. The VPN and HTTPS communication are the two sticking points here. Resin required both of those to operate properly at the time. Perhaps they can accommodate you now, I have not been following their progress in about a year so do your own research.
  • Have someone on the ground in China who speaks the language. We had someone there and we would not have gotten anywhere without him. Really this is super important.

Soooo there you go. Good luck! You will need it :slight_smile: