Questions around thermal restarts on the fin

We are experiencing random restarts that with the help of Balena engineers has been traced to a chip that controls USB power. This chip.

This is most likely because we are using 3 of these adapters per a device in a montioring mode which according to the specs draws 290mA during full receive mode.
This should be under the port max of 500mA hours but due to the substained nature of our usecase we are most likely:

  • Overheating that chip linked above
  • Overheating the chipset on the adapter causing it to draw more than 500 amps, and flipping current protection on that chip above

The heat is from the chipset on the adapter. (as best I can tell)

Please let me know if I am mistaken in any of my thoughts above, but here are our proposed solutions that has to be in place by next Tuesday.

  1. Hiring a 3d designer to expand the area under the lid (see the image below)
  2. Adding a fan to either a hole I make in the side or a hole in the new lid.
  3. Buying a bunch of these 6-inch USB extension, then plugging the adapters into those and hot glueing them to the side of the case to just reduce the heat passed back from the case to the fin
  4. With the expanded case, putting something like this powered usb hub that would draw from the 5v rail exposed on the Giop pins.
  5. In addition to the fan, add little heat sinks to the back of the USB ports and if we get the right sized thermal pad straight on the chips themselves.

Questions regarding the above.

  1. Would our 3d cases melt? Has anyone done this before? What should we look out for here?
  2. Can I power the fan off of the black pins labeled ext (the ones the left side of the fin if the USB ports are facing down)? I found one with 3.3v and one that was a ground. But I can’t find them in the schematics.
  3. This is a question of if the issue is heat alone or extra amps drawn because of the heat? Other than being really really ugly for production devices is there anything I am missing with this plan?
  4. This might the best one. Can I power this safely of the 5v GIOP pins? Will this put load on something that wasn’t meant to have load put on it? Will heat become an issue in the power management circuits of the fin?

If anyone has anything at all to add this would be great. This power issue only showed itself after putting a lid on the cases so we know we are close to being able to ship.

Thanks for your time and thank you to
@chrisys
@samothx
@ntzovanis
For your many support chat messages that got us this far!

-Thomas

image

An early design of the expanded lid for the case.

-Thomas

1 Like

Also @chrisys during the support chat you meantioned:

I’m seeing a lot of messages in the log about usb over-current

Was this just using:
journalctl --list-boots
journalctl -b -X
journalctl -b -X -r | grep over-current

because I have never managed to find that error and would love to be able to do so.

Thanks

Hey @taclog yep it was just using journalctl, I saw it on one of the devices as the last thing in the log before a reboot.

1 Like

Hello @tacLog thanks for sharing this. Here’re my thoughts on your questions:

  1. There are many plastics used for 3D printing that will serve the purpose. You can take a look here for examples. Look for any that has a glass transition temperature (or “heatproof”) of 70C+

  2. Those headers are exposed from the co-processor (J24 on Sheet 7 of the datasheet). There’s only 3.3V and limited to 100mA. You should use the coloured HAT headers to get 5V with higher current capabilities.

  3. My theory is that it’s just heat that triggers the safety feature. Another test we could try is to put a temperature sensor inside the case and see how hot it gets at full load.

  4. The 5V pins on the HAT header are intended to power external components. There is a maximum current of 2.5A (shared between all the devices connected to 5V) so should be enough for a fan or a USB hub.

Cheers,
Nico.

Hey,

Thanks for your response, Nicolas.

  1. We are getting the first batch of the 3d printed cases today. I have no idea what they are printed out of so we should assume the lowest quality plastic out there. I have yet to have them in hand, but my thoughts are to try and melt one to make sure it doesn’t happen in the field. i.e. Put in a fin in the case with the 3d printed lid. Put the whole unit in a box with a jacket over it and run stress to see how hot we can make things.

  2. I figured that out after I used the Co-processor headers and then fan speed was the definition of slow. I had to use some cable extensions but I have them running on the 5v rail now. They are very small and don’t move a lot of air, but they make the difference.

  3. I don’t have a temperature sensor of any kind. I should have ordered one Friday. I generally touch things and if they nearly burn me they are really hot. What I do have is a USB draw reader now. Under my use case, my USB wifi adapters run at .16-.17 amps at 4.77 volts. Please note that because this USB draw reader distances the adapter from the case it reduces the heat load on both the adapter and the chips. I would expect an adapter running hotter to draw more. We notice a 2x difference in data volumes between an adapter running cold or warm, vs hot.

  4. Thanks, my only thought is for the future if we use the fin over a raspberry pie zero w, is that what would get hot if we are drawing 3 or 4 watts from that. My assumption is that it would handle it though.

Thanks for your responses they are very helpful. A general update will follow. We should be shipping with 80 percent confidence tomorrow.

-Thomas

A more general status update.

We think we solved our problem 3 key parts.

  1. Tiny copper pads that go on the chips:

    Cut with a Dremel to size. They only are placed on the bottom two, the left one never got past warm to my finger test. I think this is because the heat is mostly from the devices plugged into the USB ports. See photo 1 at the end.
  2. A larger heat sink on the back of the USB ports that takes most of the heat away before it gets to the chips in the above photo.
  3. A fan installed on the left side of the case powered by the 5v rail.

Things I am still worried about:

  • The coper pads slipping off and shorting out.
  • The fan failing early and the device becoming completely unstable
  • The wires getting in the way of the fan after being moved around during shipping
  • The solution not being enough cooling if I increase CPU load later

The copper pads slipping off and shorting something out.

Reference photo 1: (the copper pads are peaking out from beneath the larger heatsink

Reference photo 2: (the fan and its wiring)


Reference photo 3: (the fan slats cut by hand, things get a bit smoother with each case I do)

Thanks @tacLog for sharing your solution, very ingenuitive as always! No doubt @ntzovanis will be interested to see your photos.

1 Like

Cases came and they look and feel rough, but good enough to work.

They support the stock screws which make things easy.

Time to test if I can melt them!

I am working to get permission to open source the design and will update this thread if I succeed.

1 Like

@tacLog nice! That would certainly be a valued contribution if you are able to share the design.

Small update. We shipped these our last week. The screw supports were weak and one of them broke form over tightening, but overall they where sufficient for the pinch I was in.

I received the OK to share the design files, and I will post them this week.

They melt if you apply direct heat, so I would not recommend 3d printing the bottom part of the case as the CPU might get hot enough to sag the plastic. However, I couldn’t see how the top would ever get that hot with a fan running.

Files released here: