Error Copying file from container to External HDD

I’m trying to copy data from balena docker container to an external HDD. I’m using NodeJS for copying the files. The problem I’m getting is When I copy I get the acknowledgment that the file has been copied but the bytes written are zero.

I’m using bash command in the nodeJS process but still, I’m not able to copy files reliably. It is really confusing as Files show up with data if I wait long enough but there is no feedback/ acknowledgment on file copy complete.
I think when a device is mounted we get a mount path and that is a virtual location, not the exact external HDD location. So when we copy from the container it copies the files to that virtual location and sends feedback that files are copied. But after this balena internally handles the file copy and copy the files to the actual external HDD location. So if there is a way to directly write to the actual External HDD location that would be great.

Command I’m using " require('child_process').execSync(cp -f ${source} ${destination})

Methods I have tried so far in NodeJS.

  1. Copy files using fs.copyFiles
  2. Copy files using fs.copyFileSync
  3. Read and write data stream.
  4. Copy using Bash command i.e. cp -f

Methods to verify files have been copied.

  1. writeStream.on(‘finish’) to check for copy complete
  2. matching file sizes on source and destination and trust me they match exactly but still not to the actual External HDD location just to virtual mnt/ location inside the container.
  3. Matching the Hash value of the files and they match exactly but still not to the actual External HDD location just to virtual mnt/ location inside the container.

I am experiencing this same exact issue. The weird part is that the event fires that the file has been copied, but 0 bytes are written. Its very odd, as the files do eventually copy and the bytes get written, but its always after a seemingly random amount of time. This means that I have no way to know and alert the user exactly when the bytes have actually been written to the external HDD.

You may want to try the sync command to verify the files have been written to disk.
https://www.man7.org/linux/man-pages/man1/sync.1.html

The behaviour may also be different depending on the filesystem of the target disk. Are you using ext4 in this case?

I’m having a very similar problem as well. The copy seems to be massively delayed or not occur at all.