Balena Etcher - clone drive, now have two blank drives

balenaEtcher
1

Used Etcher in win10 to clone a drive which had Libreelec/Kodi on it for RPI.
Instead of cloning, I have ended up with two blank drives.
And I set it up correctly.

Reading around the forum it seems a lot of people are having problems with this software erasing their drives.
It sounds like it’s not fit for purpose.
But you’re still pushing it and people are loosing vital information and work.

There isn’t much to do, expect redo the entire kodi setup once again. That’s a day of my time wasted because of a tool which not only doesn’t work, that would have been ok, but is actually wiping the source drive when it should be treating it as read only.
That’s a basic concept in drive cloning, to protect the source.

Back to clonezilla, only reason I didn’t use it was because I didn’t wish to reboot and etcher came “recommended”.
It’s a joke. A really bad one.

You’ve basically written malware.

5

Hi Dave,

It’s possible you may have discovered an issue with Etcher’s new drive-cloning functionality. If this is the case, we’d love to make sure it gets fixed. My colleague has just tested this in version 110 of Etcher and it worked fine. If you’re not already on v110 we suggest updating Etcher and trying with that version.

7

You can install v110 from the Etcher releases page on GitHub as it’s not officially released yet.

9

Sorry but running far far away from the software and never looking back is my next step.
Its just not conceivable that one would allow a source drive to be erased.

Etcher was 375Mb installed - I checked when uninstalling.
I have an old 16Mb (yes that old) USB stick which boots into dos, runs ghost and still manages to copy drives just fine (certain size limitations).

13

I understand your perspective Dave, and based on your report / feedback, we are investigating what could have gone wrong, and perhaps some UX / UI improvements can come out of this. Most folks out there don’t have a 16mb USB drive with Ghost on it, or may not have ever heard of that, so getting it fine-tuned for a user-friendly experience on today’s hardware and modern OS’es is important, thanks.

14

So to be fair i have now tried imaging the drive with Rufus, 3.11.1678 and it also had issues.
It failed with a read error, looked to have corrupted the drive but unplug and replug and all was still ok.

Drive image was of LibreELEC-RPi2.arm-9.2.6.img.gz written to a sandisk 8Gb Ultra uSD card.
Having been configured in a RPI2 - idea to duplicate the card for several other RPIs wanting the same config.
uSD was put into a generic USB adapter for reading in windows10 and attempted to clone to the internal SD card reader with uSD sandisk adapter in a Dell E6430 with a sandisk 8Gb Extreme uSD.

Clonezilla was able to image to another drive and rewrite fine, as well as clone with the same adapters.
I tried Etcher because I didn’t want to reboot and pause other work (for clonezilla) and it came recommended for writing RPI images.

15

Well, with a second application having trouble as well (Rufus), it may be that some of the blocks are beginning to fail on the SD Card. Those SanDisk Extreme are what we generally recommend, but even so, they definitely have a limited amount of read/writes they can handle. Flash memory in this form factor just isn’t as reliable as enterprised grade SSD’s, or of course spinning disk. Either way, we are updating some of the iconography, UX text, and have double-verified that we only open Source media in “Read-Only” to hopefully prevent other users from a similar experience. Thanks!

1 Like
17

If you only open source media in read only mode how do we think it became corrupted?

Whilst I agree that RPIs have a reputation for harming uSD cards, these cards are not very old and have not been used for long periods of time or for high write cycles.
Incidentally I have used sandisk (and other branded cards) in high write cycle environments namely video and it does take a very long time (overwrites) to see such problems - over hundreds of examples tested.

And again clonezilla in standard mode (hit enter, enter, enter, etc) did not present a problem with the same cards.
Perhaps ignoring read errors (for whatever process might cause them) would be worthwhile.

19

Hi Dave,

If you only open source media in read only mode how do we think it became corrupted?

I get why you’re annoyed - you just want to get on setting up Kodi not messing about with drive cloning. The team here at balena are all similar to you, hacking projects and using devices like RPIs to add to our homes. So we take these sorts of reports seriously. Etcher is, after all, a tool we all use daily too. This is why we’ve been trying hard to see if there is anything we could have done this end to cause your problem.

In terms of code, since Etcher is all opensource you can take a look at it yourself. Here we open the source drive when you select it. You can see we open it as a BlockDevice:

let source;
			if (options.image.drive) {
				source = new BlockDevice({
					drive: options.image.drive,
					direct: !options.autoBlockmapping,
				});

the code for a block device is here, in which you can see we set it to read-only:

export class BlockDevice extends File implements AdapterSourceDestination {
	private drive: DrivelistDrive;
	private unmountOnSuccess: boolean;
	public oDirect: boolean;
	public emitsProgress = false;
	public readonly alignment: number;

	constructor({
		drive,
		unmountOnSuccess = false,
		write = false,
		direct = true,
	}: {
		drive: DrivelistDrive;
		unmountOnSuccess?: boolean;
		write?: boolean;
		direct?: boolean;
	}) {
		super({ path: drive.raw, write });

All of that means the drive you select as the source, when cloning, CANNOT be written to in any way. We can only read what’s on that drive.

As well as reviewing our code we’ve also been trying to recreate your problem with Etcher, and cannot. The only explanation left to us is that you unfortunately mixed up your source and destination drives and wrote the contents of the empty drive onto the one you wanted to clone, containing kodi.

What we’re now doing is trying to think of things we can do our end, to make the UI clearer about which drive is which, and mitigate this happening to others (or ourselves).
This may be updated iconography and/or additional information about the drives.

Phil

21

Appreciate that is what the code says but that is but one block making the call.
If code ran as it’s supposed to, there would be no need for debugging nor a raft of technical support guys finding problems in the real world and people fixing code.
There would be no google and apple bounty programs.
No cyber security problems.
Life would be great !

Sorry but this is a problem I’ve encountered a lot in my work. The default explanation is that it’s something the customer is doing wrong because we cannot recreate it.
My job at times has been trying to recreate those problems and it becomes (depending on the customer, severity and money at risk) quite an effort of repeatability testing to eventually find the root cause.
And when you do recreate it, the response is often interesting; The ostrich effect is quite common.

So whilst you come to this conclusion, let me throw a spanner in the works.
Drive A the source drive contained the archive (and config) I listed above. (if you’re testing, I’d install it as it puts two partitions on the drive, a FAT and an EXT)
Drive B, contained other data.

Drive A was inserted, clone was selected in Etcher. Only then was Drive B inserted and destination was selected.
After clone, both Drive A and Drive B were listed in disk manager as unallocated space.
Diskpart found nothing either.

I appreciate that the code says the above is impossible but there we go, I aint pulling your leg.
If I had a dollar for every time I’ve reported an impossible bug etc etc…

22

Hello @destroyed

I really can’t see how the source drive could have been written to.

We’ve tried to reproduce this issue but we couldn’t.

There is only one path in the code and it opens the source drive read-only.

The default explanation is that it’s something the customer is doing wrong because we cannot recreate it.

We’re only trying to find an explanation to what happened.

Perhaps ignoring read errors (for whatever process might cause them) would be worthwhile.

This is not something you want in an application that clones your drive. We do retry reading / writing after some delay on errors though. But if all attempts fail, the flashing fails.

Thank you for providing more details.

Now, we know that both SD cards had data and that you did not mix up the source and target drives (if you did, drives would have some data).

It would be interesting to know:

  • What raw data you have on both SD cards now if you didn’t reflash them ? Could you upload images of these SD cards somewhere ?
  • Do the SD cards work fine (you’ve mentioned that one had read issues) ?
24

@destroyed I think I’ve found something. I’ll keep you updated.

25

That’s not always a good thing, but it depends on what you’re trying to clone I suppose.
Certainly there are options in clonezilla to ignore read errors or even fix them on the fly, and to interact with files systems.
Sure here we are just writing images except for the clone, but sometimes you just want a copy of the thing, eg failing media, forensic backup, and having a copy with a little corruption is better than no copy.

Both the cards were reused after the issue sorry.
Both are working fine with the libreelec image on them.
Again I setup one card in one RPI, clone it (clonezilla this time) and then just changed the device name after boot on the cloned device.

dtischler suggested the possibility of block errors on the card. I’m not discounting it, but it cloned ok in zilla and it’s working fine at the moment.

Look forward to your findings.

28

Hey @destroyed

Thanks for the info.

So this is a serious bug. It happens only on Windows, this is why I couldn’t reproduce it at first (I was trying on Linux).
It affects all versions with the clone drive feature: 1.5.107, 1.5.108, 1.5.109 and , 1.5.110.

It is due to the fact that we run some diskpart commands (clean and rescan) on the drives before flashing them (otherwise Windows doesn’t let you write to the first bytes of the disk where the partition table is).
There is no check around this part so we also run this for source drives: https://github.com/balena-io-modules/etcher-sdk/blob/9e539f3842d91900181efef302791f469fdbd376/lib/source-destination/block-device.ts#L176

So even if we open the source drive readonly, we diskpart clean it first, erasing its partition table.
Then we clone it to the target drives, resulting in drives having all data except the partition table making them appear as empty.

This will be fixed in the next release 1.5.111, we’re removing the impacted versions in the meantime.

You can most probably recover data on impacted drives. You’d need to recreate the partition table for that. If you have the source image for the drive it shouldn’t be too difficult. I can provide help if needed.

Thanks again for reporting this.

29

Well that’s a bit of a bugger but at least we’ve all learnt something:
Windows sucks.

Thanks for the offer but I just went through re-imaging the card/drive and going through the config again.
Second run through was faster!

31

The exact same issue happened to me today. Looks like both drives has been cleaned, as mentioned.

Lost some important files. :confused:

Release 1.5.109, Windows 10.

Should have updated…

34

@zvin

My hard was completely erased and partitioned as only 1 drive, and that was a very important drive with very important data for a machinery.

THIS WILL COST ME MY JOB !!

I NEED TO RECOVER THE DATA, PLEASE HELP ASAP

38

Hello @harderased

Did you use Etcher’s clone drive on Windows ?
Which Etcher version was it ?
The source drive was erased, right ?

If yes, do you know how the source drive was partitioned ?
Was it an MBR or GPT partition table ?
How many partitions were there ? What type of partitions ?
Do you have access to a computer running Linux ?

40

Yes it was windows 10
I deleted it to avoid ever using it again- downloaded on November 26
Yes was erased

No
I do not recall
it was a windows 7 drive
i do have a Mac laptop

44

Hello @harderased

Do you know how many partitions were on the drive ?
Were they fat32 or ntfs partitions ?
How large is that drive ?

I’ll assume that there was only one partition and that you have enough disk space on your mac to make a disk image of this drive.
The partition table is missing on that drive, so we need to find where the partition is.
If the drive was readable on Windows 7, I guess the partition is either fat32 or ntfs.

Do not format this drive if any system asks you to !

In order to recover data (on a mac):

Step 1: Find the offset of the partition

  • before connecting the drive, open a terminal on your mac and type diskutil list
  • connect the drive to your mac, if macOS tells you something like “this drive is not readable on this computer”, select “ignore”, DO NOT SELECT FORMAT;
  • run diskutil list again, a new drive should have appeared, note its name (something like /dev/disk2);
  • I will use /dev/disk2 in the commands below, please use the correct disk number from the previous step;
  • run sudo hexdump -C -s 512 /dev/disk2 | head which will show you the contents of the beginning of the disk (skipping the first 512 bytes).

You should see something like:

00000200  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00100000  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS    .....|
00100010  00 00 00 00 00 f8 00 00  3f 00 ff 00 00 08 00 00  |........?.......|
00100020  00 00 00 00 80 00 00 00  ff 17 b7 03 00 00 00 00  |................|
00100030  00 00 0c 00 00 00 00 00  02 00 00 00 00 00 00 00  |................|
00100040  f6 00 00 00 01 00 00 00  31 0c 76 f0 2d 76 f0 9e  |........1.v.-v..|
00100050  00 00 00 00 fa 33 c0 8e  d0 bc 00 7c fb 68 c0 07  |.....3.....|.h..|
00100060  1f 1e 68 66 00 cb 88 16  0e 00 66 81 3e 03 00 4e  |..hf......f.>..N|
00100070  54 46 53 75 15 b4 41 bb  aa 55 cd 13 72 0c 81 fb  |TFSu..A..U..r...|

If you see the .R.NTFS on the third line of the output, this is an NTFS partition starting at 0x00100000 bytes (the number in the first column of the third line)= 1048576 bytes = 1MiB.

Another possible output would be

00000200  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00100000  eb 58 90 4d 53 44 4f 53  35 2e 30 00 02 20 2c 09  |.X.MSDOS5.0.. ,.|
00100010  02 00 00 00 00 f8 00 00  3f 00 ff 00 00 08 00 00  |........?.......|
00100020  00 18 b7 03 6a 3b 00 00  00 00 00 00 02 00 00 00  |....j;..........|
00100030  01 00 06 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
00100040  80 00 29 24 bf 89 6e 4e  4f 20 4e 41 4d 45 20 20  |..)$..nNO NAME  |
00100050  20 20 46 41 54 33 32 20  20 20 33 c9 8e d1 bc f4  |  FAT32   3.....|
00100060  7b 8e c1 8e d9 bd 00 7c  88 56 40 88 4e 02 8a 56  |{......|.V@.N..V|
00100070  40 b4 41 bb aa 55 cd 13  72 10 81 fb 55 aa 75 0a  |@.A..U..r...U.u.|

If you see the .X.MSDOS5 on the third line of the output, this is a FAT32 partition starting at 0x00100000 bytes (the number in the first column of the third line) = 1048576 bytes = 1MiB.

We don’t really care what type of partition (fat32 or ntfs) it is, what we need is the offset: the number in the first column, 00100000 in both cases above. This is an hexadecimal number. So 0x00100000 = 1048576 = 1MiB

Step 2: Backup the partition

Copy the raw partition to a disk image with sudo dd if=/dev/rdisk2 of=copy.img bs=1048576 skip=1 (replace rdisk2 with the number of the disk from the previous steps and note the r before disk), this will take some time depending on the size of the drive.

Detail:

  • if=: input file;
  • /dev/rdisk2: the disk from the steps above, rdisk instead of disk will copy faster;
  • of=: output file;
  • bs=1048576: tells dd to copy 1MiB sized blocks;
  • skip=1: tells dd to skip one block at the beginning (which size is defined by bs= above) because out partition starts at 1MiB. This is the most important part, you need to adapt it according to the partition offset you’ve found in step1.

Now you should have a copy of the raw partition as copy.img in your home folder.

Step 3: Mount the partition

That’s the easy part: hdiutil attach copy.img

You should now see a new “drive” on your desktop, this is your partition.