Why 45MB Unallocated space at the beginning of a Silicon Power Marvel M70 64GB Flash Drive AND Why Drastically different sector counts?

Question

I have a brand new Silicon Power Marvel M70 64GB USB flash drive has 45MB of unallocated space preceding the 58.89GB of available space. I don’t care about having less than 64GB since I know it’s due to 1024 based math.

What concerns me is within the size of 45MB. Why so much? 1MB is typical for alignment.

So, I loaded up a hexeditor in Linux with this command:

hexdump /dev/sdb -C | less

And I find “strings” of data like the following:

Invalid partition table. Error loading operating system.Missing operating system.

Remove disks or other media. Disk error. Press any key to restart. U.RRaA.

This program cannot be run in DOS mode.

CpaintDC. UserException. CResourceException.

An application has made an attempt to load the C runtime library incorrectly. Please contact the application's support team for more information. Attempt to use MSIL code from this assembly during native code.

That is within the UNALLOCATED 45 MB area... Shouldn't it be zeros straight from the factory? Furthermore, an identical drive (see below) has completely different sizes for the unallocated and partitioned area; it seems executable data is or was stored there

I’ve never encountered this before. Has anyone else? I called the company and they didn’t have an explanation only to tell me that upon opening one and connecting it to the computer theirs also shows 45 MB of unallocated space at the beginning of the drive as well.

That indicates that it’s probably done like this at the factory and not just mine has this configuration.

I am concerned about what is stored there; and is it possibly executable, etc.. especially with seeing the hexdump strings of text above from within the 45 MB of “unallocated space.” If it is executable; I would assume perhaps it is accessible when booting to the drive.

Anyone seen this before? Any explanation you can think of?

EDIT: When I called the company that makes it they also plugged in a 128GB size of the same model line and the unallocated space was 31 MB. Not sure if that has any bearing on what might be going on here, but it's certainly not “proportional” or at the very least the same.

EDIT: I have subsequently tried another brand new unit (same model, batch number, size (64GB) drives. As displayed by GParted:

• Drive #1: 45.33 MiB Unallocated and 58.89GB FAT32

• Drive #2: 46.38 MiB Unallocated and 54.70GB FAT32

These are both supposed to be 64GB drives. Identical in size, etc. This is straight from the factory. I have never encountered this kind of discrepancy with other flash drives. I mean, that’s over 4GB less of storage space between the two drives in total sector size.

As displayed by "fdisk -l" as requested (for the #2 drive only since I already cleared out the partition table of drive #1):

Disk /dev/sdc: 58.8 GB, 58787364864 bytes
90 heads, 26 sectors/track, 49067 cylinders, total 114819072 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x75cbf5af

Device Boot: /dev/sdb1
Start: 94976
End: 114819071
Blocks: 57362048
Id: c
System: W95 FAT32 (LBA)

Also, this doesn't explain why there is 45MB or 46MB of unallocated space prior to the first partition, but for 2 identical 64GB drives when I run a "dd if=/dev/sdb of=/flash_drive_dd.img bs=1M" command (which should give me an indication of the entire storage capacity of the drive right?) I get two completely different numbers:

Drive #1 displays 63,283,658,752 file size

Drive #2 displays 58,787,364,864 file size

Those values are for the full drive... not just a partition of supposedly identical model 64GB drives.

Regarding the possibly related drastically different sector count on two identical devices (Perhaps this should be a different question post?) Is Overprovisioning possibly occurring ---> In light of realizing that two idential 64GB drives show wildly different available sector sizes (one coming nowhere near close to 64,000,000,000 MB) Does that mean one is overprovisioning or something while the other isn't?

The way I figure out what I should see in my OS is the following: I take 64,000,000,000 bytes and divide by 1024 ^ 3 and I SHOULD get 59.6 GB being reported by the OS (or very close).

Even if there is some "overprovisioning" (is there such a thing for USB flash drives like for SSD) at least I would expect that to be consistent and not wildly different sizes between two identical model drives.

But, fresh out of the box:

Drive #1 only has 63,283,658,752 sectors for the entire device which is 58.9G. If I've done my math correctly; I'm missing over 700 MB for this drive.

And

Drive #2 only has 58,787,364,864 which is 54.8G And again, if I've done my math correctly, I'm missing 5.2 GB for the second drive.

That's disconcerting.... identical models with WAY less space (for the entire drive) than I would expect... the second drive being way less in size than the other.

Even it was overprovisioning of some sort; I would think it should be consistent no?

That doesn't even explain the 45 or 46 MB worth of unallocated space at the beginning with seemingly executable data.

Things keep getting weirder and weirder unless I'm missing something fundamental about USB flash drives.

Sorry this is a little long... I keep discovering more things and adding as I go forward.

Answer 1

Some flash drives (and even external HDDs) have built-in software. These drives are made to look like 2 devices to the OS, the first is presented as a CD-ROM and the second is presented as a normal read/write mass storage device.

This is not partitioning as understood/done by the OS - it's done on a lower level - it's controlled by the microcontroller in the USB device.

Utilities from device manufacturers (often in Chinese) do exist to change this arrangement--and you have to use one compatible with the microcontroller--and they are difficult to find and use.

I'm betting this flash drive at one point was destined to have built in software on it, but the manufacturer changed their mind for whatever reason.

So the unallocated area probably was a fake "CD-ROM" area containing encryption or backup software from another SKU of drives this manufacturer (or possibly another manufacturer) offers.

The manufacturer probably had an oversupply of these particular devices or some other related economic condition, then ran a utility to reconfigure the flash arrangement and remove the "CD-ROM" partition, but didn't bother to wipe this section of flash before packaging.

You should run TestDisk on the unallocated partition and see what you can find!

Answer 2

Poor quality control practices. Memory is notoriously difficult to manufacture with any level of consistency. Generally ram is made, it is tested to see what size it is, and then the space is formated to appear as that size. As far the strings in the unallocated space, I am guessing that is from their quality control test before the product ships.

Answer 3

[sorry4necro?bumping] Hey mike 2000 ( "Will update this thread when I get a response" ) did you ever get response?

I have Kingston Hyperx Savage 64GB with same thing as you. There (https://www.kingston.com/us/support/technical/products?model=hxs3) is an answer:

The full capacity of my DataTraveler or Flash media is not available for storage. Why?

Some of the flash storage device's listed capacity is used for formatting and other functions and thus is not available for data storage, therefore you will not see the full capacity.

When a flash storage device is manufactured, steps are taken to ensure that the device operates reliably and permits the host device (computer, digital camera, PDA, etc.) to access the memory cells; i.e., to store and retrieve data on the flash storage device. These steps, loosely called "formatting," utilize some of the memory cells within the device and thus reduce the capacity available for data storage by the end-user.

Formatting includes the following operations:

Testing each memory cell in the flash storage device. Identifying all defective cells and taking steps to ensure that no data will be written to or read from a defective cell. Reserving some cells to serve as "spares." Flash memory cells have a long but finite lifetime. Therefore, some cells are held in reserve to replace any memory cells that may fail over time. Creating a File Allocation Table (FAT) or other directory. To enable flash storage devices to conveniently store and access customer files, a file management system must be created to allow any device or computer to identify the files stored in the flash storage device. The most common type of file management system for flash storage devices is the File Allocation Table (FAT), which is also used on hard drives. Reserving some cells for use by the flash storage device's controller, e.g., for storing firmware updates and other controller-specific information. Where applicable, reserving some cells for special features. For example, the specification for Secure Digital (SD) cards requires reserved areas to support special copy protection and security features.

End of quoted answer.

Now, I am interesting: Following that logic, flash drive with 128GB capacity would have 90 (double of 45) MB unalocated space, flash drive with 256GB capacity 180MB... and so on. I don't have that big flash drive, but will be interesting to know...

Answer 4

I'm a software support PC tech, as such, I'd go with the documentation and software that sometimes comes with such expensive drives, also checking the website for info on the product, it has a purpose for sure, you are just going paranoid mode about it.

The errors I found in your procedure: 1) You're using Linux to analyze Windows software. 2) those strings are from .exe Windows executable files and a small boot OS, if you boot from it. 3) probably you could delete all partitions and reformat it and you'll get all the available space. 4) that partition could also have UEFI(modern BIOS) information or executables. 5) Again, clearly you haven't read any information about the product, probably it says what software: Boot Operating System and/or utilities/tools it comes with.

I hope this helps you see the errors of your ways and consult a technician, if you don't want to read the documentation of a device, before using it; best of lucks.

Answer 5

The unallocated space will be used to replace memory cells which fail. Flash memory devices are manufactured with considerable excess, and when a cell fails, it can no longer be written to, but it can be read from, At that point, the controller on board will copy the memory content to a new, unallocated, cell, and then mark the old cell out of use.