Ext3 users - Aug 2013 - Re: ext3 / ext4 on USB flash drive?

On 8/29/13 10:46 AM, Theodore Ts'o wrote:
> So this is fundamentally a problem with the quality of the hardware,
> and that's not something the file system can really compensate for.
> And there's no way to tell whether a particular USB device has has a
> high quality flash device, or is a craptastic flash device.  It's not
> like we can query the device for "I ripped off the purchaser"
bit.  :-)
Yes, this is the fundamental problem.

What's the best setting for the random blob of silicon on the end of
your usb plug?  e2fsprogs can't call tech support, even to get bad
answers.

We'd like to help in software, but we can't; we have no reliable way
of knowing most of the necessary details of this class of hardware.
It's not exported in any way.

So unfortunately we are as in the dark as you are in this case.

-Eric

> We'd like to help in software, but we can't; we have no reliable
way
> of knowing most of the necessary details of this class of hardware.
> It's not exported in any way.
>
> So unfortunately we are as in the dark as you are in this case.
This is incredible, Mr Sandeen. You mean USB flash manufacturers
(what's their body - the USB Implementer's Forum?) have simply not
provided a means for software to query the underlying hardware in a
USB flash? Have software producers asked them for this?

mb.

On Fri, Aug 30, 2013 at 10:56:17AM +0100, Mark Ballard
wrote:
> 
> This is incredible, Mr Sandeen. You mean USB flash manufacturers
> (what's their body - the USB Implementer's Forum?) have simply not
> provided a means for software to query the underlying hardware in a
> USB flash? Have software producers asked them for this?
No, they haven't.  And yes we have, since there are some things (such
as the erase block size) which would be useful for tuning file system
performance.  And the technical people I've talked to at various Flash
manufacturers all agree it's pointless to hide this information, but
the product managers tend to be the roadblock.  If you are buying
several million eMMC flash devices for a mobile handset (i.e., for an
Android device), you can find out this information, under NDA.  But
otherwise, the only way you can find it out is by carrying out timing
attacks against the flash device.

Even if they did provide this sort of information, there are lots of
things which are not quantifiable.  Cars don't have interface for
telling you whether they have the crash protection of a Ford Pinto, or
a Tesla S.  Part of this is because product managers aren't going to
want to advertise that they have only provided 512MB worth of flash on
a device which is labelled as containing 8GB (and which will claim to
contain 8GB even if you query it via software; it's just that the
512MB + 4k write will cause some random block to disappear).  But the
other part is that there isn't a good single metric for "crash
safety"
(crash safety against head-on collisions?  crash safety versus side
impact collisions?  etc.)  Similarly, even for a non-fradulent USB
stick or SD card, there is no single way to measure "FTL quality".  An
FTL which is optimized for use in digital cameras (where writes are
usually sequential, and for large jpg images) may be horrible for use
a general purpose root file system.

There are sites that will do technical analysis and ratings for
various flash media (such as www.anandtech.com).  For example a quick
google search turns up:

http://www.anandtech.com/show/4523/usb-30-flash-drive-roundup

These are going to be the pricier devices, though.  They aren't going
to be the five dollar crap that you find at the checkout counter of a
computer store.  But then again, you get what you pay for....

	 	     	  	     	 - Ted