Storage In A Flash; Choosing Disk Drives For Hosting Environments

Like all providers of Internet services, ProcoliX strives to deliver the fastest experience to end users and consumers of the applications our customers place upon our infrastructure. It’s very rare now for an owner of an online service to become directly involved with the hardware powering their offering, so a significant part of our work alongside ensuring secure datacentres and fast connectivity comes in optimising the servers themselves.

The performance of a server is affected by the choice of its components, but while the memory and processor choices are relatively straightforward ones governed by the paper specifications of the day, there are many more to be made when it comes to storage hardware. In short, the choice of disk drives has a huge impact on the performance of the machine.

Slow And Cheap or Fast And Expensive: It’s Your Choice

The workings of a traditional spinning hard disk. Eric Gaba, CC BY-SA 3.0.

When it comes to disk storage there are two main options. A traditional drive has a spinning magnetic disk with a set of read and write heads on a moveable arm that tracks across its surface. It’s a decades-old and well-proven technology that’s also relatively inexpensive, but it comes with the penalty of not being the fastest storage medium. Retrieving a file requires those heads to mechanically move to whichever tracks hold its information, thus all that head-moving adds extra time to any operation and slows down the computer. Thus it’s best suited to long-term storage of files which don’t need to be read very often, rather than high-performance access to files that are in constant demand.

The silicon die of a NAND Flash chip. The memory itself has the rainbow sheen, the T-shaped atea is the address logic, and the on-board controller is a small rectangle in the top left hand corner. 2×910, CC BY-SA 4.0.

The other choice is the solid-state drive, in which the storage medium is comprised of non-volatile Flash memory chips. These are semiconductor devices in which individual bits are stored in the form of stored charge in insulated transistors. The advantage of a Flash disk is that it is extremely fast because it doesn’t have any moving heads to introduce delays, but it also has the disadvantages of increased cost and so-called Flash wear, in which repeated erasure cycles can cause individual cells to fail. Flash wear is a significant problem, and as a result every Flash drive from the most humble of micro SD cards to the highest-spec server SSD will take steps to avoid it using a technique known as wear levelling.

A Flash drive will have a stated capacity, for example a gigabyte. You as the end user can store a gigabyte of data on it, but what isn’t in the marketing for the drive is that it really contains more than its stated capacity in Flash memory cells. An onboard microprocessor implements a wear levelling algorithm, in which the number of erase cycles for each block is monitored and the data moved around the disk to ensure that no one area of it receives more than its fair share of cycles. The extra Flash cells serve to provide extra capacity as failed blocks or blocks that have received too many cycles are retired from use, so that the user still sees their gigabyte of space without being aware of all the work behind the scenes.

Why Cheap Flash Drives Don’t Cut It In The Server Room

The efficiency of the wear levelling and the amount of extra Flash cells included on the disk are dependent on its specification and ultimately its price. A cheap SD card for example will have the minimum possible overhead and is surprisingly easy to destroy with repeated reading and writing. We’ve managed to do this to SD cards in Raspberry Pi single board computers on the bench in a matter of months as the Raspberry Pi OS Linux distribution makes heavy use of its disk, proving of course that a card sold for occasional use in a camera or media player isn’t a good choice for such an application. For this reason the Flash drives sold for use in laptops and desktop computers have much more robust wear levelling and a larger number of extra Flash cells and will last for the several years the computer will typically be in use, and the drives sold for servers will have yet more spare Flash to cope with heavy loads.

Here at ProcoliX we make extensive use of Flash drives for their speed of access, particularly the high-spec models designed for use in server environments. We have found that they have exceeded our expectations; in any large array of disks it is normal for some failures to occur but by careful selection of hardware we have found our flash arrays to be extremely reliable. This piece was prompted by the impending replacement of an array installed for a particular customer requirement that has been operating faultlessly for a decade, a testament to the its designers and the efficiency of its wear levelling algorithm.

The machine you are reading this on may well already have a Flash drive as here in 2021 they are an easy route to higher performance. We hope you now appreciate some of the technology behind that speed.