In Part 1 of the series, I discussed the Power Draw of a Storinator on Startup and other conditions. Results showed that our Storinator’s Power Supply Unit (PSU) is well suited for the task at hand – starting and maintaining 45 HDDs and the other components.
However, not all drives are created equal. Some pull more current than others during start up and while the previous measurements showed the drives used are well below the maximum limit of the PSU, other drives may not fare as well.
If you have power-hungry drives, or are worried about a large start-up draw, don't worry – we have you covered! Our Storinators are able to reduce the initial current draw by staggering the spinup of the drives, with the trade-off of a slower boot time.
What is Staggered Spinup (SSU)?
When you turn your Storinator on, power flows to all the components instantaneously. This includes the motherboard, hard drives, fans and more – essentially everything that is attached to the PSU. Most of the components have a fairly constant power draw upon start-up, however, this is not the case for hard drives.
Due to the nature of how they work, the start-up power they demand to spin up the platters inside can be quite large compared to their normal operating power. This was shown in my previous blog post, start-up current draws peak around 3x steady state for the 12V and 2x steady state of the 5V.
If you have one or two hard drives, this isn't an issue. But if you are starting up a huge storage server with 30, 45 or 60 drives, this peak becomes important. In some cases, it is beneficial to reduce the load on the PSU by staggering the spinup of the HDDs so that they don't all spin up at once, but rather, start up with a small delay between each so that the startup surges don't all overlap.
Due to the nature of how they work, the start-up power they demand to spin up the platters inside can be quite large compared to their normal operating power. This was shown in my previous blog post, start-up current draws peak around 3x steady state for the 12V and 2x steady state of the 5V.
If you have one or two hard drives, this isn't an issue. But if you are starting up a huge storage server with 30, 45 or 60 drives, this peak becomes important. In some cases, it is beneficial to reduce the load on the PSU by staggering the spinup of the HDDs so that they don't all spin up at once, but rather, start up with a small delay between each so that the startup surges don't all overlap.
There are two ways to implement Staggered Spin Up:
1. Power Up In Standby (PUIS)
To enable this mode, the hard drive is literally set to power in standby. Once they are set to this mode, they will not start until the HBA issues a startup command to each drive. This is a very simple way to implement staggered spinup, as it does not require any special or non-standard hardware connections.
PUIS can be enabled/disabled using the HighpointWebGUI. This is the recommended way as it is the most simple and you cannot hurt your drives. (It can also be enabled/disabled using a Linux tool called “hdparm.” It should be noted that hdparm is for advanced users and can seriously break your drives, so make sure you know what you're doing if you use it!).
PUIS mode requires HBA controller support as well as HDD support to use. If you intend to use staggered spinup, please check our wiki to make sure you choose compatible drives!
One final note on PUIS - when you set it up, it is important to consider that your drives have been programmed to do absolutely nothing until they receive a startup command from the controller. This is exactly what you want. But if you move the drives to another machine with an ordinary controller that doesn't issue startup commands, the drives will appear to be dead.
There's a simple fix - just plug the drives back in your Storinator, start the HighpointWebGUI and disable staggered spinup. This will reprogram your drives to startup on power up. Problem solved!
2. Pin 11 signalling
In Pin 11 signalling, hard drives do not start until they receive a simple digital signal on Pin 11 of their drive connector. This sounds simple, but it requires three things: an HBA controller that is capable of sending such a signal to each drive, hard drives that are pin 11 enabled and a connection between the HBA and each drive's pin 11. This is done via a conductor in the hard-drive cable, or a trace on a backplane, which, unfortunately, are not commonly implemented.
Pin 11 of the SATA power specification controls whether SSU is enabled or not. When power is initially applied to the drive, the controller/backplane senses whether Pin 11 is left floating (high or '1' logic state) or grounded (low or '0' logic state).The drive spins up when Pin 11 is grounded. When left floating, the drive will wait to spin up until it is spoken to by a SATA controller or SATA RAID controller.
Pin 11 of the SATA power specification controls whether SSU is enabled or not. When power is initially applied to the drive, the controller/backplane senses whether Pin 11 is left floating (high or '1' logic state) or grounded (low or '0' logic state).The drive spins up when Pin 11 is grounded. When left floating, the drive will wait to spin up until it is spoken to by a SATA controller or SATA RAID controller.
How We Do It
Storinators require the Power Up In Standby (PUIS) method of staggered spinup. Our HBAs don't support pin 11, and also, there is no connection to Pin 11, as it is on the power connector which does not connect to the HBA or any other form of startup controller. Plus, since we've gone with a proven direct-wired design for our Storinators, Pin 11 signalling has become incompatible with our storage pods.
Measurement Setup
I used the same exact setup as in my previous post, using a clamp current meter (MTP iC-410 AC/DC) and measuring first the 12V lines and then the 5V lines that ran to the HDDs. The output of the current meter was fed through a signal conditioning circuit and then observed on an oscilloscope. To generate the plots in this post, a python script was used to pull the data off the scope and was saved for analysis.
45 Seagate 4TB (Model# ST4000DM000) drives were used in a RAID0, in a standard Storinator S45 pod. Our PSU is a Zippy 950W N+1 3 module redundant supply with active-active components.
Results
Here is the plot of the HDDs current draw from the moment I press the power button, to moments after the login screen appears. The data has been processed to display both the 5V line (Red) and the 12V line (black) superimposed on the same plot.
 |
| Staggered spinup current draw of 45 HDDs from the moment the power button is pressed to 125 seconds after login. The black trace corresponds to the current draw on the 12V line and the red trace is current draw on the 5V line. |
Rather than discussing the 12V line and then 5V line, let’s walk through the plot and discuss what happens with respect to time.
For the first 62.5 seconds, the 5V line dominates pulling 16A. This is approximately equal to the 5V current draw in regular (non-staggered) start up. And, because the platters aren’t spinning yet, there is minimal load on the 12 V line, with 45 HDDs pulling a sum total of only 1A.
Around the 62.5 second mark in the section marked "CARD 1", the first R750 card starts spinning up the 24 HDDs attached to it. You can see 5 distinct peaks occuring as current trends upwards. This is occurring as groups of drives are started and suck current heavily for a brief period before falling to idle current. At the same time, 5V line falls linearly from 16A to 12.5A. It takes ~90s to start up the 24 HDDs on card 1.
By the time we reach 160s, at the start of the section marked "CARD 2", the first card has finished spinning up the HDDs attached to it. The second R750 card starts spinning up its 21 drives. You can clearly see it follows the exact same pattern as the first card, until all drives are started.
At 285s, you can see an instantaneous spike of current of 22A and 16.5A on both the 5V and 12V line respectively, as the pod finishes booting, and then at 325s you can see where it gets to the login screen.
The rest of trace is me logging in, and then letting the pod sit at the desktop.
Comparing Startup Amperage Between Standard and Staggered Spinup
Comparing Startup Amperage Between Standard and Staggered Spinup
As a bonus, I placed the start-up plots with and without staggered spinup on the same graph to further show how much the initial surge is reduced by using staggered spinup (SSU). The 12V line without SSU is blue and the 12V line with SSU is black.
Note that without staggered spinup, there is a peak power draw of nearly 50 amps, in the first seconds of startup.
With staggered spinup, the peak draw during the hard drive startup sequence is less than 15 amps, which is hardly distinguish from steady state. This is a really dramatic improvement. This is truly an enabling feature that allows us to use any power-hog hard drives in a Storinator.
 |
| Comparison of the 12V line under normal startup conditions (Blue trace) and in staggered spinup (black trace). Note the peak current draw is drastically reduced when using staggered spinup. |
Note that without staggered spinup, there is a peak power draw of nearly 50 amps, in the first seconds of startup.
With staggered spinup, the peak draw during the hard drive startup sequence is less than 15 amps, which is hardly distinguish from steady state. This is a really dramatic improvement. This is truly an enabling feature that allows us to use any power-hog hard drives in a Storinator.
So there you have it – a comparison of starting a Storinator up with and without staggered spin up. My first post showed you that staggered spinup is not necessarily needed to start up a Storinator with 45 desktop HDDs, and this post showed you that if you encounter drives that are real power suckers, you can reduce your start up current draw by enabling staggered spin up!
I couldn’t say that last line without reminding you again of the disclaimer that not all HDDs on the market support the Power Up In Standby method of staggered spin up. Our wiki has a full list of compatible HDDs.
Conclusion
· Using the method of Power Up in Standby, the Storinator is able to stagger the spinup of the attached HDDs. The other method (pin 11 signalling) of SSU is incompatible with our pods since we no longer use backplanes.
· Staggered spinup greatly reduces the large start-up current draw on the 12V line, but comes at the cost of longer boot time. This is summarized in the following table, where the steady values for the 12V and 5V lines are 14A and 8A respectively. These steady state values are the same regardless of what startup you are using.
Normal Spinup
|
Staggered Spinup
| |
12V Current Draw (peak)
|
3x SS
|
1.57 x SS
|
5V Current Draw (peak)
|
2x SS
|
2x SS
|
Boot time
|
100s
|
325s
|
· It is no surprise that the peak start up current draw on the 5V is the same for both start-up conditions. The 12V line sees a significant decrease in current draw, peaking a mere 1.57x SS. The pod takes 3.25x as long to boot when in staggered spinup, but these pods are meant to be run for long periods of time with minimal downtime so longer boot times don't interfere with most applications.
What are your thoughts on staggered spinup? Sound off in the comments!
Continue on to Part 3 of this Power Draw blog series, Power Draw of Enterprise-Class Hard Drives Versus Consumer-Class Drives
No comments:
Post a Comment