Most HPC buyers look to get the most performance for the lowest budget. As mentioned in an earlier post 'Server Cost Breakdown By Components', processors are normally the most expensive component in a cluster, this is especially true for non-GPU / Co-processor accelerated clusters. Picking the right CPU is critical.
Here we compare the effective $/GFLOP over the entire E5-26xx v4 family, including server costs, power, network, and infrastructure costs over three years.
We compared the TCO for three different memory / storage / network configurations:
- Low-cost server with 64 GB of RAM, 240 GB SSD, and 1 GB Ethernet.
- HPC server with 128 GB of RAM, no storage, and Infiniband.
- High-end, virtualized server with 256 GB of RAM, 4 x 480 GB SSD, and 10 GB network.
Here are some key findings:
- Picking the wrong processor costs you as much as 30%.
- There’s no one size fits all processor. Depending on your memory, storage, and network requirements, different processors yield the lowest TCO.
- The E5-2630 v4 is a surprisingly good deal for some HPC applications.
- As general rule, the E5 2630 v4 is great for minimal memory/storage/network configurations, but as these requirements go up, the E5 2680 v4 starts to become more attractive.
- Intel is really good at pricing!
Figure 1: TCO comparison for three different configurations with various Intel E5 26xx v4 servers.
The following parameters were considered to calculate the Total Cost of Ownership (TCO) of the cluster:
- Total upfront cost of the servers, including memory, network, and storage.
- Upfront power and cooling infrastructure costs (@ $1.5/server Watt).
- Network cost per port outside of the server.
- Power consumed over 3 years.
- Fixed memory and storage per server, regardless of core count.
- Energy cost: $.10 /kW-hr.
- No software / personnel.
- Fixed LINPACK efficiency of 81%.
Configuration 1: Low-cost server with 64 GB RAM, 240 GB SSD, and 1 GB Ethernet
As the graph below shows, using the Intel E5 2630 v4 yields the lowest TCO of $6.32 per GFLOP, while the E5 2699 v4 yields the highest TCO of $8.81 per GFLOP for this configuration.
Figure 2: TCO comparison for a low-cost server configuration.
Configuration 2: HPC server with 128 GB RAM, no storage, and Infiniband network.
When optimizing for a more processor driven configuration with higher memory and network requirements, the E5 2680v4 starts to look more attractive than the 2630 v4. The 2680 v4 yields a TCO of $8.42 per GFLOP vs $8.53 /GFLOP for the 2630 v4.
It is interesting to note that the list price of the E5-2630 v4 is a mere $667 as compared to $1,745 for the 2680. However, the cost per GFLOP for each server built on this configuration is a lot closer ($5.31 for the 2630 vs $5.97 for the 2680), and what the 2680 loses on upfront server cost, it makes up in lower power use, which helps reduce both the upfront and ongoing costs associated with power and cooling.
Figure 3: TCO comparison for a processor intensive HPC server.
Configuration 3: High-end, virtualized server with 256 GB of RAM, 4 x 480 GB SSD, and 10 GB network.
When it comes to higher end configurations, the higher-end processors start to become significantly more attractive. For the given configuration with 256 GB RAM, 4 x 480 GB SSD, and 10 GB network, the E5 2680 v4 yields the lowest TCO of $10.14 / GFLOP. The E5 2697 v4 and E5 2690 v4 come in at close 2nd and 3rd with $10.22 / GFLOP and $10.23 / GFLOP, respectively. Whereas the lower cost processors such as the E5 2630 v4 and the E5 2620 v4 become significantly more expensive here with $11.17 / GFLOP and $13.47 / GFLOP.
Figure 4: TCO comparison for a high-end, virtualized server.
Picking the right configuration can be critical in optimizing the cost of a cluster. Learn more about the Minimus formula for cost savings and see a price comparison:
For a free TCO analysis of your application/ configuration, please contact us by clicking here.
 This is what a GRC customer would expect to pay. Traditional clients or collocation may be wildly higher.
 Given how reliable minimus servers are, personnel requirements are surprisingly small.