22 May 2006 - 08:31Benchmarks: Round One
We got intrigued by the benchmarks published by Neil Wilson comparing a Dell server to Sun's new T2000 servers. Then Sun offered loaner T2000 units to application developers. We jumped at the chance.
Connexitor™ Directory Services (CDS) (Symas's certified, tested, and natively installable binary distribution of OpenLDAP performed 19% to 31% more user authentications a second (faster) than Sun Java™ System Directory Server (JSDS) on the T2000. We ran the same workload on an HP Integrity dual Itanium 2 server and saw 27% to almost 32% further increase in authentications per second. A SunFire 4100 dual Opteron system showed another 26% to almost 30% performance improvement on the same workload. The Sunfire 4100 running CDS was just over twice as fast as SJSD on the T2000 (and sneaking up on three times as fast as SJDS on the Dell unit).
What we learned in school today? These Sun and HP servers can casually handle authentication workloads in excess of one million user entries (with RAM, in excess of 5 million or more because OpenLDAP's performance has been proven to be relatively flat into the tens of millions given enough RAM) and at authentication rates indicative of the loads seen in large campus environments. What was a big scary directory a couple of years ago can now be run well on solid commercial offerings.
Read on for the rest of the story.
Please refer to Neil's excellent blog entries here and here for the beginning of the story. His professional, detailed and thoughtful discussion goes into much more depth than we will here. We followed his lead as best we could. For the performance mavens, slamd data from the runs is online at SLAMD server (thanks also for that, Neil, he's the author of SLAMD!). Please refer to the View Optimized Jobs results under the CDS 3.3.2 folder. We used the Re-Run Iteration Value under Execution Data as the consistent data point.
Symas does not have access to the Dell PowerEdge™ 6850 used in the original Sun benchmark. We were loaned a SunFire™ T2000 under the loaner program. We have an HP Integrity™ rx2620-2 with similar RAM and comparable disk capacity in our labs. We were able to run on a SunFire 4100 dual 2.8GHz Opteron system in another lab to round out these runs.
The T2000 and HP Integrity came with 16GB of memory (RAM). Neil had double that (32GB) on the system he used for his measurements. It just wasn't worth investing in upgrades for the short duration of the benchmarks, so the ten million user runs weren't feasible. The Sunfire 4100 has only 8GB which was worse. We ran at half and one quarter the number of entries for our largest runs, to compensate. There is enough question about whether the results are truly comparable but as indicators they show a consistent pattern. Those runs are intended to show what happens when the directory gets resource (memory) starved to some extent and it follows a pattern that makes sense to us. The data for these larger runs has gray backgrounds in the table and probably shouldn't be used for performance analysis without further study.
The following table extends (by reference) the table on Neil's second posting. All the results report performance of CDS 3.3.2 (based on OpenLDAP 2.3.21 plus some additional fixes). At least one of those fixes is yet another Howard Chu amazing performance improvement fix that is very much in the Open Source libraries.
| System Type | Operating System | Number of User Entries | Average LDAP Authentications per Second |
| SunFire™ T2000 | Solaris | 250,000 | 8770 |
| SunFire™ T2000 | Solaris | 1,000,000 | 8800 |
| SunFire™ T2000 | Solaris | 5,000,000 | 7408.885 |
| HP Integrity rx2620-2 | Red Hat Enterprise Linux 3 (RHEL3) | 250,000 | 11402 |
| HP Integrity rx2620-2 | RHEL3 | 1,000,000 | 11089 |
| HP Integrity rx2620-2 | RHEL3 | 5,000,000 | 3924 |
| Sun Sunfire 4100 | Solaris | 250,000 | 13843 |
| Sun Sunfire 4100 | Solaris | 1,000,000 | 12987 |
| Sun Sunfire 4100 | Solaris | 2,500,000 | 10948 |
Note: Authentication rates rounded to nearest integer.
We're not in the business of sorting out the prices and power levels and all. We'll leave comparisons between the Sun and HP boxes on price, price performance, power, and all to others more qualified. Specs change so quickly, these are probably only useful for indicative purposes anyway. We highly recommend benchmarking your workload on the systems you're considering if you have performance concerns. With slamd it is not anywhere near as hard to do as it was some time ago.
Bottom line: the SunFire T2000 is a great box. The performance on the Sun software is very respectable but, as we've come to consistently find out, OpenLDAP has evolved quicker and appears to be consistently faster. This benchmark is as closely comparable as we can imagine short of back to back runs on exactly the same machine. Without knowing the price and power data, we suspect that there is plenty of room for both Sun and HP to crow about their respective servers but Sun definitely has made a major step forward.
Looking ahead, these are good sized departmental servers and clearly demonstrate that that class of machine can handle most production Directories outside of telecommunications and finance where directories of tens of millions of entries are common. These systems can clearly run much bigger numbers of User Entries or any other type of data if configured with enough memory (RAM) to provide sufficient cache space.
... Marty
five comments:
An interesting write-up. It would be good to add the CPU frequency of the rx1620 as there are two possibilities – 1.3 and 1.6 GHz and the 1.6 GHz has a faster FSB. Similarly, the frequency (1.0 or 1.2 IIRC) and number of cores (6 or 8?) on the T2000 would be a good thing to include.
FWIW, Linux on Itanium is strictly 64-bit. If you want a 32-bit native Itanium application, one needs to be running HP-UX
Just for reference, the rx2620 has two 1.6GHz CPUs. The T2000 has the identical CPU as used in Neil Wilson’s tests, 1.0GHz with 8 cores.
All of the software is 64 bit on all the platforms; a pure 32 bit server would thrash its cache above about 1 million entries.
http://www.levelvip.com http://www.saleveling.com http://www.power-leveling-game.com
adfa
[url http://www.adasd.com]adf[/url]]
sadf
One or more comments are waiting for approval by an editor.
No trackbacks: