03 December 2007 - 05:08AD and ADAM Performance: UPDATE
OK, this is the update promised a few entries ago. This is a reworked version of the previous post with corrected numbers and a couple of graphs. Enjoy.
As promised in the white paper, Symas's benchmarking team has been diligently benchmarking OpenLDAP (the new 2.4.6 release that just came out of beta ... plus a little pixie dust from HEAD that will be in 2.4.7) against both Active Directory (AD) and Active Directory Application Mode (ADAM) directory technologies from Microsoft. This is being done on an AMD Celestica A8440 with four Opteron 875s (dual 2.2GHz cores) and 16GB DDR333 RAM. (By the way, big thanks to our friends in AMD's Open Source Support team who arranged to make this machine available for these tests!The Barcelona recently arrived so we have a hot new toy!) All the runs were done using 64-bit mode.
Punchline: at 5,000,000 (5M) entries, OpenLDAP 2.4.6+ delivered 20,179 authentications a second, ADAM delivered 5,424 and AD did 216. The 1,000,000 (1M) entry runs were 23,273 for OpenLDAP, 5,738 for ADAM, and 4,662 for AD.
Howard's nice charts will be added later.
OpenLDAP benchmarks were run on Linux (we haven't got the Windows tool chain working for 64-bit Windows at the moment ... OpenLDAP performance on Windows versus Linux is a topic for another time.) It took 3 minutes and 9 seconds to load the 1M entry directory with slapadd (the off-line load tool) using the default back-hdb backend.
On the same machine using a hard drive partition with Windows 2003 SP2 Enterprise Edition for x64 (x86_64), we installed AD configuring it as recommended in this Microsoft Document. Then we loaded one million users in a sneech under four hours, twenty-four minutes (something over thirty-seven times as long as OpenLDAP). Finally, after a number of runs looking for the absolute best tuning, the best we could do with AD was 4,661 authentications a second (almost exactly 1/5th the OpenLDAP rate). AD delivered 20% of the performance of OpenLDAP 2.4 (or, it takes about five times as much capacity to deliver equivalent performance).
We then loaded up the 1M entries database into ADAM in twenty-seven and a half minutes (just over 27:33) and ran the authentication rate slamd job against it. Best we could do was 5738 authentications per second using 52 client threads. That's better than AD's 4,661K/second. That's 25% of OpenLDAP's rate or it only takes four times the processing resources to deliver equivalent performance with ADAM.
In all tests, AD and ADAM require substantially more RAM (ADAM needed 14GB of RAM for the 5M test to OpenLDAP's 8GB ... we expect AD to be even worse). So, when considering the cost differential of supporting two loads, make sure you take that into account as well. It is a significant element in the electrical requirements even if it doesn't necessarily change the machine-room volumetric requirements.
|
|
Next we look at the amount of RAM the process needs. Note that the system was only able to give AD and ADAM 15.5GB as best we could measure. The performance of AD was impacted quite strongly at 5M entries as there simply wasn't enough RAM for AD to cache the database and the I/O drove performance down substantially.
| 1M Entries | 5M Entries | |
|---|---|---|
| OL hdb | 4.0GB | 14.5GB |
| MS ADAM | 6.0GB | 14.8 - 16.0GB |
| MS AD | 10.0 - 11.0GB | 15.5+GB |
By the way, we're now using SLAMD 2.0alpha1 instead of SLAMD 1.8.2 which we used on last year's tests. Unfortunately we've had to remove the old SLAMD server that used to answer on http://rebec.symas.net:8080. The reports and detailed statistics for the current tests are available here. As always, we will provide complete copies of our configurations, data, and tuning notes for anyone wishing to independently reproduce these results.
Jordan has approved these numbers. No snowmen were harmed in the production of the results.
two comments:
Speaking of hot new toys… the new box is a Supermicro AS-2021M-UR+B with a pair of Opteron 2347s (1.9Ghz quad-core chips) and 16GB of DDR2 667 DRAM. This system loaded our 1M entry DB in about the same time as the other (3minutes, 5seconds) but delivered over 28,000 authentications/second on a 2.6.18 kernel. With an updated forcedeth ethernet driver it’s now yielding 29,000 authentications/second with OpenLDAP 2.4.7. Quite an impressive showing, considering that its core clock is 14% slower than the Opteron 875 system.
This benchmarking is tricky business. As noted in the scaling tests
http://connexitor.com/blog/pivot/entry.p..
slapd’s performance was higher when only using 7 cores, while we used all 8 cores for the tests posted here. As such, comparing peak to peak, OpenLDAP would actually be 6x and 5x faster than AD and ADAM at 1M entries. (Yes, belaboring the point a little…)
No trackbacks: