Tuesday, June 16. 2009
As promised in the previous post this is the second part in a series of testing/benchmarking 8.4 under various circumstances.
Continue reading "Benchmarking 8.4 - Chapter 2/bulk loading"
Friday, June 12. 2009
Computing platforms are constantly changing, evolving and improving. This holds true for both the hardware and the software running on that hardware.
Continue reading "Benchmarking 8.4 - Chapter 1/Read-Only workloads"
Tuesday, December 11. 2007
With 8.3 just around the corner more and more people are actually starting to test 8.3 with their code base and wondering if it will be worth to switch/upgrade and so did I.
All the testing is done on a DL380 G5 with two Quadcore Intel Xeon CPUs (X5345 - so 2,33Ghz per core) 12GB of RAM and 6 disks in a RAID1+0 with 512MB BBWC on a Smartarray P400.
The benchmark database is initialized with a scaling factor of 100 (equals to 10M rows) which seems to be a reasonable size for a table in an OLTP style database) all testing was done with 100 clients and 100000 transactions/client which comes out to 10M transactions and an average runtime of about 1,5 hours.
what we can see here is that 8.3B4 is about 2.2x faster than 8.2 out-of-the-box for this very update heavy workload and is able to keep that advantage after similar tuning is applied to both instances.
the second thing I tried to test was the effect of various shared_buffer settings on the transaction rate(the other parts of the configuration stayed the same as in the "tuned" variant above):
there are two main things to note here - one is the obvious one that the scaling on the x-axis is not linear.
All in all 8.3 seems to become a rather impressive release and as always proper tuning is very critical to getting the best out of (dedicated) database server instances.
Monday, April 30. 2007
The discussion on using SAN vs. DASD based storage is nearly a religious war(as can be seen in a lot of discussions on pgsql-performance) and in many ways similar to the infamous emacs vs. vi debate.
However - for some workloads those types of SAN are not really that appropriate. A DS4300(which is a now withdrawn from marketing) can do only a bit above 100MB/s of seq IO(nearly independent on the number of disks!) per controller(about 135MB/s if used together) which is really not much when one considers how fast modern hard drives are.
I recently got a SAN Array to play with that looks quite interesting since while expensive it still seems reasonably priced compared to what companies like IBM or others want for similar gear.
The array I got for testing is basically a non-branded LSI/Engenio 3994 with 16 2Gbit 10k 146FC drives and 2GB of battery backed cache per controller.
Due to the use of disks that are only capable of 2Gbit/s, the speed of the two drive channel loops is also limited to 2Gbit/s (using 4GBit FC drives it can be configured to use 4Gbit/s on the drive channels).
In the following test(test case 1) we use two volume groups - each a RAID10 (8 disks) and a RAID0 in the OS and write cache mirroring between the controllers(keeps both controller caches in sync so in case one controller fails the other one can take over). To utilize both controllers the HBAs are set up so that controller A is using on and controller B the other.
and the same with write mirroring disabled for both logical volumes (test case 2):
so write mirroring seems to have a 20% penalty for sequential writes and rewriting but not much impact for others - so it might be worth keeping it turned on due to the additional data integrity guarantees it provides .
and now for comparison a test using only volume group and a single controller (test case 3):
so let's see what PostgreSQL is able to do in terms of sequential IO on such device:
simple sequential scan on a large table (pgbench schema generated with a scale of 10000) using only a single controller (same setup as in test case 3):
so we are getting about 215MB/s out of 250MB/s which looks ok.so what happens with software raid 0 over two 8 disk RAID10 volume groups on different controllers (same setup as test case 1):
so that is more interesting - it seems that PostgreSQL is getting CPU bottlenecked(the array/file system can do >370MB/s) here and those ~280MB/s are pretty much in line with what Luke usually quotes (PostgreSQL getting CPU bottlenecked at around 300MB/s even on very fast AMD Opteron based boxes).
for those curious here are some other random tests (uncommented so judge by yourself):
single raid 5 with 4 logical volumes (each 500GB) and software RAID0 in the OS - two volumes per channel
A single RAID5 array over all 16 disks and two identically sized logical volumes each around 1TB in size.
on one LUN:
using both LUNs and software RAID0:
with disabled write cache mirroring:
Thursday, March 29. 2007
while trying to put new hardware into production today I found that the box (running Debian Etch/i386 with a 2.6.18 based kernel) would start to drop network connections during large transfers at Gigabit speeds.
The nic in question is an embedded Intel 82573E/L on a Supermicro PDSM4+ with the latest BIOS-Update available (1.2):
A fair bit of research turned this small shellscript up. This script basically greps the output of ethtool -e interface (which in itself dumps the eeprom contents) and flips a bit in the eeprom:
after the reboot the nic just works fine - no more stalls and transmit timeouts ...
(Page 1 of 1, totaling 5 entries)