Hello, you are not logged in.  Login or sign up
Community >> Blogs
Search Toad World Search

Blogs
Toad and Database Commentaries

 Toad World blogs are a mix of insightful how-tos from Quest experts as well as their commentary on experiences with new database technologies.

Do you have a topic that you'd like discussed?  We'd love to hear from you.  Send us your idea for a blog topic.

Database Benchmarking Preparatory Checklist
 
Location: Blogs Bert Scalzo's Blog    
 Bert Wednesday, May 28, 2008 4:20 AM
There have been several papers and blogs from Quest on the issues of benchmarking successfully, however the condition persists where people experience trying issues with Benchmark Factory and obtaining satisfactory benchmark results. I still maintain that it’s primarily attributable to a lack of proper and complete precursory work. But people generally seem to need more than a notice to prepare more, so I’ve constructed the checklist below. While you may lack the privileges to define some of these, you better be able to answer or address all the questions/issues. Otherwise you’re simply increasing the likelihood of getting suboptimal benchmark results.
 
Basics
  1. Architectural diagram of the database server, network and IO setup
  2. Reviewed the official benchmark specification to fully understand the test – critical step!!!
  3. Defined goals for satisfactory benchmark performance (i.e. TPS, Average Transaction Time, IO throughput, CPU utilization, memory consumption, network utilization, swapping level, etc)
  4. Verified the assumed capacity of each and every hardware component (i.e. manually check to guarantee that each component properly configured to deliver expected performance results)
  5. Select a database benchmarking software tool (e.g. Benchmark Factory) to automate the chosen benchmark’s base object creation, data loading and concurrent user workload execution
  6. Select a database monitoring/diagnostic tool (e.g. Foglight, Spotlight on Oracle, SQL Server, RAC) to observe the effective database throughput and to identify potential bottlenecks or areas of concern
  7. Select an operating system monitoring/diagnostic tool (e.g. Foglight, Spotlight on Linux, Windows) to observe the effective OS throughput and to identify potential bottlenecks or areas of concern
  8. Select a database performance resolution/corrective tool (Foglight, Performance Analysis for Oracle, SQL Server, or SQL Optimizer) to proactively advocate potential work-around or corrective actions to improve overall performance
Storage
  1. Number of storage arrays being used
  2. Are the storage arrays virtualized or shared
  3. Storage array nature (i.e. SAN, NAS, iSCSI, NFS, etc)
  4. Storage array connective bandwidth per storage array and total
  5. Amount of cache memory per storage array and total
  6. How many spindles per storage array and total
  7. Number of processors per storage array and total
  8. Amount of memory cache per storage array and total
  9. Sorage array caching allocation settings read vs. write
  10. Storage array caching size/algorithm for read-ahead settings
  11. Nature, size, speed and cache of disks per storage array and total
  12. Number of LUN’s available for usage per storage array and total
  13. RAID level, stripe width and stripe size/length of the LUN’s
Servers
  1. Number of database servers being used (usually one, unless clustering or replicating)
  2. Are the database servers virtualized or shared
  3. Database server architecture/nature (i.e. uni-processor, SMP, DSM, NUMA, ccNUMA, etc)
  4. Database server CPU word-size and architecture/nature (i.e. RISC vs. CISC)
  5. Database server CPU physical count (slots) per database server and total
  6. Database server CPU logical count (cores) per database server and total
  7. Database server CPU speed and cache per logical unit (core)
  8. Hyper-threading turned off if it is available – critical, otherwise will negatively skew results
  9. Amount, type and speed of RAM per database server and total
  10. Number and throughput of HBA’s per database server and total
  11. HBA interconnect nature and speed (i.e. fiber, infiniband, 1GB Ethernet, 10GB Ethernet, etc)
  12. Number and throughput of NIC’s per database server and total
  13. Database server interconnect nature and speed (if clustering or replicating)
Operating System
  1. Operating system word-size
  2. Operating system basic optimization parameters set or tuned
  3. Operating system database optimization parameters set or tuned
  4. Disk array and inter-node Ethernet NIC’s set to utilize jumbo frames
Network
  1. Matching cabling and switch/router throughput to fully leverage NIC’s
  2. Disk array and inter-node Ethernet switches set to utilize jumbo frames
  3. Disk array and inter-node paths on private networks or private VLAN’s
Database
  1. Database word-size
  2. Database basic optimization parameters set or tuned
  3. Database optimization parameters set or tuned for specific benchmark
Benchmark Factory
  1. Using most recent Benchmark Factory software version available (i.e. currently 5.5.1)
  2. Using the best available database driver for that database platform (native vs. ODBC)
  3. Starting a number of agents = max total concurrent users for the benchmark / 900
  4. Place no more than four concurrent agents for the same test on a single app server
  5. Customize the Benchmark Factory project meta-data for your specific needs:
    • Partition or cluster tables
    • Partition or cluster indexes
    • Collect optimizer statistics
    • Collect performance snapshot (e.g. Oracle Stats Pack or AWR snapshot)
    • Run the workload
    • Collect performance snapshot (e.g. Oracle Stats Pack or AWR snapshot)

 

Permalink |  Trackback

Comment:
Add Comment   Cancel 
Search Blog Entries
 
Blogger and Topic List
 

 

All Recent Entries
 

 

Johannes Ahrends
Unicode

Steven Feuerstein
Oracle PL/SQL

Daniel Norwood
Toad for Data Analysts
John Pocknell
Toad for Oracle
Bert Scalzo
Toad for Oracle, Data Modeling, Benchmarking
Jeff Smith
Toad product family
Richard To
SQL Optimization
Jim Wankowski
DB2 - LUW and z/OS
John Weathington
Compliance
Doug Williams
Database Musings
  Henrik "Mauritz" Johnson
Toad Tips & Tricks on the "other" Toads
  Toad World Editor
Toad World issues

  Toad Data Modeler Opens in a new window
Data Modeling
 

Copyright 2008 by Quest Software  | Terms Of Use | Privacy Statement | Contact Us