Benchmark Factory - Standard Benchmarks Forum

sglixman
Follow / 13 Feb 2017 at 12:13pm / Benchmark Factory
Latest post by on 17 Feb 2017 at 6:07am

DellBF fail immediately: Error obtaining max(t_id) from E_TRADE

Expand content
Hello, I am trying to start DellBF, but fail immediately: Error obtaining max(t_id) from E_TRADE Manually running: SELECT max (T_ID) FROM [Test-BF].[dbo].[E_TRADE]; Works great Any idea what...

Hello,

I am trying to start DellBF, but fail immediately: Error obtaining max(t_id) from E_TRADE

Manually running:

SELECT max (T_ID)
FROM [Test-BF].[dbo].[E_TRADE];

Works great

Any idea what can be the problem?

DB: MsSql

Complete Dell-BF message is:

TPC-E Job
Job Started
Started test TPC-E Transaction Mix
Checking scale factor...
The loaded scale factor(0) doesn't match the specified scale factor (1) for the TPC-E benchmark.
Scale factor check complete.
Agent(WIN-47G9THU8OBD) Error: Error obtaining max(t_id) from E_TRADE. Transaction "TradeUpdate Transaction" run failed by virtual user 38 in agent 2 of WIN-47G9THU8OBD

759 3 / Create an account to join the discussion.
Danny Torres
Follow / 1 Nov 2016 at 1:25pm / Benchmark Factory

We are now on Twitter!!! Contact a Technical Support expert today...

Expand content
Do you have a quick support question and short of time? Contact us Via Twitter @ QuestExperts and we will take care of you. #Jointheinnovation #WeareQuest Thanks

Do you have a quick support question and short of time? Contact us Via Twitter  @QuestExperts and we will take care of you. #Jointheinnovation #WeareQuest

 

 

 

Thanks

602 0 / Create an account to join the discussion.
snarahar
Follow / 6 Apr 2016 at 2:15pm / Benchmark Factory
Latest post by on 6 Apr 2016 at 6:47pm

BMF newbie question about how to specify where the database should...

Expand content
Hello I just started using BMF. The question I have is : How does one specify where the database files should be located? I am trying to compare the performance of two disk pools THanks Sanjay...

Hello

I just started using BMF. The question I have is : How does one specify where the database files should be located?

I am trying to compare the performance of two disk pools

THanks

Sanjay Narahari

Answered 1185 2 / Create an account to join the discussion.
dennis.carvalho
Follow / 29 Feb 2016 at 4:58am / Benchmark Factory
Latest post by on 1 Mar 2016 at 6:01pm

TPC-E benchmark test failing on deadlocks

Expand content
hi, I have been trying to to the TPC-E benchmark test for ASE 15.7 on RHEL 6.5. The process of creating tables etc. goes fine and almost everytime it fails after scale check on deadlocks. So it is not...

hi,

I have been trying to to the TPC-E benchmark test for ASE 15.7 on RHEL 6.5. The process of creating tables etc. goes fine and almost everytime it fails after scale check on deadlocks. So it is not even like halfway. I have selected single user load of 25, 50 and 100 and tables creation option of (create/load objects if objects don't exist)

Started test TPC-E Transaction Mix

Checking scale factor...

Scale factor check complete.

Agent(MYDESKTOP) Error: CT-LIBRARY Server Message #1205 - Your server command (family id #0, process id #221) encountered a deadlock situation. Please re-run your command.

Transaction "TradeOrder Transaction" run failed by virtual user 22 in agent 1 of MYDESKTOP

Test cancelled due to errors.

Any suggestions? or any settings I am missing out?

Unanswered 1211 2 / Create an account to join the discussion.
veronica.l
Follow / 20 Nov 2015 at 8:12pm / Benchmark Factory
Latest post by on 20 Nov 2015 at 11:57pm

scaling up TPC-E on MySQL

Expand content
I have no problem creating a scale factor 1 (1,000 customers) TPC-E database. But if I try a scale factor 10 (10,000 customers), then the 5 largest tables (E_TRADE_HISTORY, E_SETTLEMENT,E_TRADE,E_HOLDING_HISTORY...

I have no problem creating a scale factor 1 (1,000 customers) TPC-E database.

But if I try a scale factor 10 (10,000 customers), then the 5 largest tables (E_TRADE_HISTORY, E_SETTLEMENT,E_TRADE,E_HOLDING_HISTORY, and E_CASH_TRANSACTION) will fail with error "ODBC Error (S1000,4B6) - [MySQL][ODBC 5.3(a) Driver][mysqld-5.7.8-rc]The total number of locks exceeds the lock table size "

All of the respective  BAK tables are built fine. (so... building and loading the E_TRADE_BAK etc. tables was no problem).

My questions:

1. Any best practices regarding loading big tables from BAK with Benchmark Factory?

2. In particular, where are the LOAD statements? (I was considering customizing them for these tables)

Thanks!

Veronica.

1418 2 / Create an account to join the discussion.
tzvika.lemel
Follow / 7 Oct 2015 at 9:48am / Benchmark Factory

benchmark factory crashes when trying to create 20TB database

Expand content
Description Benchmark factory 7.2.0.208 Win 2012 R2 36 GB RAM Job: creating TPCC 20 TB on Oracle database using 10 agents and 100 virtual users after a while (day or two ) we get the follwing error...

Description Benchmark factory 7.2.0.208 

Win 2012 R2 36 GB RAM

Job: creating TPCC 20 TB on Oracle database using 10 agents and 100 virtual users 

after a while (day or two ) we get the follwing error :

   Problem signature:
Problem Event Name: APPCRASH
Application Name: BFactory.exe
Application Version: 7.2.0.208
Application Timestamp: 5551ea22
Fault Module Name: StackHash_1715
Fault Module Version: 6.3.9600.17736
Fault Module Timestamp: 550f4336
Exception Code: c0000374
Exception Offset: PCH_48_FROM_ntdll+0x000000000009177A
OS Version: 6.3.9600.2.0.0.272.7
Locale ID: 1033
Additional Information 1: 1715
Additional Information 2: 171508be98a89eed4326e51d59631f20
Additional Information 3: 2920
Additional Information 4: 2920f1a7211f6fffe4e72fb3b7d986e7

Read our privacy statement online:
http://go.microsoft.com/fwlink/?linkid=280262

If the online privacy statement is not available, please read our privacy statement offline:
C:\Windows\system32\en-US\erofflps.txt

and becnchmark factory crashes

Any idea why

868 0 / Create an account to join the discussion.
eyal_51478
Follow / 18 Aug 2015 at 5:39am / Benchmark Factory
Latest post by on 24 Aug 2015 at 2:05am

Speeding up TPC-C Load Tables

Expand content
Hi, I have a setup with a "large" VM running as the console with 16gig RAM and 8 cores, and 10x smaller VMs running as agents. I'm trying to build very large DBs - 100GB, 500GB, 1TB, 5TB...

Hi,

I have a setup with a "large" VM running as the console with 16gig RAM and 8 cores, and 10x smaller VMs running as agents. I'm trying to build very large DBs - 100GB, 500GB, 1TB, 5TB etc.  The TPC-C load time is considerable - days. This is all to test DB performance with our self-designed storage system - however as I build the DB, the IOPS we're getting on our system are very low - in the hundreds of IOPS. We can easily deliver more than that. I reduced the thinking time option, and turned on the distributed index option.

Am I missing anything? What is the recommended setup of console and agents? How many?

I'm seeing similar behavior with SQL Server, MySQL, Oracle...

Eyal.

1396 1 / Create an account to join the discussion.
srajavar
Follow / 22 Apr 2015 at 11:56pm / Benchmark Factory
Latest post by on 24 Apr 2015 at 3:36pm

Question regarding capture/replay of a 10g RAC Database

Expand content
hi We are about to migrate our existing production Oracle 10205 RAC database to new machine using shareplex . In this regard, would like to perform benchmark testing and would like to capture the existing...

hi

We are about to migrate our existing production Oracle 10205 RAC database  to new machine using shareplex.  In this regard, would like to perform benchmark testing and would like to capture the existing load from current production and replay onto New prod Test instance Database to measure memory, cpu and io performance.

I am using a trial version of 7.1 and it seems Capture using AWR is not supported on RAC databases. Tried ADDM method but it does only capture load from source database only for 30 minutes.ideally, i am interested in capturing one full day's worth of workload from current production and apply on new database to measure performance.

pls let me know a way to achieve this using benchmark factory 7.1

Regards

Srikanth

Unanswered 2270 2 / Create an account to join the discussion.
yeh928
Follow / 2 Mar 2015 at 4:04am / Benchmark Factory

The response Time in Results Summary Graph is different from Realtime...

Expand content
I have a question about TEST RESULT of DELL BMF After complete Benchmark Test { Artificial Test for Desired Effect => Base on user selection of one or more hardware stressing focal point 」} I can...

I have a question about TEST RESULT of DELL BMF

After complete Benchmark Test   { Artificial Test for Desired Effect  =>  Base on user selection of one or more hardware stressing focal point」}

I can get the test result from "Run Reports",
I found the response Time(msec) in Results Summary Graph (from:Run Reports => Results Summary) is differently from Realtime Detail Graph (from:Run Reports => More Graphs => Realtime Detail Graph)

Under User Userload 100
In the Results Summary Graph: the Avg.Response Time is 225.839 (sec) as follow Graph

In the Realtime Detail Graph: the Avg.Response Time in the range from 192 to 603285 (msec) as follow Graph

I tried to the sum of range from 192 to 603285 (Realtime Detail Graph of response time), and then sum(192 to 603285) divided by count(192 to 603285),

the result of value (convert form msec to sec) still have a huge differently with (Results Summary Graph)

What is the difference of the (Avg.Response Time) between "Results Summary Graph" and "Realtime Detail Graph" ?

Unanswered 2748 0 / Create an account to join the discussion.
DM
Follow / 6 Feb 2015 at 7:03pm / Benchmark Factory
Latest post by on 10 Feb 2015 at 6:35pm

Only one user executed certain operations

Expand content
Hi, I'd like to create a test where one user runs a set of queries(looping a number of times) while the rest(let's say 99 users) will run different queries. In other words, I'd like to assign...

Hi,

I'd like to create a test where one user runs a set of queries(looping a number of times) while the rest(let's say 99 users) will run different queries. In other words, I'd like to assign specific tasks(a sequence of queries) to specific users. I am using Benchmark Factory 7.1.1

4061 2 / Create an account to join the discussion.
sglixman
Follow / 5 Dec 2014 at 6:02am / Benchmark Factory

Is there a way to continue object creation after failure?

Expand content
After ~ 2 days execution of create object phase, the DB log file was completely filled (no disk space left), and benchmark factory stopped: TPC-E Job Job Started Started test Create Objects for TPC-E...

After ~ 2 days execution of create object phase, the DB log file was completely filled (no disk space left), and 
 benchmark factory stopped:

TPC-E Job
Job Started
Started test Create Objects for TPC-E
Checking for existing objects...
Create Tables
Load Tables
Agent(dbf-ctrl2) Error: ODBC Error (37000,232A) - [Microsoft][ODBC SQL Server Driver][SQL Server]The transaction log for database 'Test-BF-40' is full due to 'ACTIVE_TRANSACTION'. Transaction "Load E_HOLDING_HISTORY Table " run failed by virtual user 37 in agent 1 of dbf-ctrl2

I can add disk space for the log, but, can benchmark factory continue the phase? 

thanks

Unanswered 3238 0 / Create an account to join the discussion.
faceless1289
Follow / 28 Nov 2014 at 10:47am / Benchmark Factory
Latest post by on 2 Dec 2014 at 2:01pm

Agent not able to start virtual users on remote machine

Expand content
Hi, I have installed BMF on Windows Server 2012 R2 and set-up a remote agent through console as suggested in BMF user guide on a remote PC hosting windows server 2012 R2. I am able to connect to remote...

Hi,

I have installed BMF on Windows Server 2012 R2 and set-up a remote agent through console as suggested in BMF user guide on a remote PC hosting windows server 2012 R2.

I am able to connect to remote agent by changing the IP as suggested in a post in the discussion and console is able to detect and verify the remote agent.

However, when I am running a TPC-E workload, the local agent is able to start virtual users while the remote agent is shown as connected in console tab but unable to start any virtual users.

On the remote PC, in the agent window, it displays "Starting Virtual Users" and then connection time-out happens with below error:

**************************************************************************************

TPC-E Job SF 10 User 60

Job Started

Starting Agent(s) on WIN-F45C2VOEF5E

Starting Agent(s) on 107.108.207.60

Started test Create Objects for TPC-E

Checking for existing objects...

Checking scale factor...

Scale factor check complete.

The loaded TPC-E dataset might have been modified. If a refresh/reload of the dataset is not performed errors may occur during testing.

Objects already exists.

Test completed successfully.

Started test TPC-E Transaction Mix

Checking scale factor...

The loaded TPC-E dataset might have been modified. If a refresh/reload of the dataset is not performed errors may occur during testing.

Scale factor check complete.

Agent(WIN-NSDCAUBUFLT) Error: ODBC Error (08001,2AF9) - [Microsoft][ODBC Driver 11 for SQL Server]TCP Provider: No such host is known.

Test cancelled due to errors.

Shuting down Agent(s) on WIN-F45C2VOEF5E

Shuting down Agent(s) on 107.108.207.60

** Job finished.

Total Time to perform the job:

Start Time: 11/28/2014 3:56:22 PM

End Time : 11/28/2014 3:59:12 PM

Elapsed : 0 Days 00:02:50.091

**********************************************************************************************************

What am I missing?? Please suggest.

Note: I am able to connect to SQL server instance installed on console PC from remote PC. Also I am able to configure a system DSN using SQL server library. But starting agent on remote PC do not requires these things afaik. The console only needs to distribute load with the remote PC.

4577 3 / Create an account to join the discussion.
sglixman
Follow / 19 Nov 2014 at 3:44pm / Benchmark Factory
Latest post by on 20 Nov 2014 at 3:28am

Huge estimated time to complete data loading

Expand content
Hello, I'm using a strong server, with MsSql2012, with fast storage (~ 20,000 iops). Configured benchmark factory to use 2 agents, TPC-E, with benchmarkScale=40. Benchmark factory is now running...

Hello,

I'm using a strong server, with MsSql2012, with fast storage (~ 20,000 iops).

Configured benchmark factory to use 2 agents, TPC-E, with benchmarkScale=40.

Benchmark factory is now running the 'create object' phase, and estimate the duration to be 15 days (!!)

The server's disk io is low (10MB/s), server cpu utilization ~ 60%.

I cannot identify where the bottleneck is.

Any hints how to make the load phase faster?

thanks

Answered 3820 1 / Create an account to join the discussion.
sglixman
Follow / 12 Nov 2014 at 3:57pm / Benchmark Factory
Latest post by on 14 Nov 2014 at 2:57am

Console is not connecting to agent

Expand content
Hello, I've installed Dell Benchmark Factory (DBF) on two machines: console and agent1. Both machine are windows 2012 server and connected to workgroup (not domain). I've configured DBF controller...

Hello,

I've installed Dell Benchmark Factory (DBF) on two machines: console and agent1.

Both machine are windows 2012 server and connected to workgroup (not domain).

I've configured DBF controller to use two agents (via settings/Agent): localhost (console machine) and agent1.
I've run "verify selected agent" successfully.
Now, when I start ''Create Objects for TPC-E", I get the following error message:
TPC-E Job
Job Started 
Starting Agent(s) on 172.31.3.50
Not all configured agent(s) are connected in a timely manner.
Shuting down Agent(s) on WIN-LCF3MS76I0E
Shuting down Agent(s) on 172.31.3.50
Total Time to perform the job:
  Start Time: 11/7/2014 1:47:33 PM
  End Time  : 11/7/2014 1:48:34 PM
  Elapsed   : 0 Days 00:01:01.109
Additional Info
  • I've verified that an "Agent" process was created in agent1 machine.
  • Windows firewall is disabled both on console and agent1 hosts
  • The user I configured for agent1 host has admin privileges on agent1
  • Both console and agent hosts are Amazon EC2 vm hosts running Windows Server 2012

Ideas?

thanks

5265 5 / Create an account to join the discussion.
Gary Little
Follow / 16 Sep 2014 at 9:26pm / Benchmark Factory
Latest post by on 18 Sep 2014 at 7:50pm

Comparing transaction rates with HammerDB

Expand content
Hi, I am comparing the results I get from HammerDB to the results I get with Benchmark Factory. The numbers I get from HammerDB are much higher than I get from Benchmark Factory. I also notice that the...

Hi,

I am comparing the results I get from HammerDB to the results I get with Benchmark Factory.  The numbers I get from HammerDB are much higher than I get from Benchmark Factory.  I also notice that the cpu consumed by the Oracle user processes is much lower when I used HammerDB than when I run Benchmark Factory.  The target DB is exactly the same in both cases, and I create a new schema for each tool.

I am using TPC-C in both cases.  I am curious to know why there is such a high discrepancy (up to 2X) when both tools claim to do TPC-C like workloads.

Thanks,

Gary.

6329 3 / Create an account to join the discussion.
abhimanyu.singh
Follow / 17 Sep 2014 at 6:50pm / Benchmark Factory
Latest post by on 17 Sep 2014 at 7:01pm

Which benchmark test (TPC-*) is best suitable to see impact on SQL...

Expand content
Hi, I am new to BMF test. I like to measure the TPS impact when we vMotion a SQL server VM from one host to another and Database High Availability(HA) test. What is the recommended benchmark test (and...

Hi,

I am new to BMF test. I like to measure the TPS impact when we vMotion a SQL server VM from one host to another and Database High Availability(HA) test. What is the recommended benchmark test (and configuration) should be used in both cases?

From VMware HA guide, I see some graphs showing the TPS impact during the transition of VM. The TPS measurement is quite flat before and after vMotion occurred (spike occurred during vMotione. See 1,2).

I have tested with TPC-E/H. TPS measurement is never as flat (see 3) as what I saw on the whitepaper. Is something I did not configure properly, or used wrong benchmark test.

Any suggestions is appreciated.

Thanks

http://www.vmware.com/files/pdf/products/vsphere/VMware-HA-Deployment-Guide-Oracle-Database.pdf

(1)

(2)

(3)

Answered 4173 1 / Create an account to join the discussion.
HillbillyToad
Follow / 9 Apr 2007 at 3:47pm / Benchmark Factory
Latest post by on 31 Mar 2014 at 8:38pm

Is there a faster way to create a test database?

Expand content
I have been creating several test databases to test a hardware platformunderload but I have been running into time constrains for building adatabase. For a TPC-H database with a scale factor of 22 (estimated...

I have been creating several test databases to test a hardware platformunderload but I have been running into time constrains for building adatabase.  For a TPC-H database with a scale factor of 22 (estimated 22GB size for just the tables).  This takes many hours, and only createsdata at 4-7 MB/s, when under the actual TPC-H test the queries run at800 MB/s.  I am trying to vary the benckmark factory settings to testthe limits of the storage, so I am starting a test run, monitoring whatit does to the storage, then changing to a different test/scalefactor/etc to feel out which is the best test.  The many hours ofcreate time between tests, however, is really holding me back.

This post was originally created on the Code Tester forum by user: JAMON.

8077 8 / Create an account to join the discussion.
jxs2151
Follow / 14 Feb 2014 at 5:40pm / Benchmark Factory
Latest post by on 17 Feb 2014 at 11:32pm

Compare Results With Other Systems?

Expand content
So I have been playing with Benchmark Factory and have run some tests but am not really sure where to find something published to compare my results with. Things like Response Time, etc. Somewhere there...

So I have been playing with Benchmark Factory and have run some tests but am not really sure where to find something published to compare my results with.  Things like Response Time, etc.  Somewhere there has to be some results.   I have spent a good amount of time searching the web but have come up empty handed.  What is a good response time?  

I understand that there are many variables in determining that and what one may consider 'good' another may not.  Please don't simply reply with 'it depends'.  If you could guide me in the right direction, perhaps a site that lists something, that would be helpful.  I find it hard to believe that there is a tool that gives one results but there is no way of knowing what constitutes acceptable and not good.

7685 6 / Create an account to join the discussion.
vivien.xing
Follow / 10 Dec 2010 at 6:30pm / Benchmark Factory
Latest post by on 27 Jan 2014 at 7:07pm

TPC-E Transaction Mix error

Expand content
I am experimenting with BF and using version 6.1.1 build 228. When I ran TPC-E Transaction Mix against Oracle 11g (11.1.0.7.0 – 64 bit), I got this error (my job was set to Stop job after first error...
I am experimenting with BF and using version 6.1.1 build 228.   When I ran TPC-E Transaction Mix against Oracle 11g (11.1.0.7.0 – 64 bit), I got this error (my job was set to Stop job after first error): 

Agent(myagentname) Error: TradeResult insert holding_summary failure.

My question is what the condition(s) to throw this error?  Is it time out, or some other threshold(s) reached?  Seems there was no dead lock as total_deadlocks = 0 in
dbo.bf_test_results table. Vmstat showed that runqueue was 1 - 3 and CPU (user + system) usage was 81 - 94% in the last 5 minutes before the job stopped. 

I am hoping to get the answer to better understand the test results.

Thank you,
Vivien
9565 18 / Create an account to join the discussion.
jvlorenzen
Follow / 8 Jan 2014 at 9:59pm / Benchmark Factory
Latest post by on 9 Jan 2014 at 8:19pm

Running TPC-C to check on impact of increasing memory with SQL Se...

Expand content
I am attempting to study the impact of increasing the server memory on SQL Server. I am running BMF TPC-C with a number of users (I started with 90 & 100 virtual users), limiting the SQL Server memory...

I am attempting to study the impact of increasing the server memory on SQL Server. I am running BMF TPC-C with a number of users (I started with 90 & 100 virtual users), limiting the SQL Server memory to simulate server memory at 16GB, 32GB, 64GB, 96GB and 128GB. 

I had expected to see better performance at the larger memory sizes, but the TPS stayed the same with different memory sizes. This tells me that the 100 users is not enough to stress the system. I got the license for 1000 virtual users and my results with 900 & 1000 are about the same (linear) with fewer users. 

Is there a way to increase the load with 1000 virtual users or do I need to increase the virtual user count. Also, what is the max number of virtual users that I can use?

The data base scale is 7,000 warehouses. This is running on a Dell R910, with two CPUs (Xeon E7-4850 @ 2.0 GHz)

Answered 4452 2 / Create an account to join the discussion.