Minimize
Blogger List

Johannes Ahrends
Toad and Oracle

Ben Boise
Toad SC Discussions

Kevin Dalton
Benchmark Factory

Steven Feuerstein
PL/SQL Obsession

Devin Gallagher
Toad SC discussions

Stuart Hodgins
JProbe Discussions

  Henrik "Mauritz" Johnson
Toad Tips & Tricks on the "other" Toads
  Mark Kurtz
Toad SC discussions
  Michael Lumbard
Toad SC discussions
Daniel Norwood
Toad for Data Analysts,
Toad Extension for Visual Studio
Debbie Peabody
Toad for Data Analysts
Gary Piper
Toad Reports Manager
John Pocknell
Toad for Oracle, JProbe
Kuljit Sangha
Toad SC discussions
Bert Scalzo Indicates Oracle ACE status
Toad for Oracle, Data Modeling, Benchmarking
Jeff Smith
Toad product family
Richard To
SQL Optimization
Jim Wankowski
DB2 - LUW and z/OS
John Weathington
  Toad Data Modeler Opens in a new window
Data Modeling
 
  Real Automated Code Testing for Oracle
Quest Code Tester blog
 
Minimize
Blog Tags
toad for oracle (122)
oracle (62)
plsql (46)
sql optimization (37)
toad for data analysts (28)
code tester (19)
toad for ibm db2 (13)
automation (11)
batch optimizer (10)
virtualization (10)
schema browser (9)
toad for sql server (9)
data grid (8)
sql (8)
sql editor (8)
toad data modeler (8)
benchmark factory (7)
excel (7)
query builder (7)
report manager (7)
toad extension (7)
visual studio (7)
11g (6)
configuration (6)
freeware (6)
health check (6)
vmware (6)
connect (5)
dba module (5)
er diagrammer (5)
F4 (5)
linux (5)
refactoring (5)
spotlight (5)
unicode (5)
compare (4)
debugger (4)
export (4)
formatter (4)
make code (4)
rman (4)
strip code (4)
benchmark (3)
bfscript (3)
bulk collect (3)
code templates (3)
code xpert (3)
database browser (3)
db2 (3)
notebook (3)
oem (3)
RAC (3)
session browser (3)
speed (3)
sql optimizer (3)
toad for mysql (3)
tpc-c (3)
9.7 (2)
alert log (2)
app designer (2)
awr (2)
code insight (2)
code snippets (2)
collection (2)
compare and sync (2)
compliance (2)
data generator (2)
data warehouse (2)
database explorer (2)
database monitor (2)
explain (2)
forall (2)
ftp (2)
group execute (2)
handbook (2)
installation (2)
job scheduler (2)
multi-task (2)
nested table (2)
os command (2)
profiler (2)
recovery (2)
release history (2)
save as (2)
schema compare (2)
sql recall (2)
stats pack (2)
subversion (2)
team coding (2)
trace file browser (2)
while loop (2)
10g (1)
64 bit (1)
7zip (1)
action (1)
addm (1)
alter (1)
ansi join (1)
array (1)
ccleaner (1)
code coverage (1)
code road map (1)
CRON (1)
cursor for loop (1)
data browser (1)
data subset (1)
database probe (1)
dbms_flashback (1)
dbms_profiler (1)
ddl (1)
feuerstein (1)
filezilla (1)
flash drive (1)
flow control (1)
for loop (1)
group policy manager (1)
hints (1)
import (1)
index (1)
inheritance (1)
invoker rights (1)
ipad (1)
java (1)
latency (1)
log switch (1)
logical model (1)
ltrim (1)
master-detail browser (1)
monitor (1)
multi-select (1)
naming standards (1)
network (1)
object explorer (1)
OEBS (1)
package (1)
parser (1)
partitioning (1)
performance (1)
pragma (1)
project manager (1)
RAT (1)
revo (1)
REXEC (1)
schema report (1)
script manager (1)
search (1)
set operator (1)
sga (1)
slow (1)
sonarsource (1)
source control (1)
space projection (1)
sql monitor (1)
sql navigator (1)
sql script (1)
sql tracker (1)
sql*plus (1)
standards (1)
statistics (1)
stored procedure (1)
string parser (1)
sub-model (1)
sub-type (1)
synch (1)
synchback (1)
TELNET (1)
toad (1)
trace (1)
unit test (1)
unix (1)
usb (1)
utility (1)
v10 (1)
v9.5 (1)
version control (1)
waits (1)
workload replay (1)
workspace (1)
xml (1)
 
WELCOME, GUEST
 
 

Blogs
Toad and Database Commentaries

Toad World blogs are a mix of insightful how-tos from Quest experts as well as their commentary on experiences with new database technologies.  Have some views of your own to share?  Post your comments!  Note:  Comments are restricted to registered Toad World users.

Do you have a topic that you'd like discussed?  We'd love to hear from you.  Send us your idea for a blog topic.


Jan 16

Written by: JohnWeathington
Friday, January 16, 2009  RssIcon

By now, I’m sure you’ve heard about the Hudson River landing of US Airways flight 1549. What a remarkable story of an unbelievably low probability risk event actually occurring, and a heroic contingent effort that mitigated a potential disaster. The popular theory on causation is that a flock of birds flew into both engines, causing them both to fail. But how do we know for sure that’s what happened?

When the dust settles, I’d be really surprised if this wasn’t the case. I’m not trying to pitch a conspiracy theory here. My point is that we won’t know the definitive answer until the NTSB (National Transportation Safety Board) completes its investigation. And fortunately, they’re well prepared. Every airplane is fitted with two “black boxes” at the tail of the plane that records vital information for the investigators: the cockpit voice recorder and the flight data recorder.

Now the Hudson River case may seem obvious, and maybe they have enough ancillary evidence to make a decisive conclusion on the bird theory, however maybe not. In fact, in a lot of cases the NTSB investigates, these black boxes are all they really have.

These boxes serve a specific purpose; they support NTSB investigations. The NTSB at some point realized that it would be a smart idea to collect a lot of data on every flight of every plane. The large majority of this data is never used. However, in the unlikely event of a water landing (can you tell I’ve had my share of plane trips), or any other unusual (and usually unfortunate) happening, they have the data that they need to determine what happened.

Your company is in the same position when it comes to compliance. The intelligent architect of a compliance program will design a component of the data system that collects data in anticipation of an investigation. If you follow my work, you know that what I’m suggesting is a contingent control for the risk event of an investigation.

In most cases your controls are dealing with an unfavorable risk to an objective. In this case however, the objective itself is the efficient compliance program, and the risk is that you’ll be audited and subsequently investigated.  The effects of the investigation are probably detrimental to the company (i.e. penalties and fines), so to deal with the effect before the risk event occurs, is what I call a contingent control. Assuming your company is doing the right things, lack of data to prove this can kill you in an investigation. So the approach is to be prepared with lots of good data. And in honor of the aviation system just described, the component of the compliance data system that supports this control is what I call the Black Box Data Store.

The Black Box Data Store is simple in concept, but powerful in function. It can connect to any component of your compliance data system, including your source systems, compliance operational data store, compliance enterprise data warehouse, or even your compliance data marts. Think of it as being downstream from your core compliance data system. When designing your Black Box Data Store, keep the following design principles in mind:

Design Principle #1: Assume Your Company Will Be Investigated

Don’t assume that since an investigation is unlikely, you don’t have to put effort into preparing for it. The incident that just happened with US Airways had a probability of less than a million to one. You must approach this with the attitude that you will be investigated, and design the system appropriately.

Design Principle #2: Collect More Data than You Need

I’ve been on several data warehouse analysis sessions, where the user is mandated to provide an exhaustive list of data points that they want brought from the source system to the data warehouse. This may be appropriate for some strategic purposes, but for a compliance data system it’s the wrong approach. More often than not, during an investigation, you will discover the usefulness of data points that never served a purpose prior to the investigation. You definitely want this data available. Trying to pull in data after the fact is always painful and in some cases impossible.

Design Principle #3: Plan for Design Iterations and Mock Investigations

As with most designs, you will not get it right the first time. To think you will is arrogant, and will cost your company dearly. As with the Hudson River landing, you only get one shot to get it right, so you need to practice several times. Assemble a team of coders, designers, experts on the regulation or standard, and auditors. Build the system you think is appropriate, and have a mock investigation. Do a thorough post-mortem and go back to the drawing board. Plan on at least three iterations, and more is better.


Investigations are a good thing when the NTSB is trying to get to the bottom of an airplane crash, but not so good when your company is the one being investigated. You can help your company be prepared for an investigation with a Black Box Data Store. Assume your company will be investigated, collect more data than you need, and conduct as many design iterations and mock investigations it takes to get it right. You probably won’t get to the point where your company is being investigated; however, you’ll be happy you have a Black Box Data Store if you ever do.

Search Blog Entries