WGT is developed
in C++, and utilizes TCP/IP sockets and various IPC mechanisms to support
large scale and distributed, performance and stress test environments.
Key technical features include:
Buffered
Messaging: All communication between external components utilizes
a message-based architecture. Data content is normalized to a standard
format to support interfacing with heterogeneous architectures within
a single test. The communication mechanism used is sockets running across
TCP/IP. Each message is wrapped in a frame that facilitates the usage
of TCP/IP sockets. Buffering on a per communication channel basis, allows
for high performance, by receiving partial as well as multiple messages
when the data arrives with a single system call.
Thread
Support: Threads are used through out the architecture to create an
environment that is event driven, as opposed to poll driven. The driver
manages a set of threads to read data results from the controllers, as
well as to manage the dialog with multiple consoles. The controller and
handlers also use threads to manage the communication and monitoring of
the various communication components. The locking level is very granular
to eliminate contention from the different components.
Controller/Handler
Architecture:
The controller and handler pairs manage a test workload. One controller
is used to manage a set of test agents on a single system for each business
function being tested. Multiple controllers can be run on multiple systems
to provide a scalable test environment, as well as multiple controllers
on a single system managing different test agents. Each controller manages
a shared memory queue that is used to generate work for the agents at
a specific message rate, as well as capturing response time and transaction
performance data. This data is then sent to the driver for storage in
the test repository. The driver will communicate directly with each controller
in the test and has no knowledge of the number and type of agents in the
test. This approach distributes the amount of work the driver must do
to control and monitor a test, and supports testing of very large user
or transaction based workloads. The handler bridges the formalized driver/controller
interface to the customized processing of the agent. The handler is responsible
for starting and monitoring the agents in a test. Typically, while any
number of agents may be designed, a test will reuse handlers of a similar
architecture, which speeds the implementation of a customized performance
test.
User and
Work based testing: System testing usually has two different metrics
and workloads. User based testing consists of multiple suites of complex
transactions that run completely, but only last a number of seconds. The
volume is based on a total and active number of users, with a variable
amount of think time between transactions. eCommerce systems are indicative
of user based testing. Workload based testing has a number of discrete
transactions that are executed over a period of time. The workload is
usually identified as a number of transactions executed over a period
of time. WGT facilitates workload based testing, as well. A console is
used to control and monitor the current state of the test. Console commands
can be used to change the overall message rate, or the number of users
that you want to run with at any time. The driver will dispatch controllers
to produce the desired workload at the specified rate. Each controller
will add work to a work queue, which the agents are waiting for work.
As an agent becomes ready, it will run when there is work on the shared
work queue. Summary results of the response times of each business function
being tested are displayed on the console, and are used to identify the
current well being of the test. When bottlenecks start to appear, based
on changes in the response times, the test workloads can be changed interactively
to identify the exact break point.
Data Server:
Providing data for a test is critical to the validity of a test. For a
test to be successful, large sets of varied data must be used to eliminate
the effects of caching that can occur when using limited subsets of the
data. A general data server is implemented to provide a structured set
of data to agents as they need it. At the beginning of a transaction,
an agent will request a consistent set of data from the data server. The
agent will select a record from the set of data (which was previously
extracted and formatted) and log the record as used. This journaling of
data allows for quick restarting of tests when modifying a database, so
that the same data will not be reused.
Performance
Data Granularity:
The performance of each transaction is captured by the driver for later
analysis. At the highest level, the total response time of each business
function is measured and correlated with the current workload volume (number
of users or work message rate) as well as any performance data being captured.
In addition, the agent can also provide contextual data for each transaction
such as the number of rows returned, or the number of images downloaded.
For long running transactions that are composed of many smaller single
transactions, the test agent can generate sub-transaction records. These
records are tied back to the complete transaction, and can be used, when
needed, to drill down into the details of a bottleneck.
Performance
Data Collectors:
The WGT has the ability to capture data from external sources, and have
it stored in the repository. Data from traditional sources, such as
vmstat, sar, and netstat can be captured from each machine in the test, as well
as general SNMP data from hardware, databases and transaction processing
systems. This data is integrated into the repository along with the transactional
data to provide a point-in-time analysis capability. For example, you
may want to correlate the response time for a transaction with the physical
I/O of a specific disk or network collisions from a router.
System
Dependencies:
- Hardware
- memory and processor configurations for workload generation
systems (test harness) can vary widely, depending on the workload and
agent complexity
- Console
and Driver Platform - Linux, Solaris, and HP-UX
- Agent/Data Collector Platform - Linux, Solaris, and
HP-UX, with GNU C++
Compiler and OpenSSL packages installed
Back
to Tools Overview
