WebLogic 8.1/Grinder Load Test - Execute Threads vs. JDBC Connection size Effects

Dynamic Charting

Todd Nichols, Anser Enterprise

Introduction

In a previous article, techniques were presented for automatically configuring SUT performance tuning parameters and running hundreds of stress tests in a short period of time. The motivation was to have enough useful information to improve performance by more than an order of magnitude by adjusting tuning parameters, as others have done. However, performance tuning parameters are often interrelated so one needs to examine results for all permutations of settings, one change at a time. But how can reports effectively communicate cause-and-effect relationships to users with that much data? Producing dozens of separate static charts using Excel is an improvement somewhat over just giving users tabular data. But what kind of charting engine could reduce the labor of producing charts on-line and at the same time provide better navigation through different tuning parameter settings within a single chart for the reader?

In this article, a Java applet viewer addresses these needs using an open source charting class library, embedded database-like query capabilities and charting controls that includes animation of families of curves. Using HTML applet parameter tags, an author can configure the applet viewer in a web page to focus the reader's attention on results for particular test conditions along with commentary in the same web page. Additional data for many more tests, downloaded by the applet, is only a click away to answer 'what-if' questions on system configuration or modeling parameter changes. With chart animation, trends or subtle differences in individual curves within families of performance curves become more apparent as the applet viewer automatically scans through tunable parameter settings.

Browser Setup to use SUN's Java 1.5 Plug-in

The Java charting applet has been tested with MS Internet Explorer, Version 6.0 and Mozillia Firefox, Version 1.0. The charting applet requires SUN's Java Plug-in, Version 1.5 to be installed and available to the browser. If this has not already occurred as you view this web page, your browser will prompt you to download the correct version (1.5.0_04) of the Java Plug-in. The browser prompt to download could occur in two places:

As a new horizontal bar just below the browser's last toolbar
At the top of the rectangular area where the Java applet will load.

The prompt to download says: <Shield Icon> "This site might require the following ActiveX control: 'J2SE Runtime Environment 5.0 Update 4' from 'Sun Microsystems, Inc.'. Click here to install..."

The applet uses an Open Source charting library, JFreeChart. With the JFreeChart class files, the applet's JAR file is 1.09MB in size but is available in the browser's cache as other web pages containing the applet are viewed.

Report 1 is an example of what an on-line report using the charting applet might look like.

Report 1: Sample Performance Analysis Report using the Java Applet Viewer

Report: WebLogic Server Load Test Results, 9/9/05
Study Objectives: Measure performance at different settings of Execute Queue ThreadCount and max-beans- configuration parameters.
Application: Bean-managed Banking Client/Server
Performance Measurements: TPS, Mean Response Time, Successful Tests, and Errors
Configuration Parameters studied: Server Execute Queue ThreadCount, Max-Beans-In-Free-Pool and Max-Beans-In-Cache
Initial Chart Parameter Settings

Client Workload Threads: 5, 10, 15, 20, 25, 30, 35, 40
Server Execute Queue ThreadCount: 5, 10, 15, 20, 25
max-beans-in-free-pool and max-beans-in-cache: 15
JDBC Connection Pool MaxCapacity: 30

Chart 1: Decreasing TPS with Increased Errors at Low Max-Bean Configuration

Test Results: Max-beans-in-cache and max-beans-in-free-pool set at 15 creates a bottleneck. Errors begin to occur at X1:Grinder_Threads (agent workload) = 20 (on X axis) and X2:weblogic.kernel.Default_ThreadCount >= 15 (in the animated legend). At weblogic.kernel.Default_ThreadCount = 25, there are hundreds of errors, which causes the superimposed TPS and Successful Tests curves to drop off (25 workload threads and higher).

Accessing Charting Controls

The Applet Viewer can be initially configured by the HTML author using applet parameter tags. The reader can also re-configure the viewer using charting controls. Clicking on the right arrow at the top of the split pane separator located on the left side of Chart 1 will expose its charting controls.

Chart 2 has been configured to show just the Input Parameter pane of the charting controls and a single TPS curve with no animation (reduced in size to save space). The viewer is in a mode for increasing/decreasing input parameter settings (X2+/- radio button checked) so the reader can manually step through the same data that was animated in Chart 1. Follow Chart 2's instructions to see how this is done.

Change Parameter Settings	Chart 2: Chart with Input Parameter Controls Panel
Mouse over input labels. Click labels with other settings. e.g. Default_ThreadCount 20.0<25.0> Use Shift-Click(s) to decrease (or use right mouse button) e.g. 25->20->15->10->5 Use left button to increase

Show Families of Curves using a Secondary X-Axis Variable

Both animation and single stepping through input parameter settings can make differences in performance easier to see, but charting a static family of curves, as in Chart 3, will often be the best choice. The reader can re-configure Chart 2 to look like Chart 3 by changing the input parameter control mode to select a secondary X-Axis variable (X2 radio button checked) and clicking on the label, weblogic.kernel.Default_ThreadCount.

To add metrics for Mean Reponse Time, # of Successful Tests and Errors, continue with the instructions for Chart 3. Turning animation on and expanding to full chart view as a final step makes Chart 3 look like Chart 1.

Select Multiple Statistics, Animate, Full Chart View	Chart 3: Using Statistics and Input Parameter Charting Controls Panes
Click/drag-down horizontal controls split pane separator. Click labels, Mean Time, Successful Tests, Errors in the Statistics Pane. Check Animate checkbox Click left arrow on vertical split pane separator.

Performance Data Files and Applet Viewer Parameter Tags

On startup, all of the differently configured applet viewers in this article automatically downloaded statistics and input parameter data from these files, InputParmNames.txt, InputParms.txt, StatsNames.txt, and StatsMean.txt.. These files were created automaticaly using the modified load testing tool, The Grinder, as described in the previous article.

The Applet Viewer is designed to focus attention on a specific subset of performance data downloaded from these files using pre-configured charts with performance analyst's commentary on the same web page. Authoring performance results web pages involves specifying the applet parameters embedded in the HTML page, which determine the desired initial state of the applet Viewer. Listing 1 shows the applet parameter tags configuring the viewer in Chart 4 and an explanation of those parameters follows.

Listing 1. Applet Viewer Parameter Tags

These are the parameter tags for the configuration of the applet in Chart 4.

		<OBJECT type="application/x-java-applet"  WIDTH=950 HEIGHT=400>
		<param name="CODE" value="net.viewer.GraphApplet">
		<param name="archive" value="../viewer.jar">
		<param name="scriptable" value="false">
		<param name="initValsRunNum" value="41">
		<param name="modeName" value="X1">
		<param name="x1Name" value="Grinder_Threads">
		<param name ="x2Name" value="max-beans-in-cache">
		<param name="autoScale" value="On">
		<param name="animate" value="Off">
		<param name="fullChartView" value="false">
		<param name ="y2Name" value="Grinder TPS,Grinder Mean time">
		</OBJECT>

Chart 4: TPS and Mean Time vs. Client Workload and Max-Beans Configuration Settings

The following provides details of some of the applet parameters including the ones used for the Viewer in Chart 4.

SCRIPT language="JavaScript" src="../objectBegin.js"" Depending on the type of browser, the JavaScript sourced by this initial tag will embed the correct choice of <OBJECT>, <EMBED>, or <APPLET> tag. Microsoft Internet Explorer 6.0 and Firefox 1.0 browsers have been tested with these HTML tags. Early versions of MS IE may have a problem showing any HTML that follows the applet.

name ="initValsRunNum" value="41" The initValsRunNum applet parameter configures the Viewer to use a particular testing/modeling run number for setting all non-primary (and non-secondary) input parameters to constant values. The applet in this example is configured to use "41" as its initial (Run ID) variable for charting. Run ID numbers are defined during the load testing or model simulation phase and are saved with the input parameter and testing/modeling statistics results in unique records for each run. For input parameters, each record has its Run ID and as many additional fields as there are input parameters defined in the load test/simulation. For statistics results, an associated record has this Run ID number and as many fields as there are statistics defined in the test/model. As the user changes the Viewer's controls and label selections, the constant settings for non-primary parameters will change and the value of the internal variable initially set to initValsRunNum will change accordingly.

name ="x1Name" value="Grinder_Threads" The x1Name applet parameter tells the Viewer which input parameter to use as its primary independent variable. The applet in this example is configured to use "Grinder_Threads" as its primary X axis (X1) variable. The Viewer computes the set of values, which the X1 variable can take on for charting purposes by comparing all input parameter records against the initValsRunNum record. The set of X1 values used for a chart includes X1 values taken from all Run ID's where all other input parameter values equal those in the initValsRunNum record.

name ="x2Name" value="max-beans-in-cache" The x2Name applet parameter tells the Viewer which input parameter to use as the secondary independent variable. The applet in this example is configured to use "max-beans-in-cache" as its secondary X axis (X2) variable. Selecting a secondary independent variable can result in multiple curves of the same color but with different symbols being substituted as the datapoint indicators on the curves as in the example above.

name ="y2Name" value="Grinder TPS,Grinder Mean time" The y2Name applet parameter tells the Viewer which statistics to chart. The applet can be configured to use a comma-separated list of statistics, which in this example consists of the "Grinder TPS" and "Grinder Mean time" statistics.

name ="autoScale" value="On" The autoScale applet parameter configures the initial setting of the AutoScale checkbox. In this example AutoScale is set to On. The analyst may want to instruct the reader to scan through certain non-primary input parameter values to see their effect on the performance curves. The web page author can initialize AutoScale to OFF so that as the reader clicks Input Parameter labels to cause their values to increase/decrease, the curves can be seen to move accordingly. Scaling will stay the same. With Autoscale ON, the cause and effect sensation is visually less noticeable because the curves stay centered in the chart while the scaling is adjusted.

name ="animate" value="Off" The animate applet parameter configures the initial setting of the Animate checkbox. In this example Animate is set to Off. With Animate turned on, AutoScale is disabled so that curves move up and down in the chart frame with fixed scaling. Only the symbol used as datapoint indicator for the curve will appear in the X2 legend box next to the X2 value used to produce the curve being drawn. There is a one second delay between drawing the curves and a two second delay at the end of the sequence of curves in the family of curves (X2 selected).
Animate is useful when curves within a family of curves are close together and it's difficult to see minor differences in the curves.

name ="fullChartView" value="false" The fullChartView applet parameter tells the Viewer to use the entire Viewer width to present the chart, or reserve space to show the control panels as well.

Conclusions

Load testing to determine the optimal settings for many configuration parameters produces a lot of performance data as does performance modeling. Dynamic, client-side charting using a custom Java applet and an open source charting library has several advantages over manually creating many static charts including:

Allowing readers to explore cause-and-effect relationships using embedded database-like query capabilities with single mouse clicks.
Comparing subtle difference and recognizing trends by stepping through settings or animating curves.
Reducing the effort to create charts for the performance analyst.
Reporting from central data files or eventually performance databases.

The author welcomes comments on usability and ideas for additional charting features and capabilites.

Uses JFreeChart Open Source Charting Library

SCRIPT language="JavaScript" src="../objectBegin.js""	Depending on the type of browser, the JavaScript sourced by this initial tag will embed the correct choice of <OBJECT>, <EMBED>, or <APPLET> tag. Microsoft Internet Explorer 6.0 and Firefox 1.0 browsers have been tested with these HTML tags. Early versions of MS IE may have a problem showing any HTML that follows the applet.
name ="initValsRunNum" value="41"	The initValsRunNum applet parameter configures the Viewer to use a particular testing/modeling run number for setting all non-primary (and non-secondary) input parameters to constant values. The applet in this example is configured to use "41" as its initial (Run ID) variable for charting. Run ID numbers are defined during the load testing or model simulation phase and are saved with the input parameter and testing/modeling statistics results in unique records for each run. For input parameters, each record has its Run ID and as many additional fields as there are input parameters defined in the load test/simulation. For statistics results, an associated record has this Run ID number and as many fields as there are statistics defined in the test/model. As the user changes the Viewer's controls and label selections, the constant settings for non-primary parameters will change and the value of the internal variable initially set to initValsRunNum will change accordingly.
name ="x1Name" value="Grinder_Threads"	The x1Name applet parameter tells the Viewer which input parameter to use as its primary independent variable. The applet in this example is configured to use "Grinder_Threads" as its primary X axis (X1) variable. The Viewer computes the set of values, which the X1 variable can take on for charting purposes by comparing all input parameter records against the initValsRunNum record. The set of X1 values used for a chart includes X1 values taken from all Run ID's where all other input parameter values equal those in the initValsRunNum record.
name ="x2Name" value="max-beans-in-cache"	The x2Name applet parameter tells the Viewer which input parameter to use as the secondary independent variable. The applet in this example is configured to use "max-beans-in-cache" as its secondary X axis (X2) variable. Selecting a secondary independent variable can result in multiple curves of the same color but with different symbols being substituted as the datapoint indicators on the curves as in the example above.
name ="y2Name" value="Grinder TPS,Grinder Mean time"	The y2Name applet parameter tells the Viewer which statistics to chart. The applet can be configured to use a comma-separated list of statistics, which in this example consists of the "Grinder TPS" and "Grinder Mean time" statistics.
name ="autoScale" value="On"	The autoScale applet parameter configures the initial setting of the AutoScale checkbox. In this example AutoScale is set to On. The analyst may want to instruct the reader to scan through certain non-primary input parameter values to see their effect on the performance curves. The web page author can initialize AutoScale to OFF so that as the reader clicks Input Parameter labels to cause their values to increase/decrease, the curves can be seen to move accordingly. Scaling will stay the same. With Autoscale ON, the cause and effect sensation is visually less noticeable because the curves stay centered in the chart while the scaling is adjusted.
name ="animate" value="Off"	The animate applet parameter configures the initial setting of the Animate checkbox. In this example Animate is set to Off. With Animate turned on, AutoScale is disabled so that curves move up and down in the chart frame with fixed scaling. Only the symbol used as datapoint indicator for the curve will appear in the X2 legend box next to the X2 value used to produce the curve being drawn. There is a one second delay between drawing the curves and a two second delay at the end of the sequence of curves in the family of curves (X2 selected). Animate is useful when curves within a family of curves are close together and it's difficult to see minor differences in the curves.
name ="fullChartView" value="false"	The fullChartView applet parameter tells the Viewer to use the entire Viewer width to present the chart, or reserve space to show the control panels as well.