Load Testing with Random Walks, Part Two

Read "Load Testing with Random Walks, Part One" here.

This article is a little late coming.  That's OK.  We're here to examine how to interface with a webpage from a simple script.  For this discussion, I'm going to be using Microsoft JScript and the XMLHTTP object provided by Microsoft (this is the same magical object that makes Google Maps work).  A quality reference that I will be using is "Dynamic HTML and XML: The XMLHttpRequest Object" published by Apple.  This article covers the use of XMLHTTP on the client-side for webpages.  We'll be using it in a stand-alone command line application.  Also see the official MSDN reference.

The goal here is to wrap some sort of component that does HTTP requests.  The actual component that is used is unimportant.  I chose XMLHTTP for this because it's a different object than the one I use with my employer.  I'd hate to get into an intellectual property spat.  The nice thing about the wrapper is that you can use it and then chuck it away when you're done.

First off, we need to define a data structure for a page and a few constants:

// "Constants" var GET  = "GET"; var POST = "POST"; function Page() { this.url = ""; this.requestType = GET; this.postData = ""; this.responseCode = 0; this.pageSource = ""; this.timeToLoad; }

So far, so good.  Next we want the wrapper class itself -- I call it "Browser".  Note that we use the ServerXMLHTTP component because it does not suffer from the domain restrictions that regular XMLHTTP does.

function Browser() { this.xmlhttp = WScript.CreateObject("Msxml2.ServerXMLHTTP"); if ( !this.xmlhttp ) throw new Error("Browser could not instantiate Microsoft.XMLHTTP"); this.cookies = new Array(); }

Now that we have a constructor function, we need some methods!  The first, and the most obvious, is "GetPage".  It goes out, retrieves the page we're after, and returns a nice, compact data structure.

Browser.prototype.GetPage = function( url, requestType, postData ) { // precondition: url and requestType specified. if ( !url || !requestType ) throw new Error("Browser::GetPage requires at least a URL and a request type (usually GET, but sometimes POST)."); var timeStarted, timeFinished; // 1.  Do the request timeStarted = new Date(); // "false" denotes synchronous mode (versus asynchronous) this.xmlhttp.open( requestType, url, false ); if ( this.cookies.length > 0 ) { this.xmlhttp.setRequestHeader( "Cookie", this.cookies.join(";") ); } this.xmlhttp.send( (postData) ? postData : null ); timeFinished = new Date(); // 2.  Handle the results. var page = new Page(); page.url = url; page.requestType = requestType; page.postData = (postData) ? postData : ""; page.responseCode = this.xmlhttp.status; page.timeToLoad = timeFinished.getTime() - timeStarted.getTime(); try { this.SetRequestedCookies( this.xmlhttp.getResponseHeader("Set-Cookie") ); } catch(e) { // filter out the "requested header was not found" error that occurs when the server // does not set a cookie.  Rethrow anything else. if (e.number != -2147012746) { throw e; } } if (page.responseCode == 200) { // The page was found successfully. page.pageSource = this.xmlhttp.responseText; } return page; }

A few points:

• XMLHTTP supports both asynchronous and synchronous operation.  I chose to use synchronous for this demo.  The program stops running and waits for the request to finish before continuing on.  This is fine for the intial Random Walker implementation, because we'll just run many copies of the program in different processes to simulate load.
• You'll notice that headers are both read and written depending on the state of the cookies.  This is important for our purposes because we want to simulate a single user in a single session.
• If the request type is POST, we can send data (the format is "field1=value1&field2=value2") to the server.  If this is not specified, XMLHTTP likes to be passed null.
• Sometimes the server does not ask us to set a cookie.  The try/catch handles the fact that XMLHTTP throws an error when you ask for a non-existant header.  This is exactly the sort of component-specific implementation detail that this wrapper is designed to hide.
• XMLHTTP follows server-side redirects automatically (ie: HTTP response codes 3xx).  Other components do not, so that would have to be handled by our wrapper.  Again, we're trying to simulate the user's experience: their browser follows redirects too.

One additional method is required to handle the addition of cookies.  JScript arrays are extremely flexible.  We'll use the cookie array as a hash.  If the cookie was already set on a previous page request, it will be overwritten without duplication.

Browser.prototype.SetRequestedCookies = function( cookiesHeader ) { // precondition: cookiesHeader specified if ( !cookiesHeader ) throw new Error("You must pass a Set-Cookie header string to Browser::SetRequestedCookies"); var regEx = /Set-Cookie:\s*([^=]*=[^;]*)/i; var cookieMatch = regEx.exec( cookiesHeader ); this.cookies[cookiesMatch[1]] = cookiesMatch[1]; }

That's really all there is to the browser object!  Here's a sample of it in action:

var b = new Browser(); var p = b.GetPage("http://www.google.com/", GET, ""); WScript.Echo("Response Code = " + p.responseCode); WScript.Echo(p.pageSource); WScript.Echo("Took: " + p.timeToLoad / 1000 + " seconds.");

The Browser object is both very simple and very useful.  We can automate all sorts of things with it!  Put a bunch of Page objects in an array and we have a session!  We're going to do just that in Part Three of this series: a simple Random Walker.

Please feel free to download the Browser code here: Download Browser.js (2.1K)

[Many thanks to Alex Gorbatchev's dp.SyntaxHighlighter for making this post painless.]