Catchpoint Review

Catchpoint Domain Override

In my previous blog A/B testing on CDN improvement we shared how to use A/B test to demonstrate the webpage performance brought by CDN. No matter what tool we used, it required customers to change their webpages to insert some JavaScript codes.

What if our goal is to minimize customer changes during the demonstration/proof of concept stage? I am going to demonstrate Catchpoint performance measurement service to achieve this goal.

Well, let’s take one step further to avoid ANY customer changes!

Catchpoint is one of the popular performance monitoring services. Many enterprises and cloud service providers buy Catchpoint solutions to monitor their websites and Internet services.

One cool Catchpoint feature is domain sharding/override. When it does page load test, Catchpoint can be configured to change the hostnames, say from A to B, in the HTML codes. By doing so Catchpoint will GET the HTTP objects from host B instead of A.

Let’s use this original webpage as an example. 82% of the page size comes from four images. I am going to do an A/B test on it. Original webpage is Test A. In Test B the four images will be delivered from EdgeCast CDN rather than the customer origin server in Singapore. This simulates actual CDN deployment on this webpage.

Step 1: I set up EdgeCast CDN services to cache the four images from the customer origin.

Step 2: I set up Catchpoint to test the original webpage from several Asia countries. This is Test A.

Step 3: I configure another test to test the original webpage again and use the domain override feature to change the original hostname (of the host of the four images) to the EdgeCast CNAME I set up in Step 1. This is Test B.

Step 4: This step is not a must-have and I use it for Control only. I add a new webpage in the customer origin. The new webpage is the same as the original webpage but in the HTML code I change the hostname of the four images to the EdgeCast CNAME I set up in Step 1. This is Test C.

I run the tests for 24 hours. Online reports: Test A vs B, Test A vs B vs C.

Below screen capture shows the page load time, response time and availability test results of Test A and B in the last two days. It shows Test B performed much better than Test A, Average Webpage Responses are 3962ms vs 5131ms, a 23% reduction. We did the test WITHOUT any customer changes!

Test A vs B
Test A vs B

You may wonder if Test B is a good simulation. Below screen capture shows the test results of all Test A, B and C. Test B and C results are similar (Average Webpage Responses are 3959ms & 4089ms respectively). I find the Catchpoint domain override feature is a good simulation of CDN deployment.

Test A vs B vs C
Test A vs B vs C

Disclosure: My company is partners with several vendors

**Disclosure: I am a real user, and this review is based on my own experience and opinions.
Add a Comment