Checking HTTP status codes from your Selenium/Java tests with the Meddler extension

• Add Your Comment
• 30 July 2018
• Tags: extension, firefox, meddler
• Posted in Browsers, QA and Functional Testing, Selenium

Over the years, I’ve see a couple of questions repeatedly on the Selenium boards related to HTTP status codes: How do I check for broken links (404s)? How can I check the status code of a web-service request made from my web-app? The answer is usually “Selenium can not do that” because, of course, Selenium is a browser automation tool – not a full-featured testing solution. Other answers suggest various solutions…none that I’ve seen are elegant.

This problem came to my mind last week as I was considering a task that will require writing a browser extension to solve a different, but related, problem. That extension will need to solve a much larger problem, so I was looking for ways to break down the solution into something smaller and more manageable while I learned about building and deploying browser extensions. This seemed like a perfect fit. So I set out to solve the problem of getting status codes from HTTP transactions that the browser has issued, during a Selenium test.

The solution consists of 2 main pieces:

1. Meddler Browser Extension
2. Meddler Gateway (Java lib)

Within the Gateway library there are two classes that the unit test interact with: the ExtensionConnection and the UrlStatusCounter.

Meddler Browser Extension

The extension listens to the HTTP transactions made by the browser and sends relevant metadata to the gateway. The extension must be installed into the user profile when the browser is started by Selenium. This is done by adding it to a FirefoxProfile and then using that profile to create the driver and start Firefox. If you are already creating a profile, then just this one line of code is needed in your driver setup:

profile.addExtension(new File(EXTENSION_PATH));

If not, you may need to add a few more lines, so that you eventually have something like:

FirefoxProfile profile = new FirefoxProfile();
profile.addExtension(new File(EXTENSION_PATH));
FirefoxOptions options = new FirefoxOptions();
options.setProfile(profile);
WebDriver driver = new FirefoxDriver(options);

Note that the extension path must reference the extension file (.xpi). You can download it from the GitHub releases page for the project (see resources section at the end). I suggest keeping it with the driver implementation executables (i.e. geckodriver.exe, etc). To be clear, the extension is NOT installed via the Mozilla Add-ons site.

The extension is currently available only for Firefox. But porting to other browsers that support the WebExtensions specification should not be difficult. Pull requests are welcomed 🙂

Meddler Gateway

The MeddlerGateway is part of a Java library that must be included in your (Java/Selenium) unit test project. It serves as the bridge between the Meddler extension and the Java unit test code. It runs behind the scenes – it is started automatically when it is first needed and runs until the JVM terminates. It runs a simple web server (on port 8411 by default) that listens for a websocket connection from the extension.

Extension Connection

The ExtensionConnection is where messages sent from the Meddler extension will be delivered (by the gateway). It should (usually) be created immediately after creating the driver, like this:

ExtensionConnection _connector = ExtensionConnection.establishConnection(_driver);

URL Status Counter

To receive information from the extension, you must implement a MessageListener. There is a convenience implementation for checking URL status already implemented, called UrlStatusCounter. It will count the number of responses with a status code that match the provided conditions. This example is looking for 404 responses:

UrlStatusCounter _counter = new UrlStatusCounter(_connector, 404);

Writing the test

Lets put it all together into a Selenium unit test. I am going to show the simplest example: I have a bunch of JUnit tests that are checking various bits of an application with Selenium/WebDriver and I want to look for any 404s on the pages…and fail the test if they are found.

Setup method

First, the setup method needs to establish the connection to the extension (see the ‘>>>’ lines). And then setup the UrlStatusCounter.

@Before
public void setup() throws TimeoutException, InterruptedException
    {
    try
        {
        _driver = DriverSetup.firefox();
>>>     _connector = ExtensionConnection.establishConnection(_driver);
>>>     _counter = new UrlStatusCounter(_connector, 404); _counter = new UrlStatusCounter(_connector, 404);      
        }
    catch (Throwable t)
        {
        if (_driver != null)
            _driver.quit();
        throw t;
        }
    }

The setup of the driver is hidden inside the DriverSetup.firefox() method, which is used in many of my unit tests. This one line adds the extension into the Firefox profile so it is available in all of my tests. Note that the EXTENSION_PATH should point to the extension bundle (XPI file). You will need to download this file – see the resources section at the end for a link.

public static WebDriver firefox()
    {
    System.setProperty("webdriver.gecko.driver", FIREFOX_DRIVER_PATH);
    FirefoxProfile profile = new FirefoxProfile();
>>> profile.addExtension(new File(EXTENSION_PATH));
    FirefoxOptions options = new FirefoxOptions();
    options.setProfile(profile);
    return new FirefoxDriver(options);
    }

Test method

For this use case, there is no need to add any code to my individual tests. Here is an example test with a URL that always includes a 404.

@Test
public void detect404s()
    {
    _driver.get("http://httpbin.org/status/404");
    }

Teardown method

Adding one line to the teardown method will fail the test if any 404s were found. The error message will include a list of the URLs that returned a 404.

@After
public void teardown()
    {
    _driver.quit();
>>> Assert.assertFalse("404s were detected:\n" + _counter.report(), _counter.getTotal() > 0);
    }

When matching transactions are found, the unit test will fail in the teardown and the UrlStatusCounter will report a list of offending URLs…which will look something like this:

java.lang.AssertionError: 404s were detected:
http://httpbin.org/status/404 
  at example.ExampleTests.teardown(ExampleTests.java:63)

With just 3 lines of code (per test file), you can check for 404s during every test in your Selenium test suite. Plus 1-4 lines in your browser setup. If your tests already share the browser setup and use profile, you can get by with just 4 lines total!

Checking multiple status codes

An alternate constructor for UrlStatusMonitor accepts a StatusCheck. You could check a variety of status codes by implementing the StatusCheck like this:

@Test
public void detectForbiddenAndAuthorizationFailures()
   {
   _counter = new UrlStatusCounter(_connector, new UrlStatusCounter.StatusCheck()
      {
      @Override
      public boolean countStatus(int status)
         {
         return status == 400 || status == 401;
         }
      });
   _driver.get("http://httpbin.org/status/401");
   }

This example uses a lambda to count any status code 400 or greater:

@Test
public void detectTransactionFailures()
   {
   _counter = new UrlStatusCounter(_connector, status -> status >= 400);
   _driver.get("http://httpbin.org/status/500");
   }

Conclusion

If you have questions, please open an issue on one of the GitHub projects (see below). Or use our contact form to talk with us about your Selenium test automation challenges.

Resources

The Java library (.jar) for the MeddlerGateway can be downloaded from the releases page of the MeddlerGateway project on GitHub.

The Meddler Extension binary (.xpi) can be downloaded from the releases page of the MeddlerExtension project on GitHub.

 

Resources

• Web Performance Center
• Reports
• Case Studies
• Tutorials
• Calculators
• Books
• Links
• Dictionary
• Research