Python for Test Automation APIs

(See the beginning of this series here)

So here at Adept Labs we build test automation frameworks a lot, and we’ve learned a few (ok, more than a few) tricks that make for really flexible and, frankly, pretty cool systems. That is, if you’re the type of person to see some code and say, “Wow! That’s cool.” I know I am. So what does Python offer for building good APIs for test automation?


 1. Positional and Named Arguments

Ok, we’ll start simple and build from here. But if you’ve ever tried to maintain a test automation system without this feature, you probably know a bit of pain. I once worked on a test automation system in Java, and method signatures became the bane of my existence very quickly. If I wanted to add an option to an existing automation API function, there were a few options, none of which made me happy.

Adding a new optional argument to a method changes its signature, so by doing this every single test using the old one needs to be updated. Nope, can’t do that. Every update to the API can’t cause changes to every legacy test.

Ok, so we can overload the method to have a new signature with the new parameters. Fine, that works… for the first few updates. But we want these tools to live and be maintained easily for years. what happens after this and a dozen other API calls get a few new options each? Each API call may have several overloaded definitions. Even if you abstract the actual logic into a private method, we’ve suddenly got piles of code for no real benefit. If you want your API to remain backward compatible (and you do), once you publish one of those method signatures, it will live for a very long time. You can easily spend hundreds of lines of code on this approach, just adding new options to existing methods. Yikes.

Lastly (yes, I’ve read Effective Java, and it happens to be one of my favorite programming books ever), we can follow a factory design pattern. Instead of accepting a bunch of parameters to the method, we’ll accept a single object of type SomeApiCallOption. Then we just need to update that “options” class with a new method setting the new option. It works, and it’s what I resorted to in a few places, but the code is unnecessarily verbose (insert slight jab at Java here). You end up with something like:

NetworkTrafficOptions nto = new NetworkTrafficOptions();


Enter Python, where the start_traffic API call could instead have required parameters as positional, and take everything else as kwargs. Adding a new option means updating the docstring, because you’re a good API provider that documents all options, and having the method check for the new option and handle the default case when it’s not there. No extra classes, no boilerplate code. Existing uses of the method work as expected, and new ones can take advantage with a new keyword option. Pretend that the L4 Protocol is the new option. In Python it might look like this:

start_traffic('', '10m', l4_protocol='tcp')

And that kwarg for l4_protocol can be made completely optional, which is good for backward compatibility. This is extremely convenient when there are half a dozen optional arguments to a function.

2. Everything is an object

That line gets thrown around a lot in Python, and for good reason. It is good to keep it in mind when building a test automation API. Instead of getting philosophical, let’s look at a very real example where this can translate into some very slick functionality.

In test automation, we very often perform actions that need to be undone later. The trickiest part about this is keeping track of which things need to be done in which order, especially when test cases can branch or even die in the middle. We need to know exactly what to roll back, based on how far we got down the test path. Because Python functions and methods are objects, they can be passed around just like everything else. This means that your test API can keep a stack of “teardown actions” that need to happen, which will be a collection of other API functions to call, but wait to call them until it’s time. The stack only contains the “undos” for actions that have already taken place.

Let’s get concrete. Suppose we have the following

def configure_interface(network_device, interface, ip_address):
    network_device.cli.send_command('ifconfig {0} {1}'.format(interface, ip_address))

def unconfigure_interface(network_device, interface, ip_address):
    network_device.cli.send_command('ifconfig {0} delete {1}'.format(interface, ip_address))

It makes sense that there is an “undo” relationship in configure_interface for unconfigure_interface. But we don’t want to run that immediately, we want to remember that when it’s time to tear down we want to call unconfigure_interface with certain options. So it might look like this:

def configure_interface(network_device, interface, ip_address):
    network_device.cli.send_command('ifconfig {0} {1}'.format(interface, ip_address))
    add_teardown(unconfigure_interface, [network_device, interface, ip_address])

def unconfigure_interface(network_device, interface, ip_address):
    network_device.cli.send_command('ifconfig {0} delete {1}'.format(interface, ip_address))

Notice that we pass the function itself, with no “()”.  The add_teardown function will keep a list of functions (and their arguments), and when it’s time to undo the test actions it will iterate the list in reverse, calling them with the given arguments. Similar solutions in some other languages are very messy, since it’s not valid syntax to pass around the function itself. Yes it’s still possible in other ways, and yes other languages can do it this way too. But in Python it’s clean and easy, and has some very tangible benefits.


3. Meta Programming

Stay tuned for the next installment where we will dive into what some of Python’s meta programming features can offer for a test automation system. This is where it really starts to get fun!

Posted in Automation and tagged .