To be consistent with my “what I’ve been up to” style of blogging I wanted to talk about the benefits of creating a functional test framework and what that framework might look like.

If you’re like me then you probably think that automated functional testing using Selenium is really cool. If you aren’t like me, but automated functional testing is a necessary evil in your position, then simplifying your interaction with the entire experience should be really cool. In either case, creating, employing, or utilizing a framework for your functional tests can drastically improve your testing in a way that benefits the entire team.

Computer with magnifying glass and gears

The Order to Things

A member of our team explains functional testing like this; instructions sent to a man who lives in our server, who carries out those instructions on his computer in the exact manner in which they were sent to him. This really isn’t that far off. One of my largest pain points in writing functional tests was that sometimes I would ask this unnamed guy masquerading as computer hardware to click on an element I expected to be on page, but forgot to tell him to wait for it to show up. In fact, I found that there were usually always a certain set of steps and protocols that I should always follow when performing the simplest of actions.

For a click, I need to wait until my element exist on the page, then I need to move my mouse to that element, I need to wait for it to be in a clickable state, and I need to perform the click itself. While I can easily handle this in the test itself, it probably isn’t the easiest place to manage it. Instead, I can setup this sequence as a method in a framework, and call to it in the test. This works well for a couple reasons. The first being that in my test I only need to write one line of code instead of 4. Secondly, with this action handled in the framework I never accidentally forget a step. By doing this I am making my test easier to write, easier to read and I am ensuring consistency across the scope of potential actions.

public static void Click(IWebDriver wd, By by)
        {
            try
            {
                WaitActions.WaitUntilElementExist(wd, by);

                var element = wd.FindElement(by);

                FindActions.MoveToElement(wd, element);

                WaitActions.WaitUntilElementClickable(wd, element);

                ClickElement(element);
            }
            catch (Exception ex)
            {
                throw ex;
            }
        }

Don’t Make Me Repeat Myself

This idea of repeatable code extends beyond the steps to perform basic actions in tests. In our functional test infancy we handled most things at the test level. The framework aims to handle most of that work upfront, so the focus can shift to writing the test instructions themselves. The framework handles things like setting up the driver or Eyes for Applitools, in addition to addressing the different actions we can take in our tests.

Another measure along those lines is taking sections of the application that are tested repeatedly and moving them out of the individual tests so that they can be called upon in multiple tests when needed. When we need to update any of these sections, we can do it in one place. If a header on the site changes, we do not have to update every test that interacts with the header, but rather reference the portion of the project that handles interacting with the header.

public void CheckHeader(IWebDriver wd)
        {
            var header = FindActions.FindVisibleElement(driver, By.ClassName("header-inner"));

            var logo = FindActions.FindVisibleElement(driver, By.CssSelector(".logo-inner img"));

            var topNavLinks = FindActions.FindVisibleElements(driver, By.CssSelector(".mobile-nav .dropdown"));

            var secondLevelLinks = FindActions.FindElements(driver, By.CssSelector(".dropdown-menu li"));
        }

Batteries Included

Another cool feature of this framework is that it was made into a NuGet package that can be included in any projects at our company. If I want to create a new functional test solution to go with my existing webapp, I can simply pull in the package and hit the ground running. The NuGet package has all the references it needs already included. The setup of my driver and other essential classes have already been handled in the framework. I can immediately start writing tests. If I focus on the repeatable pieces of my application and create those methods in my best test, I can gain a good bit of test coverage on my application with relatively little overhead.

Batteries

Simplicity for Adoption

What I hope proves to be the largest benefit of creating the framework is promoting adoption throughout the team because of its simplicity. Team members who were unfamiliar with testing, or more specifically, testing with Selenium and the nuance associated with that, will be able to test simply and reliably with little overhead. Before, we were asking a test developer to understand the finer points of Selenium and how it interacts with the browser to create effective tests. Now, creating a test should translate much more simply into what interacting with the application feels like in real world terms – find this element, click this button, type into this search bar.

Simplifying how testing is done and lowering the barrier to entry on who can create tests should allow for more testing. Troubleshooting and understanding tests should be much easier for developers that interact with the test after they have been created or product owners trying to understand what kind of coverage they have on the application.

Do Less. Get More

A functional test framework will allow your company to do less work and get more from it. The framework ensures that tests follow the required protocols with more consistency and less overhead. The framework reduces the amount of work and shortens the time required to get a test project up and running. Finally, simplicity in testing can promote adoption throughout the team, leading to more team members creating tests and ultimately more testing as a result. Having a framework in place is essential to any functional test program.

As a relatively new architect-level hire at Mercury, I have spent a large amount of time exploring code bases that are new to me. In that exploration, I have come across a variety of source code peculiarities and problems related to issues with naming conventions. These are fairly rare situations and as such can take a while to diagnose and get to the true root problem. As a whole, naming schemes can vary greatly between organizations, with some of the bigger names like Microsoft contributing their recommendations on best practices. The issues I will describe below are specific to situations in Mercury’s development process, but might help to identify possible pitfalls in yours.

Case Sensitivity Differences in .NET Framework

The .NET framework allows development in VB.NET as well as C#, with interesting differences in case sensitivity. C# is a case-sensitive language, which means that it can recognize the difference between the following methods:

void myFunction()
void MyFunction()

This is definitely not a best practice programming-wise, and coworkers might frown upon your decision here, but according to the framework, this is allowed. However, VB.NET is case-insensitive, meaning that it cannot allow the same methods with the only difference being capitalization. This also applies to parameters in a method signature, namespaces and properties.

Where this could cause conflicts is when introducing compiled code into your solutions. For example, if you are working in a VB.NET solution and you introduce a NuGet package which was initially developed in C# with two methods that vary only in name capitalization, you are going to face compilation errors.  It is not always easy to track down what that NuGet package might be doing, so keep this case in mind.

Variable names in JavaScript

Similar to the above, it’s important to realize that JavaScript is also a case-sensitive language, it identifies a difference between the following variables:

var myName()
var myname()

This is something that you can potentially come across when working in either a vanilla JavaScript project or while using a JavaScript library such as jQuery. It’s more likely that you will come across this situation when using a full JavaScript framework, such as AngularJS or ReactJS.

Especially if you are integrating with a webservice of some kind, you could face issues with the naming convention coming from the service versus the binding used in the framework. In our case, we were integrating with a WebAPI service which we had developed, and we had specified the following naming for the output from our WebAPI:

config.Formatters.JsonFormatter.SerializerSettings.ContractResolver = 
new CamelCasePropertyNamesContractResolver();

This says that, regardless of how the naming convention is utilized in our WebAPI project models, the output is going to be such:

startsWithSmallLetter

The issue in our AngularJS application was that we were reading most properties correctly, with the camelCase formatting, and a very large application was working 95% as intended. Later we noticed that one of the columns in our database was always null. Backtracking the problem, from the models through the WebAPI layer and into the AngularJS application, we discovered that we had improperly capitalized the binding in this way:

ng-model="model.AccidentallyCapitalizedFirstLetter"

Because JavaScript is case-sensitive, this created a new property to our JSON object:

 var model = {
            AccidentallyCapitalizedFirstLetter: "",
            accidentallyCapitalizedFirstLetter: ""
        }

Whatever was entered into our application on the front-end was never properly bound back to the WebAPI for saving to the database, and this information was perpetually lost. We had discovered a very small mistake, with a large effect which wasn’t easy to determine while debugging through a missing data issue.

Confusing Third Party Namespaces

Sometimes the naming issues you run into aren’t your fault or even in your control: we have faced several instances of plugins and third party tools which are nearly identical in name to other tools, sometimes differing by only one letter. Sometimes the plugin is named such that it seems like it would be supported by a major corporation, but in fact it’s one developer trying to exploit the similarity. One non-development example is the court case of Nissan Motors vs. Nissan Computer, two different entities with claims to the same name.

In our case, we ran into an issue relating to Microsoft and a SDK they provide to access their cloud-based APIs. In 2014 a version 2.0 was released for what they called the ‘Microsoft Azure Graph API Client’, a tool to access Azure Active Directory and a few other Azure resources in code. This SDK was made available via NuGet, Git and Microsoft’s site.

In 2016 Microsoft released a new tool, this time called the ‘Microsoft Graph API Client’. This incredibly similar name is a tool replicating the complete set of functionality of the Microsoft Azure Graph API Client, with some additional features of being able to access Office365 and the rest of Microsoft’s suite of cloud tools. Both of these tools are currently supported, and because of the cross-over between functionality, it’s very likely that either or both will come up when doing any form of web search.

If your architecture is currently setup to support one of these versions of the Microsoft SDKs, then it’s likely that you want that specific library instead of the first one to come up from a search. This could become even more confusing when a newer developer is tasked with extending some functionality and chooses to add a new library to the existing stack because closely named projects introduce ambiguity.

Apache Hosted URLs

Probably one of the most interesting, and hard to solve issues, that came up as part of a naming-related problem, was related to an Apache hosted application. In this case, we were using a set of testing tools, and in code we would reach out to those tools with a standard HTTP-based web request. During the lifecycle of these testing tools we had consistently been working with a naming convention like this:

www.our-testing-appliance.com/InitTest

With this approach, we had never run into any issues that we could notice, and we were initiating functional testing using this location across dozens of different projects. Considering that our approach was a successful one, we looked into refactoring our testing solution into a NuGet package. This is where we started to run into errors.

Specifically, in our NuGet package we defined static classes to initiate the connection to the testing application. Our implementation of this library would simply define a test, and the results of this were coming back with the connection being denied. This was surprising, especially given that we could go back into an older application immediately after and have a test work successfully.

With some debugging of the network, we determined that the URL above was giving a 301 response, it was redirecting to the correct URL:

www.our-testing-appliance.com/inittest

Since the Apache host is case sensitive, it wasn’t responding to the two URLs as the same destination, and instead was redirecting from the capitalized case to the lower case. The static class implementation was not handling the redirect, whereas our previous implementation did follow the redirect and get the response from the new route.

Conclusion

These issues nearly all turned out to be minor problems in code, but each gave different indications as to what failed. They all started out as some different symptom, either database oriented or networking related. Starting from an assumed problem, it was surprising to see how much of an effect capitalization could have, beyond just making everything a bit more confusing. In the case of third party naming conventions, I’ve learned to be very clear in my documentation, to include links, and to not trust external libraries to always follow sensible conventions.

References