Web Testing and QA - It's not as straight forward as you think

What is involved

Testing is not just clicking on all the buttons to make sure they work - there are so many other things to be aware of and test. Namely how the website looks and functions across the plethora of browsers that are now widely used. Not just the browser but also the versions per browser, Operating System, device size, as well as the user journey etc!

Browsers

There are something like 80 different browsers - of course a lot of them are outdated but if you think about the main 6 (Chrome, Firefox, Safari, Netscape, IE, Opera) even within these Internet Explorer, for example, has at least 4 current versions (IE 8-11) that are still widely used. See a full list at Webdeveloper Notes.

Image: Main 6 Browsers

Operating Systems

Then there are the Operating Systems (a software program that enables the computer hardware to communicate and operate with the computer software), of which there are so many versions for both PCs and Macs (broken down into 3 main types of Windows, Macintosh, Linux - see Wikipedia's full list).

Then mobile phones have a different OS again i.e. (Apple iOS, Windows Phone, Google Android etc)!

Device size

And the type of device that is being used i.e. Computer, Laptop, Tablet, Mobile phone along with the various screen sizes widely available.

Image: Every box on this chart represents a different Android device that developers have to try to support!

Even the BBC find testing hard! In the Smashing Magazine article ‘Prioritizing Devices: Testing And Responsive Web Design’,  Tom Maslen (a web developer working in the Visual Journalism team for BBC News) states

“Because of all the things we have to support now, testing has become really, really, really difficult and also super-expensive."

"Before responsive web design, we had to test five web browsers on a desktop computer. Now with responsive web design, we have at least 15 browsers working on a myriad of different-sized devices, with many different input types, multiple pixel resolutions and hugely varying connection speeds.” and “Because of all the things we have to support now, testing has become really, really, really difficult and also super-expensive…..With unlimited time, money and human resources, you’d be able to test your entire website on every device, every day of the week. In the real world, even large organizations like the BBC don’t have the resources to attempt this perfect solution.”

There is so much testing that can and needs to be done that there are numerous companies dedicated to doing just that and have built a strong business on this vital work. But it should also not be ignored that the client has a large part ot play in testing too. Clients need to take responsibility for their own websites and test to make sure what they want to happen happens. It is a team effort between the development company, design agency, client and testing company that may be involved.

Everplaces (a community where people who love travel, food and design share recommendations) have built 70 apps, both for Android and iOS and they state that 

“...platform providers keep coming out with new updates you have to be compatible with. For example, when Apple released the iPhone 5 we suddenly had to make all our apps fit the new taller design of all the pages (iPhone5 is taller)."

"Then Apple came out with iOS7, the new operating system which has “flat design”. This meant that to keep looking good, we had to redesign the icons, reconsider several pages, and had to decide what to run with in the changeover period, where we have users on both systems...Recently we spent a month building a deep Facebook integration, which was offered to us because we have a special partnership with Facebook. Four days after we released it with big hurrah, Facebook changed something at their end and a lot had to be rebuilt.”

The frustration is widespread from the developers point of view and the clients in terms of the time and effort to have to re-do something you’ve spent so long doing and for the clients the cost and stress of ‘looking bad’. We’ve had instances of facebook or twitter having changed something in their code (but not announced it) which has then affected how we integrate with them and we as the development company get the flack.

Most clients aren’t aware and often annoyed at the fact that a website isn’t programmed and then works perfectly forever - there will be bugs and issues that need to be fixed, tweaks and constant improvements to make. Websites are like living organisms - in order to keep them alive and working effectively for your business you need to think about frequent updates and improvements.

Summary

As a client please be aware (and not dismissive) of all the hard work that goes into creating your website, portal or application and the huge amount of testing variants and user journeys that have to be considered at every stage in the development progress.

Technology waits for no-one - if you want to stay ahead, you have to make sure you have set aside budget for keeping your website up to the latest standards to reach the widest and/or your target audience depending on what devices they are using, how they are viewing/using your website and what payment methods they find easiest (if it’s an ecommerce website). So as well as development budget, please also consider having a testing and improvements budget and allow time in the schedule too, as this is often a major factor where expectations are unrealistic.  

Webigence have built and support over 300 websites. To see the kind of larger sites we have programmed please look at our case studies and do get in touch should you wish to talk to us either by phone 020 8739 0030 or email info@webigence.com.

Blog written by Natalie Wiggins