Overcoming front end testing challenges

This month we are showcasing how Judopay achieved our black belt in frontend test automation.

Let me introduce you to Eugene, Judopay’s Senior Test Automation Engineer, who made this achievement a reality, overcoming the frontend challenges he faced, amongst the complexities of testing within the payments industry.

‍

Eugene Zhernakov
Senior Test Automation Engineer

I am an experienced and multi-disciplined test automation engineer with over 8 years across fintech, media streaming, and banking industries.

Deeply passionate about all things quality, with a keen interest in using automation to drive improvements and deliver best-in-class software.

I've only been at Judopay for less than a year and the team has been so welcoming, it feels like I've been here for much longer.

With this Engineering Blog I aim to explore the challenges we faced with the sizeable testing effort, and how we worked to overcome it with the power of automation.

‍

Software testing at Judopay.

Software testing is a pivotal part of our development process at Judopay.

We offer a variety of products to our clients including:

• Web based SDKs
• Native and hybrid Mobile SDKs

   resulting in an extended test matrix and coverage, to ensure quality throughout.

‍

Challenges…

Time is money as the mantra goes, and it is no different with quality assurance, especially in the payments industry, that is online 24/7 throughout the year.

Historically, we had a few manual processes in place, such as a lengthy test suite spreadsheet containing countless positive and negative user flow checks, across our whole product offering.

Our development approach is very agile focused, aiming to release as often as possible. This results in a testing effort that would take up a considerable portion of the release timeframe.

Manual testing would take a single experienced engineer half a day per product - multiply this by several apps and suddenly you are spending an entire day just running through test scenarios - that is without even reporting, or debugging any issues we find!

As with most highly functioning teams in our industry, it is common to keep teams quite lean, meaning resource stretched between testing:

• Existing backend
• Frontend changes
• New features
• Bug fixes

Each development team works to their own timescales and sprints are generally staggered, but during busy times there can be a pileup of testing to be done. This can result in holding up releases.

The obvious path to take was implementing automated tests to:

• Save time
• Increase our test coverage
• Free up busy test engineers for value adding endeavours.

‍

Automating our testing scenarios

Starting with our Web Products…

We started by narrowing down any low-hanging fruit, such as automating the web based scenarios as part of our Web SDK and Web Payments suite of products.

We primarily use Playwright as the test framework of choice, coupled with Postman to use and maintain a suite of API calls, with various configurations to cover all our bases through end-to-end testing.

Judopay seamlessly integrates into a dozen gateways and services, which perform the actual processing of payments.

One of the major challenges we faced, was storing and maintaining test data somewhere secure yet within a collaborative environment.

Personally, this was a fairly steep learning curve for me. Once I stepped back to look at the bigger picture, I could understand the need for this complexity. The result is an unparalleled and live look into all the gateways we support, with constant status and availability updates of each.

Instead of earnest searches and complicated queries on Splunk (don’t get me wrong - I still get a good few of those from time to time) to pinpoint an issue, our tests reveal which:

• Gateway
• Product
• Environment

is experiencing issues, thus shaving off hours of debugging.

As a result, our regression test execution is 95% faster compared to when all tests were executed manually.

Next our mobile products…

Tackling our Mobile SDK testing was the next step, and this was an entirely new mountain to climb in itself.

We offer Mobile SDKs that covers both native iOS and Android, in addition to their React Native counterparts on both platforms.

With a release containing any significant changes to the underlying SDK, you have your hands full with testing multiple targets, never mind the exhaustive list of mobile test devices available.

To fully take advantage of the native frameworks and enable us to easily test the more difficult or awkward functionality within our apps, we chose XCTest for iOS and Espresso for Android.

At first we considered using the likes of Appium or another Selenium-based framework, but through a team discussion we determined we can utilise the native frameworks better. Also, tests can be maintained by developers, in the absence of our test engineers.

Any time code is pushed to the main branch, a build is kicked off on CircleCI and the resulting test build is pushed to Google Firebase, where a suite of tests are executed on a real device.

Reports for mobile SDK test results are published to Slack as soon as they are available. Google Firebase is very useful in this regard, with the service providing full logs of the test execution and helpfully a video of the device as the tests run.

Automation has improved testing in our mobile payments space, saving our test engineers a substantial effort of manual testing, approximately 95-98% of the previous time taken.

iOS - More Finesse Required!

We have several sample apps as part of our iOS SDK offering and whilst we do not publish these to the App Store, they are a key component to internal testing and helping merchants with their own integration to our platform.

As with most things Apple, the configuration and setup is a tad bit more complicated than Android. We needed to make sure the sample apps were code signed with the correct provisioning profiles before they are packaged up and sent across to Firebase Test Lab for testing on real devices.

Since CircleCI only supports Fastlane for handling iOS code signing, we had to go down this route, fully integrating Fastlane into our pipeline.‍

Match handles all the profiles and certificates, pulling them down from the Apple Developer Portal and keeping a private repository up-to-date, enabling CI/CD instant access to the latest versions.

Finally, Scan generates a build of the app to send across to Firebase and the test suite is executed on real devices.

By investing the extra time and effort into setting up this pipeline, we cut valuable developer and QA time spent on generating builds manually and removing the dependency on team member availability.

As an extra win, we can now also sign and distribute test builds to Firebase for internal testing.

‍

Continuous monitoring.

Reports are published to Slack throughout the day, giving us instant updates on the health status of both our frontend and backend products.

Granted, sometimes our Slack notifications can be a little overwhelming, but we find that by jumping into a quick thread around a failure, we are able to deduce an issue quickly, efficiently and act on any bugs, before pushing to production.

An added advantage of implementing continuous monitoring, is the use of scheduled runs multiple times a day to guarantee the stability and robustness of our end-to-end environment. Through this proactive approach, our team can swiftly detect and promptly address any potential issues that may arise.

By ensuring a seamless experience for our clients in both web and mobile payments, continuous monitoring empowers us to maintain high levels of reliability and confidence in our platform.

‍

What is next?

We now have a solid foundation in place, that will help reinforce the feedback loop during development. However we can always improve and further extend our testing capabilities.

With this in mind, there are several goals we aim to achieve in the near future, to expand our test automation scope.

Cross-browser testing.

Playwright frameworks supports executing tests against several targets such as Chrome, Safari, Firefox and Microsoft Edge - and we plan to utilise this as part of our testing.

We have scripts in place which allows us to perform ad-hoc testing against a chosen browser version, using the Sauce Labs cloud platform. The end goal is having our test suite run against numerous web browsers and screen resolutions in the cloud.

New relic synthetics.

With what are called synthetic tests, we would have the ability to run end-to-end tests against a production environment, including setting up alerts to notify us of any breaking changes before our users do.

Taking into account strict PCI compliance and regulations, we would only be able to sanity check the frontend without any payment or personal details. However, this could be a powerful last line of defence immediately post release.

Load and performance testing.

Our business is 24/7, meaning our Transaction API experiences peaks of through-put during busy periods. With this in mind, we absolutely must ensure our backend (and frontend to a certain extent) performs to the highest degree, and responses are kept to mere milliseconds.

With tools such as Artillery, BlazeMeter or K6, we will be looking to spin out another test suite to verify the performance of the applicable Transaction API endpoints.

By harnessing the power of automation, our test engineers can now allocate more focus and resources towards testing new features and delivering enhanced value to our customers, elevating both our web and mobile payments solutions.