#3 Judopay's Sho~dan Programme
Following the second blog in our series: Creating Judopay's Next-Gen Tech Platform, we looked at how Judopay’s engineering team:
- Were faced with an outdated system.
- Rebuilt the core systems onto a new tech stack.
- Delivered a next-gen payment platform as part of their Sho~dan programme.
Using tech to challenge the status quo, and using it as an opportunity to innovate, we are very proud to share with you how we got there.
Introducing Judopay’s Sho~dan Programme
What is “Sho~dan”?
– [ ʃo • dɪn ]
sho – Japanese word for first, new or beginning.
dan – the black belt ranking system commonly used in martial arts introduced by the founder of Judo (the sport, not Judopay…)
Sho~dan is not just a play on our name, it has a deeper, more meaningful relevance as the name chosen for Judopay’s new tech vision.
Sho~dan is the first ‘dan’ black belt in Judo martial arts, and represents a recognised elevated level of technical proficiency. However, you don’t get to be a black belt overnight! Your journey beforehand provides the foundation on which this new level has been achieved.
It is also not our final destination. With continued perseverance and commitment we can move up the ranks, improving and evolving our platform beyond the level of Sho~dan.
If Judopay’s tech stack was stable and worked for its imminent demand, why embark on the Sho~dan initiative?
I asked David Judopay’s Head of Infrastructure and Automation Engineering, why embark on the Sho~dan initiative?
“In 2019, our engineering team was facing a number of challenges. Judopay had just spent the first 7 years of its existence building and running a .NET monolith.
During this time, the company pivoted multiple times, drastically changing the requirements of our systems.
The need to move quickly and adapt to these changing requirements introduced a tremendous amount of technical debt.
The system had become so convoluted and difficult to work with, we felt it would be easier to start from scratch.
This was not a light decision! Rebuilding your core business systems is incredibly difficult, costly, and risks disaster. It is important to carefully consider the scope of the work you are taking on.
In our case, we felt it was the only option which would allow us to meet our long term business objectives. Working within the constraints of the existing system had simply become far too time consuming.”
The Core Foundations of Sho~dan
David reflects on how Sho~dan, Judopay’s next-gen tech platform came to life.
Sho~dan came to life as a result of our desire to meet the following goals:
- Replace our monolith with a microservice-based architecture in order to:
- Enable more dynamic scaling
- Enforce domain driven design principles
- Reduce the size of deployables in order to enable rapid and frequent deployments. - Reduce the complexity in our existing systems by re-implementing software to meet today’s requirements, rather than the requirements of 5 years ago.
- Take advantage of cloud platform capabilities to:
- help us scale quickly
- leverage managed services to save us time building solutions to already solved problems. - Automate as much as possible to reduce manual toil.
- Shift the mindset of the team to building small, re-usable, consistent components and adopting functional programming paradigms.
We call these re-usable components the Sho~dan microservices framework. - Adopt reactive software principles in order de-couple system dependencies and benefit from improved flexibility and scalability.
Sho~dan is about scalable, robust and consistent software delivery, focused on maximising business agility, enabling the rapid, frequent and reliable delivery of large, sophisticated applications without needless complexity.
The rest of this blog discusses one aspect of Sho~dan - our microservices framework.
Decomposing Complexity with the Sho~dan Microservices Framework
As part of Sho~dan we wanted to go a step further and create a common microservices framework upon which all of our technology is formed to produce services that can be easily built, replicated, updated and managed.
At its core this involves components that together represent a toolkit with which to build microservices, confident in the knowledge common - yet often hard - concerns are already catered for. These can include the following:
- Data obfuscation
- Security standards
- Testing standards
- Inter-service communication
- Persistence
- Metrics, Logging & Monitoring
- Concurrency
The Sho~dan Microservice Framework uses this toolkit to provide a reference standard of common service templates and rich programming paradigms to enable the rapid, frequent and reliable delivery of large, complex applications.
As delightfully illustrated below by the Judopay mini microservice robots, each robot represents an independent application or service.
Required components can be imagined as ‘slotted’ into any given robot, as illustrated below, becoming one element of the whole project.
Developers can then work with the confidence and trust the core foundations for each project, are already correct and stable.
Working in this way allows our developers the freedom to focus on writing their specific business rules and workflows required for a feature, rather than the core builds for the project, delivering value quicker.
Microservice templating also means that while each distinct service has a specific purpose, the shared commonality between services makes their codebase easier to recognise and work on across developers. This helps reduce the overall system developer cognitive load and contain platform complexity.
Sho~dan Inside
Aside from building services and delivering features quickly, the Sho~dan Microservices Framework means across any project we have certain Sho~dan “guarantees”. For example:
- QA
- Regulatory & compliance requirements
- Security requirements
- Project structure
- Implementation patterns
- Service performance & availability
Allowing our devs the freedom to focus on business logic and deliver value quicker = lean and agile solution delivery.
The codebase is not the only place to manage complexity. Manual repetition and capacity management can also represent other forms of system complexity that hinder agility.
Evolution into a Common Library…
Philip, one of Judopay’s Senior Engineers, shares his insight into how identifying common programming patterns evolved into a common library:
“It is worth mentioning, we did not set out with a view to standardise common code into common microservice components.
Rather, as we began to flesh out more services on the Sho~dan project, we realised we had some potential common programming patterns.
For example:
- Connecting to our Postgres database
- Adding tracking metadata to log messages
- Configuration loading
So, rather than copying and pasting these patterns into each new service, we decided to move them to a common library. It was then easy to import them and reuse these into the new services we were writing, to avoid needless duplication.
The library grew from there. We added more and more shared code, until we found we had built our own extension of the Lagom framework.
As such, it was something that evolved. We learned and discovered what would be useful common code, rather than trying to design something from the start.”
A Golden Rule…
To quote Philip again,
“As mentioned previously, the development of the common code/common library was not predetermined, we did however adopt one principle early on:
We decided that the common microservice components should only ever be a suite of “software tools” that enable us to build microservices quickly, reliably and consistently.
A service should own its business logic, even if this logic is repeated across services.
What I mean by that is:
Our next blog in the Sho~dan series will provide a more in-depth look at how we have applied automation engineering and scaling to our microservices.
