When Melio realised there weren’t any Machine Learning Operations (MLOps) engineers to hire, because the field is so new, we created a career skills matrix to guide our engineers’ career development. Here’s how we created our matrix and how you can create your own using our template.
It’s been a decade since Harvard Business Review called Data Scientist the sexiest career in the 21st Century. Data science career trajectories have splintered and increased exponentially in the past ten years as the AI industry matured. An array of spin-off job titles now flood the market. As a result, it’s becoming even more confusing for wide-eyed graduates and seasoned CEOs alike to figure out what skills are required to land a job or who to hire to solve their expensive data problems.
As the AI industry matures, companies have shifted from figuring out how to use AI to focusing on its commercialisation. This is where the new cool kid enters the block: MLOps Engineer.
What is MLOps?
MLOps is a multidisciplinary field that intersects data science, software engineering, and DevOps.
With MLOps, businesses can more effectively build and manage machine learning projects throughout the development life cycle. MLOps exploded in 2020 and 2021 because the industry figured out that the true differentiator of high-performing AI teams is that their initiatives are closely linked to business KPIs, and their outputs are measurable. In addition, because of the focus on commercialisation, management teams are now required to quantify the return on investments for multi-year AI projects.
Data scientists can no longer train and tune neural networks with billions of parameters without justifying the cost. The proof of concept to production gap needs to be closed, and it needs to be closed in a fast, continuous, scalable, repeatable and observable way. This is where MLOps comes in: MLOps engineers are required to integrate the machine learning models into the current business processes and ensure continuous delivery.
Data on MLOps job postings in the US shows the massive growth in MLOps job openings from November 2020 to November 2021. In addition, MLOps engineer ranks number 4 of the most in-demand jobs on LinkedIn in 2022. The demand will rise steadily as more companies scale their AI experiments into production.
Why the MLOps Skills Matrix?
Melio is an AI consultancy that operationalises and commercialises AI projects for our clients. We have a team of talented machine learning and DevOps engineers, and when our team started expanding last year, we began to search for an MLOps engineer to join us.
The MLOps engineer role fits perfectly between our machine learning engineers and DevOps engineers, facilitating knowledge sharing and fulfilling demands.
We already had a carefully crafted job description for a machine learning engineer and another for a DevOps engineer, so we thought: Why not combine the two.
As we formed the job description, we found ourselves demanding an intermediate engineer with every skill under the sun. We were asking for skills ranging from data cleaning to setting up infrastructure-as-code, and technology stack from scipy to istio.
Because MLOps is emerging and “messy", practices and tooling vary widely and constantly change. We would never be able to find an MLOps engineer who fits our ML engineer and DevOps engineer job descriptions because there simply isn’t a single person who could know and do that much. So, we decided to train and develop our engineers.
We have a strong DevOps and Machine Learning culture and have developed a career skills matrix to help our engineers track their growth in these areas.
The skills matrix is a tool we developed in-house that maps job descriptions to the skills required for the different job levels. A career skills matrix builds on a job description, so the engineers know what skills they need to work towards levelling up their careers.
For example, both junior and senior engineers need to write data pipelines, but the expectations for their competency will differ. Everyone will be in the driver’s seat of their career while knowing in advance where it is that they want to drive to.
How we created the MLOps skills matrix
From our experiences working with various clients, building our internal projects and following the industry trends, we came up with the typical skills an MLOps Engineer requires and deconstructed them into detailed requirements for each level of seniority. This template is then tailored for each of our engineers based on their aspirations. For example, we have a machine learning engineer who enjoys data visualisation, so she opted to include business intelligence over container orchestration when creating her career skills matrix.
An example of the career skills matrix
This is how we tailor the matrix to individuals.
Step 1: Map out the different skills required
At this stage, we did not worry about the specific tools and technologies. Skills are more high-level requirements that should be largely universal across teams, while tools and technologies vary according to teams and are too specific for us to emphasise at this early stage.
These skills requirements are selected based on various MLOps course materials (see the resources we used at the end of the post) and our project needs on the ground. We advocate for tree-shaped engineers, so we chose a balanced approach to ensure the matrix develops sufficient knowledge in the three disciplines that make up MLOps: data science, software engineering and DevOps.
Some buckets below may belong to one or more disciplines, but we grouped them under one for simplicity.
Data Science:
- Data pipelines & orchestration – The key skills we look for are building and orchestrating data processing, and modelling training pipelines.
- Model building – Data scientists are responsible to train models, and MLOps engineers help productionise these models. Therefore, MLOps engineers need to understand and be familiar with the tools these models are developed in.
- Model deployment – Engineers need to package the data preprocessing, inference and data post processing in a consumable way.
Software Engineering:
- Database technology– This is a crucial component in any data project. Both SQL/no-SQL will be required, and special databases like Graph and Time Series will be necessary depending on use cases.
- Backend technology – For us, it’s microservice Python and software engineering best practices, including testing, versioning and writing clean code.
- ML Architectural Patterns – MLOps engineers should be able to architect large-scale machine learning systems.
DevOps:
- Monitoring & Observability – The Day2 operation of the models is the bread and butter of MLOps engineers.
- Automation & CI/CD - The five components include Scripting, Continuous Integration, Continuous Delivery, Continuous Training and Continuous Testing.
- Cloud expertise – This is necessary as most of our projects are built and deployed in a cloud provider.
- Container orchestration – This is pretty much Kubernetes for us.
- Infrastructure Provisioning – The ability to provision infrastructure-as-code is a vital skill for MLOps engineers.
- Additional infrastructure – Here, we mean things such as networking and security.
We are a consultancy company focusing on accelerating the time to value for our clients, so we have a couple of buckets that are essential for our engineering consultants:
- Project management – Many of our projects have fixed scope and cost, so we are 100% dedicated to successfully delivering our projects.
- Storytelling – It’s super important to be able to help translate complex technical ideas to business metrics and influence project directions.
We acknowledge that it is challenging to meet all the criteria in the career skills matrix.
The goal is not to become everything for everyone but rather for you to map your strengths and weaknesses to your career aspirations.
Step 2: Fill in details for each role from graduate to a principal
This step is where our company’s architecture and tooling come into play. We are a cloud-native company, so microservice, containers, and declarative APIs are fundamental concepts for our engineers. In this step, we made our expectations clear and wrote them from a first-person perspective. To illustrate what this means, let’s unpack the data pipelining skill requirement from Step 1.
We deconstruct what our five seniority levels — graduate, junior, intermediate, senior and principal — need to know. Some technologies are on a more need-to-know basis. For example, Kafka is required when the project adopts an event-driven architecture. We don’t expect everyone to be proficient in every tool, but a senior engineer has most likely touched on all the technologies.
The idea is that all our juniors need to focus on foundational skills that build towards MLOps, and seniors can grow into more specialised skills.
How we set up our engineers to use the career skills matrix
At first glance, it can be incredibly overwhelming when given a tool like this. To minimise this, we’ve created a process for new joiners on how to start tackling their personal skills matrices.
Step 1: Duplicate the template and tailor it to individuals
The career skills matrix is a guideline of what the company expects, but you should always be flexible and personalise the matrix to individuals and their career aspirations. Engineers are more likely to stick to it because they’re the ones who put it down.
Here’s an example of the template in Notion. You are welcome to duplicate it and make changes for your organisation and different roles.
Step 2: Prioritise the learning journey
Get each engineer to fill out the career skills matrix and rank themselves based on each bucket and its details. Next, identify 2 - 3 areas they want to focus on for the next three months, and note down the success criteria. The Monthly Contribution Template is an excellent way to structure this.
Don’t pick more than three areas to focus on - it can quickly become debilitating. We find a focused approach is more accountable and trackable.
Step 3: Treat it as a living document and assess it every three months
Set up a quarterly session to discuss progress with each individual. Our team has weekly 1-on-1s for micro-adjustments and alignments of the skills matrix, but the quarterly sessions are for a more formal review.
We live in an agile world, so keep the career skills matrix alive and review it often to suit your needs.
Benefits of a career skills matrix
From the organisation’s perspective, based on the careers skills matrix, we can transform our promotional process into an unbiased meritocracy system rather than a tenure system. We can also better identify our organisation’s overall personnel gaps, enabling us to construct the best teams for our projects.
From an engineer’s perspective, our data scientist Ruan’s experience sums it up:
“The Career Skills Matrix helps me assess my position in the career landscape in a non-ambiguous way that everyone agrees on. It gives me an overview of my progress to date to keep me motivated. It also shows me my strengths and weaknesses, so I can leverage/manage them appropriately. This means I can easily prioritise my learning/training sessions to target the most relevant skills I still lack. It also means knowing the skills of senior and junior colleagues, so I know who to approach when asking for help or delegating work.”
We spent a long time researching, developing and refining the MLOps skills matrix. We based the matrix on some best practices in software engineering and the DevOps industry but also added our opinions, tools and technologies.
We are constantly improving and updating this skills matrix. If you spot a gap or have any recommendations, hit us up!
Resources
- A great starting point for anything MLOps related
- FourthBrain machine learning engineer course
- Roundtable discussion on MLOps by ML celebrities
- Machine Learning Engineer curriculum
- Fullstack deep learning course
Merelda is the co-founder and CEO of melio.ai, an MLOps startup helping companies commercialise their AI projects.
She is an MLOps evangelist who is deeply frustrated by the slow time-to-value for machine learning projects. Her mission is to make AI frictionless by building a bridge between business, data science and engineering. Merelda enjoys leading multidisciplinary teams to build PoCs, deploying them to production and continuously improving them to create transformative values for the business.
She holds a BSc and MSc degree in Electrical Engineering from the University of Witwatersrand. Merelda also co-organizes the Cloud Native Computing Foundation’s Johannesburg chapter and is an avid meet-up speaker.
