Dotdigital Logo

Principal Data Engineer

Dotdigital|Posted 4 days ago

Skills and experience

Role:Data engineer
Other roles:Data architect, Machine learning engineer
Experience in role:4+ years
Language proficiency:English
Must-have skills:
    Python
    SQL
    AWS
    Azure

Location and salary

Remote policy:Remote
Location of job:South Africa
Visa requirements:Authorised to work in South Africa With status of citizen/passport holder or permanent resident
Visa sponsorship:Unable to sponsor visa
Employment type:Permanent

Role description

About Us

The Company: Dotdigital is a thriving global community of passionate, dedicated professionals, committed to the collective success of the organization and its clients. Our core principles of innovation, teamwork, and client-focused solutions drive us to approach challenges with a growth mindset and take ownership of our work. At Dotdigital, collaboration and curiosity pave the way for meaningful connections and learning opportunities with diverse peers. Our work environment encourages knowledge sharing, fosters exploration, and cherishes creative ideas. Combined, these guide us towards a shared vision in which brands around the world exceed customer expectations through the adoption of responsible marketing practices.

The Product: Dotdigital is an all-in-one customer experience and data platform (CXDP) that empowers marketing teams to exceed customer expectations with highly personalized cross-channel journeys. With Dotdigital, marketers can seamlessly unify, enrich, and segment customer data. Breaking down data silos, Dotdigital streamlines decision-making and paves the way for marketing creativity that delivers customer engagement at scale. With powerful AI capabilities, Dotdigital makes it easy to automate deeply personalized experiences across web, email, SMS, WhatsApp, chat, push, social, ads, and more.

About the Role

We are on the lookout for a Principal Data Engineer to help define and lead the next generation of our data platform and data capabilities. You’ll play a key role in building scalable, resilient and intelligent data systems that power real time services, insights, products and decisions across Dotdigital.

As a Principal Data Engineer, you will be instrumental in driving the architecture, development and delivery of our data platform. You will lead key initiatives, provide technical direction and collaborate with product, analytics and data science teams to ensure data value is realised across the entire ecosystem. Working across the entire data lifecycle, you will help shape how data is collected, processed and consumed across Dotdigital.

Responsibilities:

Lead the design and implementation of scalable, secure and resilient data systems across streaming, batch and real-time use cases.

Architect data pipelines, model and storage solutions that power analytical and product use cases; using primarily Python and SQL via orchestration tooling that run workloads in the cloud.

Leverage AI to automate both data processing and engineering processes.

Assure and drive best practices relating to data infrastructure, governance, security and observability.

Work with technologists across multiple teams to deliver coherent features and data outcomes.

Support the data team to help adopt data engineering principles.

Identify, validate and promote new tools and technologies that improve the performance and stability of data services.

About You

Technical Expertise

Extensive experience delivering python-based projects in the data engineering space.

Extensive experience working with SQL and NoSQL database technologies (e.g. SQL Server, MongoDB & Cassandra).

Proven experience with modern data warehousing and large-scale data processing tools (e.g. Snowflake, DBT, BiqQuery, Clickhouse).

Hands on experience with data orchestration tools like Airflow, Dagster or Prefect.

Experience using cloud environments (e.g. Azure, AWS, GCP) to process, store and surface large scale data.

Experience using Kafka or similar event-based architectures e.g. (Pub/Sub via AWS SQS, Azure EventHubs, AWS Kinesis).

Strong grasp of data architecture and data modelling principles for both OLAP and OLTP workloads.

Capable in the wider software development lifecycle in terms of agile ways of working and continuous integration/deployment of data solutions.

Engineering Leadership

Experience as a lead or Principal Engineer on large-scale data initiative or product builds.

Demonstrated ability to architect data systems and data structures for high volume, high throughput systems.

Proven experience leading data platform modernisation or cloud migration projects.

Comfortable taking ownership of difficult data problems and driving them to resolution.

Bonus

Experience using ClickHouse as part of a data pipeline and analytics solution.

Experience using Databricks or similar data platforms.

About Dotdigital

201-500 employees

About Us

The Company: Dotdigital is a thriving global community of passionate, dedicated professionals, committed to the collective success of the organization and its clients. Our core principles of innovation, teamwork, and client-focused solutions drive us to approach challenges with a growth mindset and take ownership of our work. At Dotdigital, collaboration and curiosity pave the way for meaningful connections and learning opportunities with diverse peers. Our work environment encourages knowledge sharing, fosters exploration, and cherishes creative ideas. Combined, these guide us towards a shared vision in which brands around the world exceed customer expectations through the adoption of responsible marketing practices.

The Product: Dotdigital is an all-in-one customer experience and data platform (CXDP) that empowers marketing teams to exceed customer expectations with highly personalized cross-channel journeys. With Dotdigital, marketers can seamlessly unify, enrich, and segment customer data. Breaking down data silos, Dotdigital streamlines decision-making and paves the way for marketing creativity that delivers customer engagement at scale. With powerful AI capabilities, Dotdigital makes it easy to automate deeply personalized experiences across web, email, SMS, WhatsApp, chat, push, social, ads, and more.

Our Tech Stack

Backend

  • Majority C# (dotnet 8 with some Framework)
  • Some NodeJS, some Python for data science, and PHP for ecommerce integrations turned-on flat screen monitor

Frontend

  • Angular 15+ (some older AngularJS)
  • ASP.NET (webapi, some older mvc+webforms which we’re phasing out)
  • TS/ES6/gulp/grunt/node/yarn
  • Sass

Data

  • SQL Server
  • Google BigQuery
  • Azure CosmosDB
  • Redis
  • MongoDB
  • Clickhouse
  • Databricks

Infrastructure

  • Cloud: Majority Azure; some GCP; some AWS
  • Containerized (mostly Linux containers, some Windows containers)
  • Terraform/ARM/IaC
  • Azure Service Bus/Event Hubs
  • RabbitMQ

Tooling

  • Azure DevOps
  • Visual Studio/Rider/VS Code/PyCharm/PhpStorm
  • nUnit/Moq/Robot Framework
  • Elastic Search/Kibana/Prometheus/Grafana
  • Github
  • Copilot

Perks at Dotdigital

Parental leave
Medical benefits
Dotdigital day
Wellbeing reward
Wellbeing days
Loyalty reward
Paid sick leave

Tech Stack

application and data

AngularJS
AngularJS
Sass
Sass
Javascript
Javascript
Node.js
Node.js
Python
Python
PHP
PHP
Redis
Redis
RabbitMQ
RabbitMQ
MongoDB
MongoDB
C#
C#
Microsoft Azure
Microsoft Azure
ASP.NET
ASP.NET
MS SQL
MS SQL
.Net Core 2.0
.Net Core 2.0
TypeScript
TypeScript
Kibana
Kibana
Angular
Angular
BigQuery
BigQuery

dev ops

GitHub
GitHub
Pingdom
Pingdom
Git
Git
Docker
Docker
Kubernetes
Kubernetes
Terraform
Terraform
Prometheus
Prometheus
Cloudflare
Cloudflare
Azure DevOps
Azure DevOps
Visual Studio
Visual Studio
VS Code
VS Code
Rider
Rider

business tool

Slack
Slack
Zendesk
Zendesk

Similar jobs on OfferZen: