Senior Data Engineer
Harambee Youth Employment Accelerator|Posted 4 days ago
Sign up or log in to apply:
Skills and experience
Location and salary
Role description
About the role
The Harambee Software Development team is looking for an exceptional Senior
Data Engineer to join their dynamic team based in Johannesburg. The Senior
Data Engineering role's objective is collecting, storing and processing large sets
of data, with the primary goal of providing optimal solutions to use and of
course the implementation, improvement, and monitoring thereof. The Senior
Data Engineer is also responsible to partner with the Head of Data on database
design and strategy to enable Harambee’s future strategy and technology
roadmap. Furthermore, the Senior Data Engineer will be doing all that in an
organisation that changes lives and has a culture of teamwork, openness and
collaboration. If this prospect excites you, then you may be the person we are
looking for.
THE TECHNICAL CHALLENGE
Harambee is building a ground-breaking, scalable, future-fit technology platform
that changes lives by helping young first-time work-seekers find jobs and work
experiences. Our technology needs to cater for the unique, personalised
employment journey of each workseeker – ultimately using the power of big
data to help them find their first job. We run an Agile Development Programme
in a DevOps infrastructure and culture. Our cloud-native technology stack is
predominantly hosted in Google Cloud Platform with services written in .Net
Core and Python. The data landscape includes MYSQL, Big Query and Composer.
This technology stack is currently being revised to drive a strategy of
simplification, business user enablement and data integrity. Google did a case
study on us. READ IT (https://cloud.google.com/customers/harambee/)
ROLE RESPONSIBILITIES
You will work as a technical specialist, aiding your colleagues in the creation,
maintenance and use of the data stack.
- Planning and strategy:
- Engage with various stakeholders to understand the business and data goals in context of Harambee’s strategy and vision
- Design data solutions for real-time and batch environments
- Document the planning, implementation and operation of the data platform
- Select and integrate any big data tools and frameworks required to enable the business and data analysis goals
- Work closely with the data analyst, testers, developers and business analysts to design and ensure delivery of data requirements
- Execution:
- Assist with and advise on the data platform and it’s use, including data analysis, security, architecture and modelling
- Ensure effective infrastructure for processing large volumes of transactional data
- Implement data retention policies
- Monitor performance and advise on necessary infrastructure changes
- Work with platform developers to ensure seamless and optimal data ingestion
- Build data structures and pipelines that ensure the storage and transformation of data
- Work with the data analysts to ensure necessary and accurate data is available for downstream use
- Support
- Support other teams with ad hoc data analysis to derive business insights
- Support the Data analysis and Business Intelligence team on data validation and quality assurance to ensure accurate dashboards and reports
ROLE REQUIREMENTS
Experience
- Matric / Grade 12
- A degree or diploma in Science or Engineering (Mathematics, Statistics, Computer Science or related field)
- Minimum 8 - 12 years’ experience as a data engineer
- Data Architect
- Apache Airflow
- Knowledge of and use with data tools and languages
• Exceptional data skills
o SQL proficiency
o Document stores
o Realtime and batch processing
o Models and relationships
• Large scale ETL (Apache beam or Apache spark, SSIS)
• Cloud experience (Google Cloud Platform, Azure, or AWS)
• Python
• MySQL
• R (nice to have)
• Has worked at a company of similar size and complexity to Harambee
over the past 24 months
• Importantly, you must fit with the Harambee culture, be passionate
about our mission and be an individual who leaves things better than
you found them.
• Keen to work for purpose and be inspired by the bigger social problem
we want to solve Keen to learn and grow
COMPETENCIES
• Curiosity and drive to understand, improve and deliver
• Proven track record of quality delivery
• Comfortable explaining data models, data structures and analytical
findings to technical and non-technical audiences
• Willingness to learn
• A passion for knowledge sharing and upskilling other team members
• Highly detail oriented
Sign up or log in to apply:
About Harambee Youth Employment Accelerator
South Africa has one of the highest youth unemployment rates in the world; there are over 6 million young people that are not in education, employment or training. Yet these young people are the future of our country and can be the cornerstone of a vibrant, growing economy.
What if you could wake up every morning knowing you’re working on solving this huge problem for this important group of people? What if you could be part of the team behind designing and building the technology that will change the face of youth unemployment worldwide? What if you could work with globally leading technology experts, academic researchers and business leaders who all deeply care about this problem?
At Harambee Youth Employment Accelerator, that’s exactly what we do. Every day we get to work on solutions that help young people who would otherwise be locked out of the formal economy.
We’re building a technology platform to pathway young people into income-generating opportunities. This means our technology needs to be scalable, innovative and cater for unique, personalised development journeys for each young person – we need this platform to ultimately help young people to find their way to their very first job. We’ve pulled a team of exceptional experts together to guide us through this journey, and we’re looking to now build a team internally! Our cloud-native technology stack is hosted both in Google Cloud Platform and in AWS, deployed using a microservices architecture, with services written in .Net Core and Python. The data landscape includes MYSQL, Big Query, SOLR; the analytics and ML environment utilises Tensorflow. Harambee has also developed an LMS and Assessment platform using Open edX. This technology stack is designed for scale because we know that we have to achieve big hairy audacious goals with it.
Harambee is a fast-paced environment, incredibly passionate about purpose, and intellectually demanding – be ready to problem solve daily and put the young person at the core of what you do. Be ready to not only work on the technology but help shape how we work as an organisation to address this macroeconomic issue. And be ready to build a technology platform that can address this problem at scale, not only in SA but on the continent and beyond. Be a part of the solution!
Perks at Harambee Youth Employment Accelerator
Tech Stack
utilities
dev ops
business tool
application and data
