AWS Data Engineer

HearX Group|Posted 26 days ago

Skills and experience

Role:Data engineer

Other roles:Machine learning engineer

Experience in role:3+ years

Language proficiency:English

Must-have skills:

AWS

Python

SQL

Nice-to-have skills:

PostgreSQL

NoSQL

Location and salary

Remote policy:Hybrid

Location of job:Pretoria, South Africa or Johannesburg, South Africa

Visa requirements:Authorised to work in South Africa With status of citizen/passport holder or permanent resident

Visa sponsorship:Unable to sponsor visa

Employment type:Permanent

Role description

Job Description

Job Purpose:

Responsible for creating and managing the technological part of data infrastructure in every step of data flow. From configuring data sources to integrating analytical tools — all these systems would be architected, built, and managed by a general-role data engineer.

Minimum education (essential):

Bachelor’s degree in Computer Science or Engineering (or similar)

Minimum education (desirable):

Honors degree in Computer Science or Engineering (or similar)

AWS Certified Data Engineer or

AWS Certified Solutions Architect or

AWS Certified Data Analyst

Minimum applicable experience (years):

5+ years working experience

Required nature of experience:

Data Engineering development

Experience with AWS services used for data warehousing, computing and transformations ie.AWS Glue (crawlers, jobs, triggers, and catalog), AWS S3, AWS Lambda, AWS Step Functions, AWS Athena and AWS CloudWatch

Experience with SQL and NoSQL databases (e.g., PostgreSQL, MySQL, DynamoDB)

Experience with SQL for querying and transformation of data

Skills and Knowledge (essential):

Strong skills in Python (especially PySpark for AWS Glue)

Strong knowledge of data modeling, schema design and database optimization

Proficiency with AWS and infrastructure as code

Skills and Knowledge (desirable):

Knowledge of SQL, Python, AWS serverless microservices,

Deploying and managing ML models in production

Version control (Git), unit testing and agile methodologies

Data Architecture and Management 20%

Design and maintain scalable data architectures using AWS services for example, but not limited to, AWS S3, AWS Glue and AWS Athena.

Implement data partitioning and cataloging strategies to enhance data organization and accessibility.

Work with schema evolution and versioning to ensure data consistency.

Develop and manage metadata repositories and data dictionaries.

Assist and support with defining, setup and maintenance of data access roles and privileges.

Pipeline Development and ETL 30%

Design, develop and optimize scalable ETL pipelines using batch and real-time processing frameworks (using AWS Glue and PySpark).

Implement data extraction, transformation and loading processes from various structured and unstructured sources.

Optimize ETL jobs for performance, cost efficiency and scalability.

Develop and integrate APIs to ingest and export data between various source and target systems, ensuring seamless ETL workflows.

Enable scalable deployment of ML models by integrating data pipelines with ML workflows.

Automation, Monitoring and Optimization 30%

Automate data workflows and ensure they are fault tolerant and optimized.

Implement logging, monitoring and alerting for data pipelines.

Optimize ETL job performance by tuning configurations and analyzing resource usage.

Optimize data storage solutions for performance, cost and scalability.

Ensure the optimisation of AWS resources for scalability for data ingestion and outputs.

Deploy machine learning models into productions using cloud based services like AWS Sagemaker.

Security, Compliance and Best Practices 10%

Ensure API security, authentication and access control best practicespractises.

Implement data encryption, access control and compliance with GDPR, HIPAA, SOC2 etc.

Establish data governance policies, including access control and security best practicespractises.

Development Team Mentorship and Collaboration 5%

Work closely with data scientists, analysts and business teams to understand data needs.

Collaborate with backend teams to integrate data pipelines into CI/CD.

Assist with developmental leadership to the team through coaching, code reviews and mentorship.

Ensure technological alignment with B2C division strategy supporting overarching hearX strategy and vision.

Identify and encourage areas for growth and improvement within the team.

QMS and Compliance 5%

Document data processes, transformations and architectural decisions.

Maintain high standards of software quality within the team by adhering to good processes, practices and habits, including compliance to QMS system, and data and system security requirements.

Ensure compliance to the established processes and standards for the development lifecycle, including but not limited to data archival.

Drive compliance to the hearX Quality Management System in line with the Quality Objectives, Quality Manual, and all processes related to the design, development and implementation of software related to medical devices.

Comply to ISO, CE, FDA (and other) standards and requirements as is applicable to assigned products.

Safeguard confidential information and data.

About HearX Group

51-200 employees

Your calling

Interested in the future of mobile healthcare?

Do you want to work with the global leader in the mHealth and eHealth mobile application space?

... Then keep reading! But don't just take our word for it, fast-forward to our Press section to see how 'groundbreaking' it is what we're all about.

So, who are we?

hearX Group offers the world’s first clinical smartphone hearing testing solutions with the aim to provide healthy hearing for everyone, everywhere. This has been done through developing smartphone technologies that enables a layperson to conduct hearing and vision tests remotely - only needing access to the screening device and mobile data. hearX Group has gone to market with several products since the launch of its flagship product hearScreen , an innovative mobile health application for clinically-valid, time-efficient, low-cost screening using pure tone audiometry. Following the tremendous success of the launch of hearScreen several other products were developed and launched soon after such as hearZA (Downloadable via the app stores), mHealth Studio, hearTest, hearDigits in addition to our partnered product in vision, PeekVision.

Why a mobile hearing solution on a smartphone?

It has been proven that early detection followed by appropriate health care ensures optimal quality of life outcomes.

Smartphone health surveillance and monitoring has gained significant traction with an estimated 1.7 billion downloads expected by the end of 2017 and global revenues reaching $21.5 billion in 2018 (Economist, 2016). By employing a smartphone platform, hearX Group provides an opportunity for early detection by end-users with the potential to inform and refer patients to their closest healthcare provider using SMS or e-mail add-on services.

Our mHealth solutions are cheaper and faster than traditional methods and most importantly are mobile, allowing us to reach previously inaccessible areas. We believe that this can help us change the face of preventative care across the continent.

Should you apply for this job?

If you're someone who is looking to use their skills to change the future of healthcare access and have a significant social impact by providing these enabling technologies to people who have never had access to them before - then sign up! Because the hearX Group wants you.

Our Interview Process

If you have great taste and want to join our (awesome) team, send us your info and we’ll go through a super quick initial screening call. This is a rather informal chat to suss out your personality and experience while we give you an opportunity to ask about what we do.

We will then ask you to complete a one-way interview with some technical questions. (Using Tech to recruit for Tech for the win) .

If both parties are happy to move forward, we’ll schedule an online interview with the relevant team.

Following a successful session, we’ll ask you to complete a practical test that you can complete in your own time. A team lead gives the green light if you’re as good as you say which will be followed by Psychometric Assessments and a last quick call for final questions.

Once a decision has been made, you’ll be informed, following which we’ll do all the boring legal things like an official employment offer and agreeing on an employment contract and getting it signed. Then, it’s welcome to the team and high fives all around.