DATA ENGINEER (Software Engineering)
via Mediro ICT · Johannesburg, Gauteng
- Pay
- On application
- Type
- Contract · remote / hybrid
- Experience
- 3–5 years
- Level
- Mid-level
- Sector
- Banking
- Closes
- in 14 days
- Recruiter
- Mediro ICT · All jobs at Mediro ICT
About the role
A data engineering role requiring strong software engineering skills to handle data pipelines, cloud services, and distributed processing.
2 Data Engineers = 1 Pure DE + 1 Software + Data Engineer
- Production Support - Incident management, troubleshooting, RCA
- Technical Documentation - Designs, mappings, runbooks, support documentation
- Technical Leadership - Mentoring, technical reviews, design facilitation
- Distributed Processing - Spark, PySpark, distributed processing
- Streaming & Event Processing - Kafka, event streaming, event-driven processing
- Data Warehousing - Warehouses, facts, dimensions, star/snowflake models
- Data Lake - Data lakes, zones, partitioning, file formats
- Lakehouse Engineering - Delta Lake, medallion patterns, lakehouse
- AWS Data Services - S3, Glue, Lambda, Athena, Redshift, EMR
- Serverless Data Engineering - Lambda, serverless processing
- Messaging & Events - SQS, SNS, EventBridge, Kafka
- Metadata & Lineage - Metadata, catalogues, lineage, traceability
- Infrastructure as Code - Terraform, cloud provisioning
- Performance Engineering - Partitioning, caching, query tuning, Spark optimisation
- Operational Resilience - Recovery, availability, resilience, production readiness
- Stakeholder Management - Communication, facilitation, requirements, technical decisions
What they're looking for
- SQL Engineering - Advanced SQL, CTEs, window functions, query plans, optimisation
- Python Engineering - Python, libraries, reusable modules, testing
- ETL / ELT Engineering - ETL, ELT, ingestion, transformation, mappings
- Data Modelling - Dimensional, relational, analytical, domain models
- Cloud Data Engineering - AWS, Azure, managed data services
- Data Orchestration - Airflow/MWAA, Step Functions, workflow orchestration
- Data Quality - Profiling, validation, quality rules, exception handling
- Data Reconciliation - Control totals, source-target comparison, balancing
- Data Privacy & Security - IAM, encryption, access control, privacy
- Data Observability - Logging, monitoring, alerting, metrics
- Continuous Integration / Continuous Delivery (or Continuous Deployment).(CI/CD) for Data - Git, pipelines, automated deployment, quality gates
- Automated Testing - Unit, integration, regression, reconciliation tests