AI & Data · Planned program · Preview v0.1.0

Data Engineering

Build reliable data systems. Make analytics and AI possible.

Explore a planned Data Engineering curriculum covering reliable ingestion, analytical modelling, orchestration, batch and streaming pipelines, and data-platform operations.

Program availability

Curriculum preview only. Enrollment is not open. Fees, start date, delivery mode and instructor availability are not yet confirmed.

Proposed scope: 20 weeks / 150 live hours. This is a curriculum estimate, not an announced cohort schedule.

Ask about availability Explore Data Science

Choose by the work you want to do

Data Engineering or Data Science?

Data Engineering

Design and operate dependable data flows. Focus on source ingestion, analytical storage, transformations, orchestration, quality, security, costs and recovery.

Portfolio evidence: a tested source-to-mart platform, a bounded stream-processing example and a recovery demonstration.

Data Science

Use data to investigate questions, evaluate uncertainty and build useful predictive models. Include engineering foundations so datasets and features can be reproduced and trusted.

Portfolio evidence: statistical analysis, validated models, explainability and an end-to-end prediction service.

Readiness and realistic scope

Working Python, SQL joins and aggregations, Git and command-line basics. This is not a zero-coding course; learners needing a bridge should complete foundations first.

The proposed course is an early-career foundation, not a substitute for production experience or a promise of certification, employment or expertise across every cloud. Core labs use local containers and synthetic or openly licensed datasets. A single-cloud deployment is an optional extension with an agreed budget; paid cloud accounts are not required for the core preview design.

Ten modules · Proposed curriculum

From a source contract to a recoverable data platform.

Each module pairs concepts with a lab, a portfolio artifact and an observable assessment. Tool versions and the final delivery plan will be validated before enrollment opens.

Module 01 · 2 weeks · 15 live hours

Data foundations and contracts

Python packaging, environments, logs and tests
SQL joins, windows, keys and data grain
CSV, JSON, Parquet and schema validation
Source ownership, provenance and data contracts

Build: Validate a synthetic source feed and quarantine invalid records.

Evidence: Tested Python package, SQL checks and source contract.

Verify: Reject malformed input and explain join fan-out.

DATA-ENGINEERING-M01@0.1.0

Module 02 · 2 weeks · 15 live hours

Analytical modelling and storage

OLTP versus OLAP and workload trade-offs
Facts, dimensions, star schemas and slowly changing dimensions
Warehouse, lake and lakehouse boundaries
Partitioning, columnar storage and query plans

Build: Build an analytical mart with a history-preserving dimension.

Evidence: Data model, transformation SQL and reconciliation report.

Verify: Reconcile source totals and defend the model grain.

DATA-ENGINEERING-M02@0.1.0

Module 03 · 2 weeks · 15 live hours

Batch ingestion and incremental loads

API pagination, rate limits, retries and backoff
Full versus incremental loads and change-data-capture concepts
Watermark state, deduplication and upsert keys
Deletes, schema changes, backfills and replay

Build: Load paginated API fixtures; restart a failed incremental batch safely.

Evidence: Ingestion job, state table and replay tests.

Verify: Prove a rerun does not duplicate data and handle a late update.

DATA-ENGINEERING-M03@0.1.0

Module 04 · 2 weeks · 15 live hours

SQL transformation and analytics engineering

dbt sources, staging, intermediate and mart layers
Incremental models, unique keys and full-refresh trade-offs
Uniqueness, relationship, null and freshness tests
Documentation, lineage and version-controlled review

Build: Build tested dbt models from raw records to a business mart.

Evidence: Documented model graph, tests and incremental comparison.

Verify: Compare incremental output with a full refresh, including duplicate inputs.

DATA-ENGINEERING-M04@0.1.0

Module 05 · 2 weeks · 15 live hours

Orchestration and dependable workflows

Airflow DAGs, dependencies and data intervals
Idempotent tasks, retries, timeouts and backfills
Connections and secret handling; small metadata references
Workflow tests, logs and failure alerts

Build: Orchestrate ingestion, transformation and quality checks; recover a failed partition.

Evidence: DAG, dependency diagram and recovery runbook.

Verify: Demonstrate a retry and targeted backfill without downstream duplicates.

DATA-ENGINEERING-M05@0.1.0

Module 06 · 2 weeks · 15 live hours

Distributed batch and lakehouse foundations

PySpark DataFrames, lazy execution and query plans
Joins, shuffles, skew and partition sizing
Parquet layout and the small-files problem
Iceberg snapshots, schema evolution and partition-evolution concepts

Build: Transform partitioned data with PySpark and compare execution plans.

Evidence: Batch job, benchmark notes and table-evolution design.

Verify: Explain when a local SQL engine is sufficient and justify distributed work.

DATA-ENGINEERING-M06@0.1.0

Module 07 · 2 weeks · 15 live hours

Event streaming foundations

Kafka topics, partitions, offsets and consumer groups
Spark Structured Streaming micro-batches
Event time, windows, watermarks and late-arriving data
Checkpoints, replay and source/sink delivery guarantees

Build: Process a synthetic event stream with duplicate and late events; restart from a checkpoint.

Evidence: Streaming job, lateness policy and delivery-semantics test report.

Verify: Show restart behaviour and explain why end-to-end guarantees depend on the source and sink.

DATA-ENGINEERING-M07@0.1.0

Module 08 · 1 week · 7.5 live hours

Cloud, security and governance foundations

Map one pipeline to one cloud's storage, compute and orchestration services
Least-privilege access, encryption and secret management
Classification, minimisation, retention and deletion requirements
Infrastructure-as-code concepts, budgets and resource cleanup

Build: Design a least-privilege cloud deployment using synthetic data and an explicit cost ceiling.

Evidence: Architecture diagram, access matrix and teardown checklist.

Verify: Review access boundaries and justify retention and cost controls.

DATA-ENGINEERING-M08@0.1.0

Module 09 · 2 weeks · 15 live hours

Operations, performance and recovery

Freshness, completeness and latency service-level objectives
Metrics, alert thresholds, ownership and incident triage
CI checks, environment promotion and rollback
Reconciliation, recovery objectives and cost/performance measurement

Build: Inject a pipeline failure, identify the cause and rehearse recovery.

Evidence: Dashboard specification, CI checks, incident report and recovery evidence.

Verify: Meet a stated recovery target and reconcile results after replay.

DATA-ENGINEERING-M09@0.1.0

Module 10 · 3 weeks · 22.5 live hours

End-to-end data-platform capstone

Source-to-mart batch pipeline with documented contracts
Bounded event-stream extension and late-data policy
Quality gates, lineage, access controls and operational ownership
Architecture trade-offs, measured costs and technical defence

Build: Deliver a reproducible synthetic-data platform and demonstrate normal, duplicate, late and failed runs.

Evidence: Versioned repository, data mart, stream demo, tests, lineage, runbook and demo recording.

Verify: Pass correctness, reproducibility and recovery gates; defend design and limitations.

DATA-ENGINEERING-M10@0.1.0

Demonstrate correctness, not just a successful run.

Proposed assessment: labs 30%, skill checks 15%, design review 15% and capstone 40%. Proposed completion requires 70% overall plus passing capstone correctness, reproducibility and recovery gates.

The capstone must handle normal, duplicate, late and failed inputs. Learners explain limitations and source/sink delivery semantics instead of claiming that every pipeline is automatically exactly-once.

Curriculum review

Reviewed on 9 September 2026 against current data-engineering competency guides and project documentation. This is an independent curriculum mapping, not vendor accreditation.

Fees and enrollment will be published only after delivery readiness and the admissions course setup are confirmed.

Discuss prerequisites and availability