Skip to content

What the final artifact is for

The Data Quality & Cleaning Notebook + Data Quality Report is DS 102's final artifact. You choose a real wrangling problem you actually face, and write the problem, audience, data, method, verification, output, limitations, governance considerations, human reviewer and next step. It becomes the starting point for later practice discussions.

Info

Human, AI, Human: a person defines the problem and validation rules, AI may help write the wrangling code, and a person verifies the data, the rules and the conclusion, and remains responsible for the result.

What to fill in

  • The problem, audience and data (source, grain, time coverage, key definitions)
  • The method and workflow
  • At least three verification checks
  • Evidence and output
  • Limitations
  • Risk and governance considerations, as relevant
  • The human reviewer
  • The next step

A structurally complete Data Quality & Cleaning Notebook

A structurally complete artifact includes: a clear, checkable question; a stated data source and grain; a reproducible method; at least three verification checks; output that matches the claims; at least one limitation.

Info

The platform never automatically judges whether your cleaning decisions were correct. This artifact IS the Data Preparation & Quality Practical Assessment's artifact -- see the two sections below for the five required tasks and how a human reviewer scores it.

The five Practical Assessment tasks

The Data Preparation & Quality Practical Assessment has five tasks, in order: Inspect, Clean, Reshape, Join, Validate. Write your answers to all five directly into the Data Quality & Cleaning Notebook + Data Quality Report form below.

  1. Task 1: Inspect
    State the schema, row counts, grain (what one row represents), a candidate key, a missingness summary, and any unit or definition risk you can see before changing anything.
  2. Task 2: Clean
    Handle missing values, duplicates, invalid categories, wrong data types, and whitespace/parsing problems. For every change, answer: what changed, why, what evidence supports it, and could it destroy information.
  3. Task 3: Reshape
    State the before grain, the reshape operation you applied, the after grain, and a uniqueness check proving the after-grain key is genuinely unique.
  4. Task 4: Join
    State the expected cardinality, the pre-join and post-join row counts, unmatched keys, and duplicated keys, then diagnose any row multiplication using the worked example below.
  5. Task 5: Validate
    Produce a Data Quality Report covering every field listed below, ending in an explicit pass/fail validation result.
Task 4 worked example: orders (the fact table you are joining from)
order_idcustomer_idorder_dateamount_ntd
O01C12026-01-051200
O02C12026-01-06980
O03C12026-01-091500
O04C22026-01-05760
O05C22026-01-07890
O06C22026-01-111040
O07C32026-01-062200
O08C32026-01-081750
O09C32026-01-10990
O10C42026-01-05640
O11C42026-01-09720
O12C42026-01-12880
O13C52026-01-061330
O14C52026-01-081420
O15C52026-01-131010
Task 4 worked example: customer master, exactly as extracted from three source systems
customer_idcustomer_nameregionsource_system
C1Northgate Univ. Co-opNorthCRM-A
C1Northgate Univ. Co-opNorthCRM-B
C1Northgate Univ. Co-opNorthCRM-Legacy
C2Riverside Print ShopNorthCRM-A
C2Riverside Print ShopNorthCRM-B
C2Riverside Print ShopNorthCRM-Legacy
C3Harbor BooksSouthCRM-A
C3Harbor BooksSouthCRM-B
C3Harbor BooksSouthCRM-Legacy
C4Summit StationerySouthCRM-A
C4Summit StationerySouthCRM-B
C4Summit StationerySouthCRM-Legacy
C5Lakeside SuppliesEastCRM-A
C5Lakeside SuppliesEastCRM-B
C5Lakeside SuppliesEastCRM-Legacy

Warning

Join orders to this customer master on customer_id and you get exactly the required trap: 15 pre-join rows become 45 post-join rows, with no program error. Why: orders.order_id is unique (15 distinct orders), so you would naively expect a 1:1 join (15 rows out). But customer_id is NOT unique in the customer master -- the same 5 customers were imported from three source systems (CRM-A, CRM-B, CRM-Legacy) and never de-duplicated, so every customer_id appears 3 times. That turns the join into a 1:3 fan-out: each of the 15 orders matches 3 customer rows, giving 15 x 3 = 45. Diagnosing this means checking customer_id uniqueness in the customer master BEFORE joining (for example customers.groupby('customer_id').size(), which would show 3 for every id) -- not just noticing the row count grew.
  • Row count
  • Unique-key checks
  • Duplicate count
  • Missingness
  • Unmatched keys
  • Allowed values
  • Ranges
  • Units
  • Date coverage
  • Final grain
  • Known limitations
  • Validation result (pass or fail, stated explicitly)

How it is scored

A human reviewer scores your submitted artifact against seven weighted domains. The overall result is PASS at 80 or above, REVISION_REQUIRED at 65-79, and FAIL below 65 or whenever any critical-fail rule is triggered -- regardless of the weighted score. A PASS additionally requires the Join Cardinality domain and the Validation & Data Contract domain to each reach at least 2 out of 3.

Data Preparation & Quality Practical Assessment: rubric domains
DomainWeightWhat a pass requires
Inspection & Grain15The grain is stated explicitly, with schema, row counts, a candidate key, and a missingness summary
Cleaning Decisions15Every change states what, why, evidence, and whether it could destroy information
Reshape Correctness10The before and after grain are both valid, with a uniqueness check
Join Cardinality20Row multiplication is detected and correctly diagnosed (mandatory, at least 2 out of 3)
Validation & Data Contract20Checks are reproducible, not just described (mandatory, at least 2 out of 3)
Lineage & Reproducibility10A source-to-output trace lets someone else reproduce or audit the cleaning process
Human Review & Limitations10Limitations and a human reviewer are both stated
  • Many-to-many or unintended row multiplication is not detected (including the worked 15-to-45 example above)
  • A unit mismatch materially affects the output and is not detected
  • Destructive cleaning is performed without evidence or a retained trace
  • The candidate-key uniqueness assumption is false and was never checked
  • The Data Quality Report has no validation section at all

Info

Any one of the five critical-fail rules above forces an overall FAIL, no matter how high the weighted score is. This table is published so you know exactly what is being checked -- it is not the same thing as a quiz answer key.

What you need to complete

  • Complete both DS 102 modules and pass the post-test at 80% or above
  • Submit the Data Quality & Cleaning Notebook + Data Quality Report (the Practical Assessment artifact) covering all five tasks
  • Have that artifact scored against the Data Preparation & Quality Practical Assessment rubric with an overall PASS
  • Trigger none of the five critical-fail rules above

What you will be assessed on

  • The seven weighted rubric domains above: Inspection & Grain, Cleaning Decisions, Reshape Correctness, Join Cardinality, Validation & Data Contract, Lineage & Reproducibility, and Human Review & Limitations
  • Join Cardinality must independently reach at least 2 out of 3 -- correctly diagnosing row multiplication is mandatory, not optional
  • Validation & Data Contract must also independently reach at least 2 out of 3

Evidence produced

  • Your submitted Data Quality & Cleaning Notebook + Data Quality Report
  • A recorded rubric result (PASS, REVISION_REQUIRED, or FAIL) with no critical-fail rule triggered
  • DS 101 completion is recommended background reading, but is never required to earn this certificate

What you can earn

The Data Preparation & Quality Capability Certificate, an L2 capability certificate with its own public verification page. It supplements your DS 102 Course Completion evidence -- it never replaces it, and it is never issued from the post-test alone.

What this credential does not mean

This credential confirms that the holder completed DS 102's required lessons and post-test, and demonstrated schema/grain, cleaning, reshape, join-cardinality, validation, and lineage capabilities against the Data Preparation & Quality Practical Assessment rubric.

  • Does not confer a professional data-engineering license or employment qualification
  • Does not certify that any specific dataset the holder cleans afterward is correct
  • Does not remove the need for human review before high-impact decisions (Human, AI, Human)

Exit Ticket

Finally, use the Exit Ticket to record the most important idea from today, the step you are still unsure about, where you would apply this in a real context, the risk you most need to verify, and your difficulty and usefulness ratings.

Before you submit

Warning

Do not enter company secrets, client data, personal data, or other unauthorized sensitive information. Unless authorized, never upload confidential institutional or company data to an AI tool; confirm your institution's rules before using real internal operations data.

Open the forms

What next?