A Python CSV cleaning mini-project you can verify
A verifiable Python CSV cleaning project reads a small invented file, applies explicit rules and produces an output you can reconcile with the input. Start with the standard CSV module and a narrow task. The learning goal is to explain each transformation and test a changed input, rather than assembling a large production data pipeline.
A verifiable Python CSV cleaning project reads a small invented file, applies explicit rules and produces an output you can reconcile with the input. Start with the standard CSV module and a narrow task. The learning goal is to explain each transformation and test a changed input, rather than assembling a large production data pipeline.
Key ideas
- Required columns and validation rules are written.
- The separator and quoted values are handled as CSV.
- Identifiers retain their text formatting.
- Rejected rows have an explicit reason.
Write a small data contract
Use columns such as product_id, name and quantity. State which fields are required, whether quantity must be an integer and whether a zero is allowed. Keep product IDs as text so formatting is preserved. Add a separate rule for missing values. These are choices for the fictional exercise, not universal product-data rules. Write them before coding so the script has a stable definition of a valid row.
Read the file according to its format
Use the CSV reference to choose the reader and separator that match your file. A comma appearing inside a quoted name is a useful test case; splitting every line on a comma does not understand CSV quoting. Check that the headers are the expected ones before transforming rows. Save a small input fixture with invented names, including a code that starts with zero.
Separate accepted and rejected rows
For each row, either produce the cleaned result or record a reason for review. Keep the row number and original values in the issue log. Do not silently replace an invalid quantity with zero: that can change the meaning. A duplicate needs a stated identifier rule. The script should make the decision visible so another learner can inspect how the output was created.
Test the boundary and reconcile counts
Try an ordinary row, a quoted comma, an empty required value and a quantity outside your rule. Rerun the same input and compare results. Count accepted and rejected rows and verify that together they explain the input after the header. MDN’s testing strategies support choosing meaningful cases; our fixture is an original learning example. Passing these cases proves only the limited behaviour tested here.
Your evidence checklist
Mark only what you have checked. This records your own progress, not an independent audit or a predicted result. There is no automatic saving; download the note if you want to keep it.
0 / 6 checked
In everyday language
Keep the data contract, fixture, issue log and tested output together so another person can verify the transformation.
Try it yourself
Create six synthetic product rows, including a quoted comma, a leading-zero ID and an invalid quantity. Read them with Python’s CSV module, keep valid rows and write a review log for the rest.
Expected result
An input fixture, output fixture and test record showing why every row was accepted or rejected. No real catalogue import or customer data is needed.
Check your answer: Why not split each line on commas?
CSV permits quoted fields that contain commas and other formatting details. A simple split treats those characters as separators even when they belong inside a value. Use a CSV-aware reader and test it against the format of your actual exercise file.
Questions
Why not split each line on commas?
CSV permits quoted fields that contain commas and other formatting details. A simple split treats those characters as separators even when they belong inside a value. Use a CSV-aware reader and test it against the format of your actual exercise file.
Does this exercise make a production import safe?
It verifies a small set of rules on synthetic input. Production imports require additional review of permissions, file sizes, encodings, destinations and business rules. Keep the learning result clearly scoped and avoid presenting a tiny fixture as proof about an unseen operational pipeline.
Keep the data contract, fixture, issue log and tested output together so another person can verify the transformation.
Sources and further reading
- Python — CSV reading and writing ↗Sources checked:
- MDN — Testing strategies ↗Sources checked: