CodingSaves ~2 hours
Write a data pipeline with validation
Build a robust, observable data pipeline with per-record error handling and dead-letter logging.
The prompt
You are a data engineer building a batch processing pipeline. Write a [LANGUAGE] data pipeline that processes the following input data through multiple transformation stages. The pipeline must: read from [SOURCE e.g. CSV file / S3 / database / API], validate each record against a schema at ingestion (reject invalid records to a dead-letter sink), apply transformations in configurable stages, handle errors per-record without stopping the pipeline, output successful records to [DESTINATION], output failed records with error reason to [ERROR SINK], emit metrics (records processed, failed, throughput), support checkpoint/resume for large datasets, and be testable by injecting mock sources and sinks. Input format: [FORMAT] Transformations: [LIST TRANSFORMATION STEPS] Expected volume: [RECORDS PER RUN] Language: [LANGUAGE]
Replace the [BRACKETED] fields with your details, then paste into ChatGPT, Claude or Gemini.
Want AI to fill this in for you?
Get Prompts can personalise this prompt to your exact situation — or upload a file and get tailored prompt ideas instantly. 3 free edits, no sign-up.
Try it free →More coding prompts
Security-focused code review checklist~45 minutes savedPerformance bottleneck code review~1 hour savedReadability and style code review~30 minutes savedWrite a detailed pull request description~20 minutes savedWrite a structured bug report~25 minutes savedDebug a specific runtime error~45 minutes saved