# Batch processing vs Stream processing > Batch processing computes over a bounded dataset; stream processing continuously updates results as events arrive. Choose batch when scheduled results meet the freshness requirement, and streaming when delayed results would make the product less useful or unsafe. Many systems use streaming for provisional results and batch for backfills or reconciliation. Source: https://sysround.com/compare/batch-processing-vs-stream-processing Updated: 2026-11-05 ## Side by side | | Batch processing | Stream processing | |---|---|---| | Input model | Bounded dataset or snapshot | Continuously arriving events | | Result freshness | Schedule delay plus job runtime | Processing delay plus any windowing or finalization delay | | Computation | Scans, joins, and aggregates over finite input | Incremental updates, often with keyed state | | Late data | Include in a later run or recompute | Use lateness policies and result corrections | | Failure recovery | Retry tasks or rerun bounded input | Restore state and replay from saved positions | | Capacity | Provision to finish within a deadline | Provision to keep lag within a freshness target | | Cost drivers | Data scanned, shuffle, frequency, and runtime | Continuous capacity, state, checkpoints, and retention | | Typical uses | Historical reports, bulk transforms, backfills | Live metrics, alerts, continuously updated rankings | ## The differences that matter Freshness is the first design constraint. An hourly batch job does not guarantee one-hour freshness: an event can wait almost an hour before the job starts, then wait for execution and publication. A streaming pipeline reduces scheduling delay, but queue lag, window finalization, and slow sinks still affect when users see results. Batch has a natural input boundary, which makes full-dataset joins and recomputation easier to reason about. Streaming joins and aggregations often retain state across events. You must define how that state is partitioned, how long it lives, and what happens when a popular key overloads one worker. Neither model automatically solves duplicate or late events. A batch result is complete only relative to its chosen input snapshot or cutoff. Streaming needs an explicit event-time policy, often using watermarks and allowed lateness, to decide when to emit provisional results, finalize windows, or apply corrections. Recovery and capacity planning differ. Batch jobs need enough resources to finish before a deadline, and reruns must publish safely. Streaming must preserve state and input progress across failures while preventing an ever-growing backlog. Streaming is not inherently more expensive: incremental computation can avoid repeated scans, but continuous capacity and state management add operational cost. ## Choose Batch processing when - Reports are consumed daily or hourly, and the schedule plus worst-case runtime meets the freshness target. - You need large historical joins, global sorting, or a full rebuild over a bounded dataset. - Work arrives in periodic files or snapshots, and there is no product requirement to process each record immediately. - You need backfills or reconciliation against an authoritative snapshot, with repeatable inputs and versioned output. ## Choose Stream processing when - Alerts, rankings, or dashboards need updates within seconds rather than at the next scheduled run. - The application must react to events promptly, such as evaluating a price threshold or detecting a service failure. - Results can be updated incrementally, avoiding repeated scans of a large historical dataset. - Continuous ingestion would create too much delay or too large a workload if deferred until a scheduled batch. ## Worked example: Live ad counts versus billing totals Suppose an ad platform receives 10,000 click events per second. Advertisers need dashboard counts within five seconds, while billing requires reconciled daily totals. An hourly batch cannot meet the dashboard target, so a streaming pipeline deduplicates clicks by event ID, updates per-ad windowed counts, and publishes provisional results. Define how long deduplication state is retained and how clicks arriving two hours late update old windows. Store raw events durably so a daily batch can recompute totals using a declared cutoff and the same counting rules. Publish reconciled totals as a new version rather than adding them to the live counts. If daily reporting were the only requirement, the streaming pipeline would be unnecessary. ## In the interview - State the freshness target before naming technology. Distinguish event-to-visible latency from API response latency, and say whether results may be provisional. - Estimate average and peak event rates, bytes per event, key cardinality, and retained state. For batch, estimate the completion deadline; for streaming, explain how much lag is acceptable and how recovery catches up. - Define correctness using a concrete duplicate and a late event. Expect probes about event time, window boundaries, deduplication retention, and whether a late correction changes a previously published result. - Walk through a worker failure and a sink timeout. Explain input replay, checkpoint recovery, and idempotent output; do not claim end-to-end exactly-once behavior without covering the destination. - If proposing both models, explain how they share transformation rules and how reconciled results replace provisional ones without double counting. Two pipelines introduce extra testing and operational work. ## Common mistakes - Calling streaming instantaneous or batch inherently slow. Actual freshness depends on scheduling, runtime, queue lag, windowing, and publication. - Assuming batch produces correct totals automatically or streaming provides exactly-once effects automatically. Both need explicit input boundaries, retry behavior, and safe output writes. - Keeping streaming state forever without estimating its size or defining expiration and late-event behavior. - Adding separate batch and streaming pipelines by default, without showing why one execution model cannot meet the requirements. ## Interview questions where it comes up - [Design an ad click aggregator](https://sysround.com/questions/design-ad-click-aggregator): Fresh click counts and reconciled billing totals expose the trade-offs around incremental aggregation, late events, and bounded recomputation. - [Design a metrics monitoring system like Datadog](https://sysround.com/questions/design-metrics-monitoring-system): Alerts require timely stream processing, while historical rollups and recomputation can use batch jobs. - [Design Google Analytics](https://sysround.com/questions/design-google-analytics): Live dashboards and historical reports need different freshness guarantees while sharing retry-safe event counting and late-data policies. - [Design a top K trending system](https://sysround.com/questions/design-top-k): Continuously changing rankings require incremental window maintenance, while periodic batch rankings are sufficient only when stale results are acceptable. - [Design a system to sort huge datasets](https://sysround.com/questions/design-distributed-sort): Global sorting over a bounded dataset illustrates why some workloads naturally favor batch execution rather than continuous updates. ## FAQ ### What is the main difference between batch and stream processing? Batch processing operates on a bounded dataset and finishes that computation. Stream processing continuously consumes events and updates results. The practical choice depends on required freshness, computation shape, and the complexity of maintaining incremental state. ### Is micro-batching batch processing or stream processing? Micro-batching processes small bounded groups of events repeatedly and can implement a streaming system. Its freshness includes the trigger interval, processing time, and output publication delay. Evaluate whether it meets the latency and correctness requirements rather than treating the label as decisive. ### Can batch and stream processing be used together? Yes: streaming can serve fresh provisional results while batch handles historical backfills or reconciliation. Use shared counting rules, explicit cutoffs, and versioned or replacement outputs so the two paths do not double count. Only accept the extra complexity when the requirements justify it. ## Related concepts - [Stream processing](https://sysround.com/concepts/stream-processing) - [Windowing and watermarks](https://sysround.com/concepts/windowing) - [Backpressure](https://sysround.com/concepts/backpressure) - [Idempotency and idempotency keys](https://sysround.com/concepts/idempotency) - [Data lakes and warehouses](https://sysround.com/concepts/data-lake) --- SysRound (https://sysround.com) is an AI practice tool for system design interviews: a voice or typed AI interviewer, a shared whiteboard, then feedback on every skill and a model answer. Its feedback is AI-generated and can be wrong.