Glossary
Stream processing
Processing data continuously as individual events arrive, rather than waiting to accumulate a batch.
Stream processing computes results on data as each event arrives, typically within milliseconds to seconds, instead of waiting to collect a batch. A stream processor might update a running total of active users, flag a fraudulent transaction, or trigger an alert the moment a threshold is crossed, all continuously rather than on a schedule.
Frameworks such as Apache Flink, Kafka Streams and Spark Structured Streaming read from an event streaming platform like Apache Kafka and maintain state, running counts, windows, joins, as events flow through, updating results incrementally instead of recomputing from scratch. This differs from batch processing, which processes a bounded, already-collected dataset; stream processing operates on an unbounded sequence that never formally "ends," and it must decide how to handle events that arrive out of order using techniques like watermarking.
Stream processing matters wherever the value of an insight decays quickly, operational monitoring, live personalization, real-time fraud detection, but it comes with real complexity: guaranteeing exactly-once processing semantics, managing state that can grow unbounded, and handling late or duplicate events all require more engineering care than an equivalent batch job. Teams that don't need sub-minute latency often get most of the benefit at much lower cost from micro-batching instead.
Last reviewed September 22, 2026