11.10Streaming

With large data, the word:

streaming

becomes important.

Bad approach:

read 5 million rows
↓
construct one giant in-memory array
↓
transform everything

Potential result:

OutOfMemoryError

Better architecture processes data incrementally where the connectors and use case permit.

The Bulk API v2 query result operation exposes streaming strategies including in-memory, file-store, and non-repeatable-stream options.