Streaming Data
Learn the process for streaming data from a Synnax cluster.
This page walks you through the high level concepts for streaming real-time data from a Synnax cluster. If you’d like a practical guide on reading data using a client library, look at the respective pages for Python and Typescript.
Overview
Streaming data from a Synnax cluster is useful for real-time data processing, live
visualizations, and control sequences. The main interface for streaming data is the
Streamer object.
A single streamer can read data from multiple channels at a single time, and only updates when new data is available. This makes processing both network and local data streams easy and efficient.
Streamer
Opening a Streamer
To open a streamer, you’ll need to provide a list of channels that you’d like to stream data from. The streamer will validate that the channels exists and are readable before starting the stream.
Optional Arguments
There are also a few optional arguments we can provide when opening a streamer:
| Argument | Description |
|---|---|
| Downsample Factor | A positive integer that determines the reduction in the stream’s original data rate. A value of n means only every nth sample is retained. Defaults to 1, meaning no downsampling. |
Reading Data
Once the streamer is open, you can read data from the stream using the read method.
This will block until any of the channels have new data available, at which point the
method will return a frame with the new data.
The frame returned by the read method will not always contain data from all channels specified when opening the streamer. It will only contain data from the channels whose values changed since the last read.
The series in each frame may also have varying lengths, as certain channels may be receiving samples in batches and/or at different rates.
For example, if you’re reading from channel temp_1 and temp_2, the frame returned
may look like this:
{
"temp_1": [20.0, 21.0, 22.0],
"temp_2": [30.0, 31.0]
}
Or it may look like this:
{
"temp_2": [30.0, 31.0, 32.0]
}
Updating the Streamer’s Channels
It’s possible to change the channels that the streamer reads from at any time. This is useful for dynamically changing the data that you’re interested in, such as when a user selects a new set of channels to load in a plot.
Closing a Streamer
When you’re done receiving data, it’s very important ot close the streamer. This guarantees that all network connections are closed and local resources are freed.