Skip to main content
API docs by Redocly

LogStore API

Query Data 

/stores/{id}/data/partitions/{partition}/{queryType}

The Query Data endpoint fetches data from a specific stream partition using three types of queries: last, from, range. The endpoint returns a stream of data, which can be processed in real-time or piece-by-piece, rather than all at once.

  • format to specify response data format, which can be object or raw.
  • version to define the version of the data to query.
  • Additional parameters like count, fromTimestamp, fromSequenceNumber, toTimestamp, toSequenceNumber, publisherId, and msgChainId to narrow down the data based on the chosen query type.
  • verifyNetworkResponses to include additional metadata on the response, enabling client-side network verification.

The Request Type

This API can be accessed either through regular HTTP requests or by utilizing Server-Sent Events (SSE). Regular HTTP responses are limited to 5000 messages per request. This limitation is in place to prevent overloading the server or the client with too much data at once.

If your system is compatible, we recommend using SSE. This approach allows for the processing of event messages in real-time or incrementally, as opposed to processing all at once. This method can help conserve memory in your application, especially when dealing with lengthy messages.

By default, your request will be processed as a standard HTTP request. However, if you include Accept: text/event-stream in your request header, it will be treated as an SSE request.

When you use SSE, the final event that is streamed contains metadata about the request. Please refer to the subsequent details for more information.

Response

Given these types:

// as specified in https://github.com/streamr-dev/streamr-specs/blob/master/PROTOCOL.md#stream-protocol
type RawMessage = [
  version: number,
  msgIdFields: string[],
  msgRefFields: string[],
  messageType: number,
  contentType: number,
  encryptionType: number,
  groupKeyId: string,
  content: string,
  newGroupKey: string | null,
  signatureType: number,
  signature: string,
];

type MessageObject = {
  metadata: {
    id: {
      streamId: string;
      streamPartition: number;
      timestamp: number;
      sequenceNumber: number;
      publisherId: string;
      msgChainId: string;
    };
    prevMsgRef: null | object;
    messageType: number;
    contentType: number;
    encryptionType: number;
    groupKeyId: string | null;
    newGroupKey: string | null;
    signature: string;
  };
  content: object;
};

type HTTPQueryJSONResponse = {
  messages: MessageObject[];
  metadata: {
    hasNext: boolean;
    nextTimestamp: number;
    nextSequenceNumber: number;
    totalMessages: number;
    participatingNodesAddress?: string[]; // included when network is verified
    requestId?: string; // included when network is verified
  };
};

These are the response shapes depending on the type of request and format parameter used:

Shape of the response based on format parameter and request type HTTP SSE
raw Stringified RawMessage, separated by ‘\n’ Each event is a stringified RawMessage.

The last event is a StreamResponseMetadata
object HTTPQueryJSONResponse Each event is a MessageObject.

The last event is a StreamResponseMetadata

Content Encryption

Content encryption depends on the privacy settings of your stream. For public streams, the content of your messages remains unencrypted. However, for private streams, the content is encrypted for added security.

Our client, which extends the StreamrClient class, is designed to handle the decryption process for you. This process involves a key exchange mechanism, which necessitates the online presence of the publisher(s). For more detailed information, please refer to the Streamr Documentation.

Authorizations:
basicAuth
path Parameters
id
required
string

The ID of the stream.

partition
required
integer

The partition number within the stream.

queryType
required
string
Enum: "last" "from" "range"

The type of query to perform.

query Parameters
format
string
Enum: "object" "raw"

The format of the response. See the descriptions above

version
integer

The version of the data to use. Default is the latest.

count
integer

The number of records to fetch (only for 'last' query type). Note: If a negative number is provided, it will fetch the first messages.

fromTimestamp
integer

The starting timestamp for the query (for 'from' and 'range' query types).

fromSequenceNumber
integer

The starting sequence number (for 'from' and 'range' query types).

toTimestamp
integer

The ending timestamp for the query (only for 'range' query type).

toSequenceNumber
integer

The ending sequence number (only for 'range' query type).

publisherId
string

The ID of the publisher (only for 'range' query type when publisherId is given).

msgChainId
string

The ID of the message chain (only for 'range' query type when publisherId is given).

verifyNetworkResponses
boolean

When set to true, the response will include additional metadata, enabling client-side network verification.

Responses

Response samples

Content type
Example
{
  • "messages": [
    ],
  • "metadata": {
    }
}