Stream API

Stream API Reference

The Stream API allows you to automate the creation of new DataSets in your Domo Warehouse, featuring an accelerated upload Stream. A Domo Stream expedites uploads by dividing your data into parts, and uploading all of these parts simultaneously.

Best practices
This API should be used to create and update massive, constantly changing, or rapidly growing DataSets. For creating and updating smaller DataSets that occasionally need data updated, leverage the DataSet API. 

 

 

The stream object

Stream Attributes

Property Name Type Description
stream_id String ID of the Stream
modifiedAt String An ISO-8601 representation of the time the Stream was last updated
updateMethod String The data import behavior
createdAt String An ISO-8601 representation of the create date of the Stream

Stream's DataSet Attributes

Property Name Type Description
dataset_id string  The id of the DataSet associated to the Stream
name   string  The description of the DataSet associated to the Stream
description   string  The id of the DataSet associated to the Stream
rows   string  The number of rows in the DataSet
columns  number  The number of columns in the DataSet's schema
owner_id number  The ID of the owner of the stream's underlying DataSet
owner_name   string  The name of the owner of the stream's underlying DataSet

Retrieve a stream

Retrieves the details of an existing stream.

Definition

https://api.domo.com/v1/streams/{SREAM_ID}

Arguments

Property Name Type Required Description
stream_id string Required The ID of the Stream
fields string - Return desired fields: {all} or {stream_id, dataset, updatemethod, createdat, or modifiedat}

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
GET https://api.domo.com/v1/streams/42
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a Stream object if valid Stream ID was provided. When requesting, if the Stream ID is related to a DataSet that has been deleted, a subset of the Stream's information will be returned, including a deleted property, which will be true.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

{
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "rows" : 0,
    "columns" : 0,
    "owner" : {
      "id" : 27,
      "name" : "DomoSupport"
    },
    "createdAt" : "2016-05-27T17:53:04Z",
    "updatedAt" : "2016-05-26T21:03:35Z"
  },
  "updateMethod" : "APPEND",
  "createdAt" : "2016-05-26T21:03:35Z",
  "modifiedAt" : "2016-05-26T21:03:35Z"
}

Create a stream

When creating a stream, specify the DataSet properties (name and description) and as a convenience, the create stream API will create a DataSet for you.

In addition, you can only have one stream open at a time. If you need to add additional data, we recommended adding more parts to the currently open stream or executing a commit of the open stream before creating a new stream.

Known limitation
The StreamAPI currently only allows you to import data to a DataSet created via the Stream API. For example, it is currently not supported to import data to a DataSet created by a Domo Connector.

Definition

POST https://api.domo.com/v1/streams

Arguments

Property Name Type Required Description
dataSet Object Required The DataSet object associated with this Stream
updateMethod String - The data import behavior: "APPEND" or "REPLACE".

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
POST https://api.domo.com/v1/streams
Content-Type: application/json
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

{
  "dataSet" : {
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "schema" : {
      "columns" : [ {
        "type" : "STRING",
        "name" : "Friend"
      }, {
        "type" : "STRING",
        "name" : "Attending"
      } ]
    }
  },
  "updateMethod" : "APPEND"
}

Returns

Returns a DataSet object when successful. The returned object will have DataSet attributes based on the information that was provided when DataSet was created from the Stream created.

Sample Response

HTTP/1.1 201 Created
Location: https://api.local.domo.com/v1/streams/42
Content-Type: application/json;charset=UTF-8

{
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "rows" : 0,
    "columns" : 0,
    "owner" : {
      "id" : 27,
      "name" : "DomoSupport"
    },
    "createdAt" : "2016-05-27T17:53:04Z",
    "updatedAt" : "2016-05-27T17:53:10Z"
  },
  "updateMethod" : "APPEND",
  "createdAt" : "2016-05-27T17:53:05Z",
  "modifiedAt" : "2016-05-27T17:53:05Z"
}

Update a stream

Updates the specified Stream’s metadata by providing values to parameters passed.

Definition

PATCH https://api.domo.com/v1/streams/{STREAM_ID}

Arguments

Property Name Type Required Description
updateMethod String Required The data import behavior

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
PUT https://api.domo.com/v1/datasets/317970a1-6a6e-4f70-8e09-44cf5f34cf44
Content-Type: application/json
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

{
  "name": "Leonhard Euler Birthday Bash",
  "description": "VIP Guest List",
  "pdpEnabled": true
}

Returns

Returns a full DataSet object of the Stream.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

{
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "rows" : 0,
    "columns" : 0,
    "owner" : {
      "id" : 27,
      "name" : "DomoSupport"
    },
    "createdAt" : "2016-05-27T17:38:54Z",
    "updatedAt" : "2016-05-27T17:38:54Z"
  },
  "updateMethod" : "REPLACE",
  "createdAt" : "2016-05-27T17:38:54Z",
  "modifiedAt" : "2016-05-27T17:43:21Z"
}

Delete a stream

Deletes a Stream from your Domo instance. This does not a delete the associated DataSet.

Definition

DELETE https://api.domo.com/v1/streams/{STREAM_ID}

Arguments

Property Name Type Required Description
stream_id string Required The ID of the Stream to delete

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
DELETE https://api.domo.com/v1/streams/42
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a Stream object and parameter of success or error based on whether the Stream ID being valid.

Sample Response

HTTP/1.1 204 No Content

List streams

Get a list of all Streams for a specific DataSet.

Definition

GET https://api.domo.com/v1/streams

Arguments

Property Name Type Required Description
q string optional Search query. Possible parameters to search on: 'dataSource.id', 'dataSource.name', 'dataSource.owner.id'.
fields string optional A response filter allowing you to control the amount of data returned. Applicable fields are "DataSet", "updateMethod", ''createdAt", and "modifiedAt".
sort string optional The Stream field to sort by. Fields prefixed with a negative sign reverses the sort (i.e. '-name' does a reverse sort by the name of the streams)
limit long optional The amount of Stream to return in the list. The default is 50 and the maximum is 500.
offset long optional The offset of the Stream ID to begin list of users within the response.

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
GET https://api.domo.com/v1/streams/search?q=dataSource.owner.id:27&fields=all
Accept: application/json
Host: api.domo.com
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns all Stream objects that meet argument criteria from original request.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 500

[ {
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "rows" : 0,
    "columns" : 0,
    "owner" : {
      "id" : 27,
      "name" : "DomoSupport"
    },
    "createdAt" : "2016-05-26T21:03:34Z",
    "updatedAt" : "2016-05-26T21:03:35Z"
  },
  "updateMethod" : "APPEND",
  "createdAt" : "2016-05-26T21:03:35Z",
  "modifiedAt" : "2016-05-26T21:03:35Z"
} ]

Retrieve a stream execution

Import data into a DataSet in your Domo instance. This request will replace the data currently in the DataSet.

Known limitation
The only supported content type is currently CSV format.
Best practices
To upload data in CSV format, the Domo specification used for representing data grids in CSV format closely follows the RFC standard for CSV (RFC-4180). For more details on correct CSV formatting, click here.

Definition

GET https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}

Arguments

Property Name Type Required Description
stream_id string Required The ID of the Stream of data being imported into a DataSet
execution_id string Required The ID of the Stream execution within the Stream

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
GET https://api.domo.com/v1/streams/42/executions/1
Accept: application/json
Host: api.domo.com
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset fields of a Stream's object.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 384

{
  "id" : 1,
  "startedAt" : "2016-05-26T22:20:21Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-26T22:20:21Z",
  "modifiedAt" : "2016-05-26T22:20:21Z"
}

Create a stream execution

When you’re ready to upload data to your DataSet via a Stream, you first tell Domo that you’re ready to start sending data by creating an Execution.

Warning
Creating an Execution on a Stream will abort all other Executions on that Stream. Each Stream can only have one active Execution at a time.

Definition

POST https://api.domo.com/v1/streams/{STREAM_ID}/executions

Arguments

Property Name Type Required Description
stream_id string Required The ID of the Stream to create a Stream execution

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
POST https://api.domo.com/v1/streams/42/executions
Content-Type: application/json
Accept: application/json
Host: api.domo.com
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset of the stream object.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 227

{
  "id" : 1,
  "startedAt" : "2016-05-26T22:20:21Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-26T22:20:21Z",
  "modifiedAt" : "2016-05-26T22:20:21Z"
}

List stream executions

Retrieve a policy from a DataSet within Domo. A DataSet is required for a PDP policy to exist.

Definition

GET https://api.domo.com/v1/streams/{STREAM_ID}/executions

Arguments

Property Name Type Required Description
stream_id string Required The ID of the stream
limit long optional The amount of Stream to return in the list. The default is 50 and the maximum is 500.
offset long optional The offset of the Stream ID to begin list of users within the response.

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
GET https://api.domo.com/v1/streams/42/executions?limit=50&offset=0
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset of the Stream execution object from the specified Stream.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

[ {
  "id" : 1,
  "startedAt" : "2016-05-26T22:20:21Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-26T22:20:21Z",
  "modifiedAt" : "2016-05-26T22:20:21Z"
} ]

Upload a data part

Creates a data part within the Stream execution to upload chunks of rows to the DataSet. The calling client should keep track of parts and order them accordingly in an increasing sequence. If a part upload fails, retry the upload as all parts must be present before committing the stream execution.

Best practices
Parts can be uploaded simultaneously in separate threads assuming that each part has a distinct part ID and is ordered correctly. To reduce upload time, compress each data as a gzip file (application/gzip).

 

 

Definition

PUT https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}/part/{PART_ID}

Arguments

Property Name Type Required Description
stream_id string Required The ID of the Stream of data being imported into a DataSet
execution_id string Required The ID of the Stream execution within the Stream
part_id string Required The ID of the data part being used to upload a subset of data within the Stream execution

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
PUT https://api.domo.com/v1/streams/42/executions/1/part/1 HTTP/1.1
Content-Type: text/csv
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

"Pythagoras","FALSE"
"Alan Turing","TRUE"
"George Boole","TRUE"

Returns

Returns a subset of a stream object and a parameter of success or error based on whether the data part within the stream execution being successful.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 209

{
  "id" : 1,
  "startedAt" : "2016-06-16T17:54:02Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-06-16T17:54:01Z",
  "modifiedAt" : "2016-06-16T17:54:02Z"
}

Commit a stream execution

Commits stream execution to import combined set of data parts that have been successfully uploaded.

Known limitation
The Stream API only supports the ability to execute a “commit” every 15 minutes.

Definition

PUT https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}/commit

Arguments

Property Name Type Required Description
stream_id string Required The ID of the Stream of data being imported into a DataSet
execution_id string Required The ID of the Stream execution within the Stream

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
PUT https://api.domo.com/v1/streams/42/executions/1/commit HTTP/1.1
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset of a stream object and a parameter of success or error based on whether the stream execution successfully committed to Domo.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

{
  "id" : 1,
  "startedAt" : "2016-05-27T15:16:01Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-27T15:15:59Z",
  "modifiedAt" : "2016-05-27T15:15:59Z"
}

Abort a stream execution

If needed during an execution, aborts an entire Stream execution.

Best practices
To abort the current stream execution within a Stream, simply identify the Stream’s ID within request

Definition

PUT https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}/abort

Arguments

Property Name Type Required Description
stream_id string Required The ID of the Stream of data being imported into a DataSet
execution_id string Optional The ID of the Stream execution within the Stream, if no Stream execution ID is provided, the current Stream execution will be aborted

Sample Request

See it in your language

See this sample request in Java, Python, and Node.
PUT https://api.domo.com/v1/streams/42/executions/3/abort
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a parameter of success or error based on whether the Stream ID being valid.

Sample Response

HTTP/1.1 204 No Content