Documentation

Stream API

Stream API Reference

The Stream API allows you to automate the creation of new DataSets in your Domo Warehouse, featuring an accelerated upload Stream. A Domo Stream expedites uploads by dividing your data into parts, and uploading all of these parts simultaneously.

Best practices
This API should be used to create and update massive, constantly changing, or rapidly growing DataSets. For creating and updating smaller DataSets that occasionally need data updated, leverage the DataSet API. 

 

 

The Stream Object

Stream Attributes

Property Name Type Description
id Number ID of the Stream
modifiedAt String An ISO-8601 representation of the time the Stream was last updated
updateMethod String The data import behavior
createdAt String An ISO-8601 representation of the create date of the Stream

Stream's DataSet Attributes

Property Name Type Description
id String  The UID of the DataSet associated to the Stream
name   String  The description of the DataSet associated to the Stream
description   String  The ID of the DataSet associated to the Stream
rows   Number  The number of rows in the DataSet
columns  Number  The number of columns in the DataSet's schema
owner_id Number  The ID of the owner of the Stream's underlying DataSet
owner_name   String  The name of the owner of the Stream's underlying DataSet
pdpEnabled Boolean Indicates if PDP [Personalized Data Permission] policy filtering on data is active on this DataSet

Retrieve a Stream

Retrieves the details of an existing stream.

Definition

https://api.domo.com/v1/streams/{STREAM_ID}

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream
fields String Optional Return desired fields: {all} or {id, dataset, updateMethod, createdAt, or modifiedAt}

Sample Request

See it in your language

See this sample request in Java and Python.
GET https://api.domo.com/v1/streams/42
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a Stream object if valid Stream ID was provided. When requesting, if the Stream ID is related to a DataSet that has been deleted, a subset of the Stream's information will be returned, including a deleted property, which will be true.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

{
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "rows" : 0,
    "columns" : 0,
    "owner" : {
      "id" : 27,
      "name" : "DomoSupport"
    },
    "createdAt" : "2016-05-27T17:53:04Z",
    "updatedAt" : "2016-05-26T21:03:35Z"
    "pdpEnabled" : false
  },
  "updateMethod" : "APPEND",
  "createdAt" : "2016-05-26T21:03:35Z",
  "modifiedAt" : "2016-05-26T21:03:35Z"
}

Create a Stream

When creating a Stream, specify the DataSet properties (name and description) and as a convenience, the create Stream API will create a DataSet for you.

In addition, you can only have one Stream open at a time. If you need to add additional data, we recommended adding more parts to the currently open Stream or executing a commit of the open stream before creating a new stream.

Known limitation
The StreamAPI currently only allows you to import data to a DataSet created via the Stream API. For example, it is currently not supported to import data to a DataSet created by a Domo Connector.

Definition

POST https://api.domo.com/v1/streams

Arguments

Property Name Type Required Description
dataSet Object Required The DataSet object associated with this Stream
updateMethod String Optional The data import behavior: "APPEND" or "REPLACE".

Sample Request

See it in your language

See this sample request in Java and Python.
POST https://api.domo.com/v1/streams
Content-Type: application/json
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

{
  "dataSet" : {
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "schema" : {
      "columns" : [ {
        "type" : "STRING",
        "name" : "Friend"
      }, {
        "type" : "STRING",
        "name" : "Attending"
      } ]
    }
  },
  "updateMethod" : "APPEND"
}

Returns

Returns a DataSet object when successful. The returned object will have DataSet attributes based on the information that was provided when DataSet was created from the Stream created.

Sample Response

HTTP/1.1 201 Created
Location: https://api.local.domo.com/v1/streams/42
Content-Type: application/json;charset=UTF-8

{
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "rows" : 0,
    "columns" : 0,
    "owner" : {
      "id" : 27,
      "name" : "DomoSupport"
    },
    "createdAt" : "2016-05-27T17:53:04Z",
    "updatedAt" : "2016-05-27T17:53:10Z",
    "pdpEnabled" : false
  },
  "updateMethod" : "APPEND",
  "createdAt" : "2016-05-27T17:53:05Z",
  "modifiedAt" : "2016-05-27T17:53:05Z"
}

Update a Stream

Updates the specified Stream’s metadata by providing values to parameters passed.

Definition

PATCH https://api.domo.com/v1/streams/{STREAM_ID}

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream
updateMethod String Required The data import behavior

Sample Request

See it in your language

See this sample request in Java and Python.
PATCH https://api.domo.com/v1/streams/42
Content-Type: application/json
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

{
    "updateMethod":"REPLACE"
}

Returns

Returns a full DataSet object of the Stream.

Sample Response

HTTP/1.1 200 OK
Location: https://api.local.domo.com/v1/streams/42
Content-Type: application/json;charset=UTF-8

{
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "name" : "Leonhard Euler Party",
    "description" : "Mathematician Guest List",
    "rows" : 0,
    "columns" : 0,
    "owner" : {
      "id" : 27,
      "name" : "DomoSupport"
    },
    "createdAt" : "2016-05-27T17:53:04Z",
    "updatedAt" : "2016-05-27T17:53:10Z",
    "pdpEnabled" : false
  },
  "updateMethod" : "REPLACE",
  "createdAt" : "2016-05-27T17:53:05Z",
  "modifiedAt" : "2016-05-27T17:53:05Z"
}

Delete a Stream

Deletes a Stream from your Domo instance. This does not a delete the associated DataSet.

Definition

DELETE https://api.domo.com/v1/streams/{STREAM_ID}

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream to delete

Sample Request

See it in your language

See this sample request in Java and Python.
DELETE https://api.domo.com/v1/streams/42
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a Stream object and parameter of success or error based on whether the Stream ID being valid.

Sample Response

HTTP/1.1 204 No Content

List Streams

Get a list of all Streams for which the user has view permissions.

Definition

GET https://api.domo.com/v1/streams

Arguments

Property Name Type Required Description
limit Number Optional The amount of Stream to return in the list. The default is 50 and the maximum is 500.
offset Number Optional The offset of the Stream ID to begin list of users within the response.

Sample Request

See it in your language

See this sample request in Java and Python.
GET https://api.domo.com/v1/streams/?offset=0&limit=1&fields=all
Accept: application/json
Host: api.domo.com
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns all Stream objects that meet argument criteria from original request.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 500

[ {
  "id" : 1,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "pdpEnabled" : false
  },
  "updateMethod" : "APPEND",
  "createdAt" : "2016-05-26T21:03:35Z",
  "modifiedAt" : "2016-05-26T21:03:35Z"
} ]

Search Streams

Returns all Stream objects that meet argument criteria from original request.

Definition

GET https://api.domo.com/v1/streams/search

Arguments

Property Name Type Required Description
q String Required The search qualifiers to search by. Available qualifiers: dataSource.id or dataSource.owner.id
fields String Optional Return desired fields: {all} or {id, dataset, updateMethod, createdAt, or modifiedAt}

Sample Request

GET https://api.domo.com/v1/streams/search?q=dataSource.owner.id:27&fields=all
Accept: application/json
Host: api.domo.com
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns all Stream objects that meet argument criteria from original request.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 500

[ {
  "id" : 42,
  "dataSet" : {
    "id" : "0c1e0dbe-9f71-4625-9b50-b79e6e4266f2",
    "pdpEnabled" : false
  },
  "updateMethod" : "APPEND",
  "createdAt" : "2016-05-26T21:03:35Z",
  "modifiedAt" : "2016-05-26T21:03:35Z"
} ]

Retrieve a Stream execution

Import data into a DataSet in your Domo instance. This request will replace the data currently in the DataSet.

Known limitation
The only supported content type is currently CSV format.
Best practices
To upload data in CSV format, the Domo specification used for representing data grids in CSV format closely follows the RFC standard for CSV (RFC-4180). For more details on correct CSV formatting, click here.

Definition

GET https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream of data being imported into a DataSet
execution_id Number Required The ID of the Stream execution within the Stream

Sample Request

See it in your language

See this sample request in Java and Python.
GET https://api.domo.com/v1/streams/42/executions/1
Accept: application/json
Host: api.domo.com
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset fields of a Stream's object.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 384

{
  "id" : 1,
  "startedAt" : "2016-05-26T22:20:21Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-26T22:20:21Z",
  "modifiedAt" : "2016-05-26T22:20:21Z"
}

Create a Stream execution

When you’re ready to upload data to your DataSet via a Stream, you first tell Domo that you’re ready to start sending data by creating an Execution.

Warning
Creating an Execution on a Stream will abort all other Executions on that Stream. Each Stream can only have one active Execution at a time.

Definition

POST https://api.domo.com/v1/streams/{STREAM_ID}/executions

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream to create a Stream execution

Sample Request

See it in your language

See this sample request in Java and Python.
POST https://api.domo.com/v1/streams/42/executions
Content-Type: application/json
Accept: application/json
Host: api.domo.com
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset of the stream object.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 227

{
  "id" : 1,
  "startedAt" : "2016-05-26T22:20:21Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-26T22:20:21Z",
  "modifiedAt" : "2016-05-26T22:20:21Z"
}

List Stream executions

Returns all Stream Execution objects that meet argument criteria from original request.

Definition

GET https://api.domo.com/v1/streams/{STREAM_ID}/executions

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the stream
limit Number Optional The amount of Stream to return in the list. The default is 50 and the maximum is 500.
offset Number Optional The offset of the Stream ID to begin list of users within the response.

Sample Request

See it in your language

See this sample request in Java and Python.
GET https://api.domo.com/v1/streams/42/executions?limit=50&offset=0
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset of the Stream execution object from the specified Stream.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

[ {
  "id" : 1,
  "startedAt" : "2016-05-26T22:20:21Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-26T22:20:21Z",
  "modifiedAt" : "2016-05-26T22:20:21Z"
} ]

Upload a data part

Creates a data part within the Stream execution to upload chunks of rows to the DataSet. The calling client should keep track of parts and order them accordingly in an increasing sequence. If a part upload fails, retry the upload as all parts must be present before committing the stream execution.

Best practices
Parts can be uploaded simultaneously in separate threads assuming that each part has a distinct part ID and is ordered correctly. To reduce upload time, compress each data as a gzip file (application/gzip).

 

 

Definition

PUT https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}/part/{PART_ID}

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream of data being imported into a DataSet
execution_id Number Required The ID of the Stream execution within the Stream
part_id Number Required The ID of the data part being used to upload a subset of data within the Stream execution

Sample Request

See it in your language

See this sample request in Java and Python.
PUT https://api.domo.com/v1/streams/42/executions/1/part/1 HTTP/1.1
Content-Type: text/csv
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

"Pythagoras","FALSE"
"Alan Turing","TRUE"
"George Boole","TRUE"

Returns

Returns a subset of a stream object and a parameter of success or error based on whether the data part within the stream execution being successful.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8
Content-Length: 209

{
  "id" : 1,
  "startedAt" : "2016-06-16T17:54:02Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-06-16T17:54:01Z",
  "modifiedAt" : "2016-06-16T17:54:02Z"
}

Commit a Stream execution

Commits stream execution to import combined set of data parts that have been successfully uploaded.

Known limitation
The Stream API only supports the ability to execute a “commit” every 15 minutes.

Definition

PUT https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}/commit

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream of data being imported into a DataSet
execution_id Number Required The ID of the Stream execution within the Stream

Sample Request

See it in your language

See this sample request in Java and Python.
PUT https://api.domo.com/v1/streams/42/executions/1/commit HTTP/1.1
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a subset of a stream object and a parameter of success or error based on whether the stream execution successfully committed to Domo.

Sample Response

HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

{
  "id" : 1,
  "startedAt" : "2016-05-27T15:16:01Z",
  "currentState" : "ACTIVE",
  "createdAt" : "2016-05-27T15:15:59Z",
  "modifiedAt" : "2016-05-27T15:15:59Z"
}

Abort a Stream execution

If needed during an execution, aborts an entire Stream execution.

Best practices
To abort the current stream execution within a Stream, simply identify the Stream’s ID within request

Definition

PUT https://api.domo.com/v1/streams/{STREAM_ID}/executions/{EXECUTION_ID}/abort

Arguments

Property Name Type Required Description
stream_id Number Required The ID of the Stream of data being imported into a DataSet
execution_id Number Optional The ID of the Stream execution within the Stream, if no Stream execution ID is provided, the current Stream execution will be aborted

Sample Request

See it in your language

See this sample request in Java and Python.
PUT https://api.domo.com/v1/streams/42/executions/3/abort
Accept: application/json
Authorization: bearer <your-valid-oauth-access-token>

Returns

Returns a parameter of success or error based on whether the Stream ID being valid.

Sample Response

HTTP/1.1 204 No Content