Service API

Table of Contents

Introduction

Sesam provides a RESTful API for controlling the service and for working with the data in the datahub.

If you follow the Getting Started guide, the api will be served on the url you find on the management studio "settings" page under the "Connection url" heading. For a cloud instance it will typically be on the form "https://instance-guid.sesam.cloud/api".

All references to this URL in this document should be substituted with the real URL for your Sesam instance. The rest of this document will assume that the api can be found on this URL.

The API documentation is also self-hosted and can be explored using the Swagger user interface at "<host:port>/api/reference".

Most API calls will require you to authenticate using a JWT token .

When using the Sesam command line client you can get a token using its login command:

sesam login --server_base_url https://instance-guid.sesam.cloud/api

This will prompt you for your username and password, download a JWT token and store it locally. All subsequent sesam client commands will then use this token.

You can also use a commandline tool like curl or wget to explore the API. This will be the most useful approach if you are planning to communicate with Sesam using a script or a program.

When using curl to access the API url's directly, you will need to supply the JWT token for each HTTP request. Here's an example that creates an alias for curl that does this for you (remember to change the example URL to your real sesam instance URL):

# Create a text-file with the email and password you use to log in to Sesam:
echo "email=YOUR_EMAIL_ADDRESS&password=YOUR_PASSWORD" > cred.txt

# Download the authorization token for the specified email and password and store it in an environment variable:
export SESAM_AUTH_HEADER="Authorization: Bearer $(curl -d @cred.txt https://instance-guid.sesam.cloud/api/jwt)"

# Make an alias to run curl with the authorization token:
alias curlJWT='curl -H "$SESAM_AUTH_HEADER"'

You can then substitute curl with curlJWT in all calls to the API. The rest of this document assumes that you have created this alias.

Note that if you are communicating with the API using your own program or script, you must remember to set the HTTP header property "Authorization" to the string "Bearer " concatenated with the token you get from the command:

curl -d @cred.txt https://instance-guid.sesam.cloud/api/jwt

This was built-in to the shell script snippet we used to set up the culrJWT alias previously; the "cred.txt" file contains the text "email=YOUR_EMAIL_ADDRESS&password=YOUR_PASSWORD" without the quotes and substituted with your real username and password. For security reasons, remember to remove the cred.txt file after you have retrieved the JWT token.

The most important api urls are:

/api/pipes (i.e. https://instance-guid.sesam.cloud/api/pipes)

This returns a list of pipes. For an explanation of what a pipe is, read the Pipes concept definition.

/api/datasets (i.e. https://instance-guid.sesam.cloud/api/datasets)

This returns a list of datasets. (For an explanation of what a dataset is, read the Datasets concept definition).

For a full description of the /pipes and /datasets endpoint, read the "Pipes" and "Dataset" sections in the API Reference.

Examples

In the following we will go through a few examples of what you can do with the api. As mentioned previously, we will use the curlJWT alias we created to interact with the api from the command line. If you are using MS Windows you can install for instance Cygwin in order to install and use the curl command (and create the curlJWT alias).

Get a list of all the pipes

With curl:

curlJWT https://instance-guid.sesam.cloud/api/pipes

Using the Sesam client (after first having done sesam login- see the start of this document):

sesam get-pipes --server_base_url https://instance-guid.sesam.cloud/api

Get information about one specified pipe

To only get one specific pipe, add the pipe's "_id" attribute to the pipes-url. To get the pipe with the _id "Northwind:Products", you would do this:

With curl:

curlJWT https://instance-guid.sesam.cloud/api/pipes/Northwind:Products

Using the Sesam client:

sesam get-pipe Northwind:Products --server_base_url https://instance-guid.sesam.cloud/api

Run operations on a pipe

A pipe typically has a number of operations that can be triggered via the api. These are listed in the pipeinfo["runtime"]["supported-operations"] attribute. A typical value looks like this:

"supported-operations": [
            "enable",
            "disable",
            "start",
            "stop"
        ]

These operations are triggered by sending a POST-request to the url /pipes/{pipeID}/pump. For example: to disable the "Northwind:Products" pipe you would do this:

With curl:

curlJWT --data operation=disable https://instance-guid.sesam.cloud/api/pipes/Northwind:Products/pump

Using the Sesam client:

sesam get-pipe Northwind:Products --server_base_url https://instance-guid.sesam.cloud/api

To manually start the pipe's pump, you would do this:

With curl:

curlJWT --data operation=start https://instance-guid.sesam.cloud/api/pipes/Northwind:Products/pump
sesam start-pump Northwind:Products --server_base_url https://instance-guid.sesam.cloud/api

To stop a running pump, you would do this:

With curl:

curlJWT --data operation=stop https://instance-guid.sesam.cloud/api/pipes/Northwind:Products/pump

Using the Sesam client:

sesam stop-pump Northwind:Products --server_base_url https://instance-guid.sesam.cloud/api

Get a list of all the datasets

With curl:

curlJWT https://instance-guid.sesam.cloud/api/datasets

Using the Sesam client:

sesam get-datasets --server_base_url https://instance-guid.sesam.cloud/api

Get information about one specific dataset

To only get one specific dataset, add the dataset's "_id" attribute to the dataset-url. To get the dataset with the _id "Northwind:Products", you would do this:

With curl:

curlJWT https://instance-guid.sesam.cloud/api/datasets/Northwind:Products

Using the Sesam client:

sesam get-dataset Northwind:Products --server_base_url https://instance-guid.sesam.cloud/api

Get the content of the dataset

To see the entities in the dataset, add "/entities?limit=3" to the dataset's url, like this:

With curl:

curlJWT https://instance-guid.sesam.cloud/api/datasets/Northwind:Products/entities?limit=3

The "limit" parameter limits the number of returned entities to a managable number. Without this parameter, all the entities in the dataset would be returned. Depending on the size of the dataset, that could take a while, so it is generally a good idea to include a "limit"-parameter.

Using the Sesam client:

sesam get-dataset-entities Northwind:Products --limit 3 --server_base_url https://instance-guid.sesam.cloud/api

Get the content of the dataset as SDShare

To see the entities in the dataset as a SDShare feed, add "/sdshare-fragments" to the dataset's url, like this:

curlJWT https://instance-guid.sesam.cloud/api/datasets/Northwind:Products/sdshare-fragments

Parameters such as limit also apply to this URL.

The corresponding SDShare collection feed is available from:

curlJWT https://instance-guid.sesam.cloud/api/datasets/Northwind:Products/sdshare-collection

This collection feed URL is usually the URL you need to supply in a SDShare client.

Note that for the conversion of the entities to RDF to work, the entities must either:

  1. be pre-processed to consists of full URIs for all properties (including the _id property)

or:

  1. be pre-processed to CURIEs form AND the dataset id need to be registered as en entry in the RDF registry with appropriate prefix settings and prefix rules.

See Working with RDF for more information on how to prepare your data for RDF output.

Note that the SDShare feeds are not available through the Sesam client.

API Reference

GET /pipes

Returns a list of all the pipes

Status Codes
POST /pipes

Creates one or more pipes

Query Parameters
  • force (boolean) -- When this is set to "true" it forces the server to accept configuration errors and do a best-effort attempt to apply the new configuration. This is rarely a good idea.
Status Codes
  • 200 OK -- An array containing the new pipes
GET /pipes/{pipe_id}

Returns the specified pipe

Parameters
  • pipe_id (string) -- The id of the pipe
Status Codes
DELETE /pipes/{pipe_id}

Deletes the specified pipe

Parameters
  • pipe_id (string) -- The id of the pipe
Status Codes
GET /pipes/{pipe_id}/config

Returns the configuration of the specified pipe

Parameters
  • pipe_id (string) -- The id of the pipe
Status Codes
  • 200 OK -- The pipe configuration
PUT /pipes/{pipe_id}/config

Updates the configuration of the specified pipe

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • force (boolean) -- When this is set to "true" it forces the server to accept configuration errors and do a best-effort attempt to apply the new configuration. This is rarely a good idea.
Status Codes
  • 200 OK -- The pipe configuration
GET /pipes/{pipe_id}/entities

Returns the entities produced by the specified pipe

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • limit (integer) -- Specifies the maximum number of entities to return. If this is not specified, all available entities are returned..
  • since (string) -- Specifies the minimum since-value.
  • stage (string) -- Can be set to 'source', 'before-transforms', 'after-transforms' or 'sink' (default). 'source' returns entities as they come out of the source. 'before-transforms' returns entities as they are fed into the first transform. 'after-transforms' return entities as they look after the last transform. 'sink' returns entities as they look as they enter the sink.
Status Codes
GET /pipes/{pipe_id}/generate-schema-definition

Returns info about the schema of the entities processed by the pipe

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • sample_size (integer) -- Specifies the number of entities to sample from the source
  • keys_only (boolean) -- Specifies if only the array of keys should returned, or the whole schema
Status Codes
GET /pipes/{pipe_id}/pump

Returns info about the pump of the specified pipe

Parameters
  • pipe_id (string) -- The id of the pipe
Status Codes
POST /pipes/{pipe_id}/pump

Runs some operation on the pump. The operation is specified by the 'operation' form parameter in the request.

The following operations exists:

"start":

Starts a new run of the pump. A pump is typically run automatically at scheduled intervals, but with this operation a pump can be started manually.

"stop":

Stops the pump, if it is running. If the pump is not already running, this operation has no effect.

"enable":

Enables the pump. This means that the pump can be run automatically by the system (usually on some fixed schedule).

"disable":

Disables the pump. This means that the pump cannot be run automatically by the system. Note that it is still possible to start the pump manually with the "start"-operation.

"update-last-seen":

Updates the "last-seen" value of the pump. This is the value the pump uses to figure out at which point in the source-data it stopped at its last run. The format of the "last-seen" form parameter depends on the source-system. It could be a date, and it could be some opaque binary value.

To clear the "last-seen" value of the pump, set the "last-seen" form parameter to an empty string. This will make the pump behave as if it was running for the first time. This is useful in cases where one wants to reprocess the data from scratch for some reason.

"set-last-seen-to-end":

Updates the "last-seen" value of the pump such that the pump thinks that it has seen all the source-data. Not all source-types supports this operation. NOTE: Doing this may result in data-loss, since any unread entities in the source will not be read. But

it is sometimes useful during development and testing; for instance when one has a huge existing source-dataset, but only want to run the pipe against new entities.

"reset-tracking" (Experimental):

Resets the dependency tracking so that it will continue starting from the end of dependent datasets.
Parameters
  • pipe_id (string) -- The id of the pipe
Status Codes
GET /datasets

Returns a list of all the datasets

Status Codes
  • 200 OK -- An array of datasets
GET /datasets/{dataset_id}

Returns the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
Status Codes
DELETE /datasets/{dataset_id}

Deletes the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
Status Codes
POST /datasets/{dataset_id}
Parameters
  • dataset_id (string) -- The id of the dataset
Status Codes
GET /datasets/{dataset_id}/entities

Returns the entities in the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
Query Parameters
  • limit (integer) -- Specifies the maximum number of entities to return. If this is not specified, all available entities are returned..
  • since (integer) -- An opaque value that can be used to efficiently skip already seen entities.
  • history (boolean) -- If this is true (the default) all entities, including replaced ones, will be returned. If this is false, only the latest version of the entities will be returned
  • reverse (boolean) -- If this is false (the default) the entities will be returned from the start of the dataset log. If this is true, the entities will be returned from the end of the dataset log instead
  • deleted (boolean) -- If this is true (the default) the entities marked as deleted will also be returned from the dataset log. If this is false, the entities that are marked as deleted will not included in result. If you request the complete history the deletes will always be included.
Status Codes
POST /datasets/{dataset_id}/entities

Puts the specified entities into the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
Query Parameters
  • force (boolean) -- If 'false', then the hashes of the posted entity and the previous version will be compared before storing the entity. The entity will not be written to the dataset if the hashes are the same. If 'true', then the entity will always be written to the dataset. The default is 'false'.
Status Codes
GET /datasets/{dataset_id}/entities/<path:entity_id>

Returns the specified entity in the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
  • entity_id (string) -- The id of the entity
Status Codes
GET /datasets/{dataset_id}/sdshare-collection

Returns a sdshare collection feed for the entities in the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
Status Codes
GET /datasets/{dataset_id}/sdshare-fragments

Returns a sdshare fragments feed for the entities in the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
Status Codes
GET /datasets/{dataset_id}/sdshare-fragments/<path:entity_id>

Returns a sdshare fragment for the specified entity in the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
  • entity_id (string) -- The id of the entity-id
Status Codes
GET /datasets/{dataset_id}/indexes

Returns the indexes on the specified dataset

Parameters
  • dataset_id (string) -- The id of the dataset
Status Codes
  • 200 OK -- The indexes of the dataset.
GET /config

Returns the entire configuration of the node as an array of json objects, or as a zip-archive, depending on the value of the 'Accepts' header.

Status Codes
PUT /config

Overwrites the entire configuration of the node.

Status Codes
GET /
Status Codes
POST /
Status Codes
GET /env

Returns the currently defined environment variable values.

Status Codes
POST /env

Adds a variable or list of variables to the environment variable manager. If a variable already exists, it is overwritten.

Status Codes
PUT /env

Adds a variable or list of variables to the environment variable manager. If a variable already exists, it is overwritten.

Status Codes
PATCH /env

Adds a variable or list of variables to the environment variable manager. If a variable already exists, it is overwritten.

Status Codes
DELETE /env/{env_var}

Deleted the specified environment variable.

Parameters
  • env_var (string) -- The name of the environment variable to delete
Status Codes
GET /secrets

Returns the names of the currently defined secrets.

Status Codes
PUT /secrets

Adds a secret or list of secrets to the secrets manager. If a secret already exists, it is overwritten.

Status Codes
DELETE /secrets/{secret_key}

Deleted the specified secret.

Parameters
  • secret_key (string) -- The name of the secret to delete
Status Codes
GET /health

Sesam api health information

Status Codes
GET /license

Returns info about the current license

Status Codes
PUT /license

Upload a new license key. The new license will replace the old license if valid.

Status Codes
GET /metadata

Returns the service metadata

Status Codes
PUT /metadata

Updates the service metadata

Status Codes
DELETE /metadata

Deletes all service metadata

Status Codes
GET /publishers/{pipe_id}/sdshare-collection

publisher endpoint sdshare collection feed

Parameters
  • pipe_id (string) -- The id of the pipe
Status Codes
GET /publishers/{pipe_id}/sdshare-fragments

publisher endpoint sdshare fragments feed

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • since (string) -- Specifies the minimum since-value.
  • limit (integer) -- Specifies the maximum number of entities to return. If this is not specified, all available entities are returned..
Status Codes
GET /publishers/{pipe_id}/entities

publisher endpoint json entities

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • since (string) -- Specifies the minimum since-value.
  • limit (integer) -- Specifies the maximum number of entities to return. If this is not specified, all available entities are returned..
Status Codes
GET /publishers/{pipe_id}/csv

publisher csv endpoint

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • since (string) -- Specifies the minimum since-value.
  • limit (integer) -- Specifies the maximum number of entities to return. If this is not specified, all available entities are returned..
Status Codes
GET /publishers/{pipe_id}/xml

publisher xml endpoint

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • since (string) -- Specifies the minimum since-value.
  • limit (integer) -- Specifies the maximum number of entities to return. If this is not specified, all available entities are returned..
Status Codes
POST /receivers/{pipe_id}/sdshare-push-receiver

Pushes n-triples to the named pipe. This is only supported for pipes that have an 'http_endpoint' source.

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • resource (string) -- Specifies the root resources
Status Codes
POST /receivers/{pipe_id}/entities

Sesam api receiver endpoint: entities

Parameters
  • pipe_id (string) -- The id of the pipe
Query Parameters
  • is_full (boolean) --
Status Codes
GET /systems

Returns a list of all the systems

Status Codes
  • 200 OK -- An array of systems
POST /systems

Creates one or more systems

Query Parameters
  • force (boolean) -- When this is set to "true" it forces the server to accept configuration errors and do a best-effort attempt to apply the new configuration. This is rarely a good idea.
Status Codes
  • 200 OK -- An array containing the new systems
GET /systems/{system_id}

Returns the specified system

Parameters
  • system_id (string) -- The id of the system
Status Codes
DELETE /systems/{system_id}

Deletes the specified system

Parameters
  • system_id (string) -- The id of the system
Status Codes
GET /systems/{system_id}/logs

Returns the logs from the microservice (1000 lines starting from 'since')

Parameters
  • system_id (string) -- The id of the system
Query Parameters
  • since (string) -- Specifies from which point in time to stream the logs. The value must be an ISO datetime value, e.g. '2016-11-08T08:28:45.880594510Z'.
Status Codes
  • 200 OK -- The microservice logs data
GET /systems/{system_id}/status

Returns the status from the microservice (subset of docker inspect)

Parameters
  • system_id (string) -- The id of the system
Status Codes
  • 200 OK -- The microservice status data
POST /systems/{system_id}/reload

Queues a operation tp pull a new docker image and recreate the microservice (i.e. asynchronously)

Parameters
  • system_id (string) -- The id of the system
Status Codes
GET /systems/{system_id}/proxy/<path:relative_path>

Passes the request through to the microservice and returns the response

Parameters
  • system_id (string) -- The id of the system
  • relative_path (string) -- The relative path to access on the microservice
Status Codes
  • 200 OK -- The proxied response from the microservice
POST /systems/{system_id}/proxy/<path:relative_path>

Passes the request through to the microservice and returns the response

Parameters
  • system_id (string) -- The id of the system
  • relative_path (string) -- The relative path to access on the microservice
Status Codes
  • 200 OK -- The proxied response from the microservice
PUT /systems/{system_id}/proxy/<path:relative_path>

Passes the request through to the microservice and returns the response

Parameters
  • system_id (string) -- The id of the system
  • relative_path (string) -- The relative path to access on the microservice
Status Codes
  • 200 OK -- The proxied response from the microservice
PATCH /systems/{system_id}/proxy/<path:relative_path>

Passes the request through to the microservice and returns the response

Parameters
  • system_id (string) -- The id of the system
  • relative_path (string) -- The relative path to access on the microservice
Status Codes
  • 200 OK -- The proxied response from the microservice
DELETE /systems/{system_id}/proxy/<path:relative_path>

Passes the request through to the microservice and returns the response

Parameters
  • system_id (string) -- The id of the system
  • relative_path (string) -- The relative path to access on the microservice
Status Codes
  • 200 OK -- The proxied response from the microservice
GET /systems/{system_id}/config

Returns the configuration of the specified system

Parameters
  • system_id (string) -- The id of the system
Status Codes
  • 200 OK -- The system configuration
PUT /systems/{system_id}/config

This service lets the user update a system's configuration by sending a PUT-request with the new configuration. The id of the system is specified by the '{system_id}' part of the url.

Parameters
  • system_id (string) -- The id of the system
Query Parameters
  • force (boolean) -- When this is set to "true" it forces the server to accept configuration errors and do a best-effort attempt to apply the new configuration. This is rarely a good idea.
Status Codes
  • 200 OK -- The system configuration