JSON source¶

The JSON source can read entities from a JSON resource available over HTTP (i.e. served by a web server). The service must conform to the JSON Pull Protocol.

Consider using the more general REST source if you’re interacting with a non-Sesam JSON capable REST api.

If the supports_since property is set to true, then the since request parameter is added to the URL to signal that we want only changes that happened after the since marker.

Prototype¶

{
   "system": "system-id",
   "type": "json",
   "url": "url-to-json-data",
   "supports_signalling": false,
   "headers": {
       "some-header": "some-value"
   }
}

Properties¶

Property	Type	Description	Default	Req
`system`	String	The id of the URL system or microservice system component to use.		Yes
`url`	String	The URL of the `JSON` data to load. Note that the data must conform to the JSON Pull Protocol.		Yes
`supports_signalling`	Boolean	Flag used to enable or disable signalling support between internal pipes (dataset to dataset pipes). If enabled, a pipe run is scheduled as soon as the input dataset(s) changes. It does not interrupt any already running pipes.	`false`
`page_size`	Integer(>=1)	If the page size is specified then the source will download the data across multiple requests until there is no more data left to download. The `limit` request parameter is passed to the endpoint to cap the number of entities in each response. Note Paging is only available if the source has `supports_since`, `is_chronological` and `is_since_comparable` all set to `true`.	No paging
`subset`	Array	An `eq` DTL expression where the left hand side is the index expression and the right hand side is the value that represents the subset. If the subset is specified then only entities that are in that subset will be read from the source. Example: `["eq", "_S.category", "tank"]` Note For this to work the source must support subsets.		No
`headers`	Dict<String,String>	A optional set of header values to set in HTTP request made using this source. Both keys and values must evaluate to strings.
`if_source_empty`	Enum<String>	Determines the behaviour of the pipe when the JSON source does not return any entities. Normally, any previously synced entities will be deleted even if the pipe does not receive any entities from its source. If set to `"fail"`, the pipe will automatically fail if the source returns no entities. This means that any previous entities in the pipe’s dataset are not deleted. If set to `"accept"`, the pipe will not fail and any previously synced entities will be deleted. The global default `global_defaults.if_source_empty` can be set for all pipes in the service metadata.	`"accept"`

Continuation support¶

See the section on continuation support for more information.

Property	Value
`supports_since`	`false` (Default)
`is_since_comparable`	`true` (Default)
`is_chronological`	`false` (Default)

Example configuration¶

The outermost object would be your pipe configuration, which is omitted here for brevity:

{
    "source": {
        "type": "json",
        "system": "some-url-or-microservice-system",
        "url": "test.json",
    }
}

HTTP endpoint source

LDAP source