Ingestion Queue Endpoints

The ingestion queue endpoints are used to view and manage the queue of repositories to be added to the service. They respond and deal with one of the following entities:

Entities

Ingestion Entity

Ingestion entities represent individual repositories in the queue, which are waiting to be ingested or are currently being ingested. As JSON they look like this:

{
	// A unique identifier for the ingestion
	"id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",

	// The repository and tag which will be ingested to create a
	// new version in the database
	"repo": {
		"url": "https://github.com/Financial-Times/example",
		"tag": "v1.0.0"
	},

	// The progress that has been made so far in ingesting this
	// repository and tag
	"progress": {

		// Whether this item is currently being ingested
		"isInProgress": false,

		// The time that the ingestion process was started, either
		// null when it is not in progress, or a date string if it is
		"startTime": null,

		// How many attempts have been made to ingest this
		// repository and tag combination
		"attempts": 0

	},

	// When the ingestion was created
	"created": "2017-11-08T12:13:00.000Z",

	// When the ingestion was last updated
	"lastUpdated": "2017-11-08T12:13:00.000Z"
}

Get the full queue

Get a list of all current ingestions in the queue as an array. This endpoint responds with an array of Ingestion entities. This endpoint requires the READ permission.

Request

Method GET
Path /v1/queue
Headers
X-Api-Key
See authentication docs
X-Api-Secret
See authentication docs

Response

Status 200 on success
401 if authentication failed
403 if authorization failed
Headers
Content-Type
application/json on success
text/html on error
Body Array of Ingestion entities

Example curl command:

curl \
	-H 'X-Api-Key: XXXXXX' -H 'X-Api-Secret: XXXXXX' \
	https://origami-repo-data.ft.com/v1/queue

Get an ingestion

Get a single ingestion in the queue by ID. This endpoint responds with a Ingestion entity. This endpoint requires the READ permission.

Request

Method GET
Path /v1/queue/:ingestion-id
(where :ingestion-id is the unique identifier for a Ingestion)
Headers
X-Api-Key
See authentication docs
X-Api-Secret
See authentication docs

Response

Status 200 on success
401 if authentication failed
403 if authorization failed
404 if an ingestion matching :ingestion-id does not exist
Headers
Content-Type
application/json on success
text/html on error
Body Ingestion entity

Example curl command:

curl \
	-H 'X-Api-Key: XXXXXX' -H 'X-Api-Secret: XXXXXX' \
	https://origami-repo-data.ft.com/v1/queue/XXXXXX

Add an ingestion to the queue

Create a new ingestion and add it to the queue. This endpoint accepts ingestion details and responds with a newly created Ingestion entity. This endpoint requires the WRITE permission.

ℹ️ This endpoint can also function as a GitHub webhook. There's a section on this below the standard request/response documentation.

Request

Method POST
Path /v1/queue
Headers
Content-Type
application/json
X-Api-Key
See authentication docs
X-Api-Secret
See authentication docs
Body
{
	// The GitHub repository URL to ingest (required)
	"url": "https://github.com/Financial-Times/example",

	// The GitHub repository tag to ingest (required)
	"tag": "v1.0.0"
}

Response

Status 201 on success
400 if the request data is invalid or malformed
401 if authentication failed
403 if authorization failed
Headers
Content-Type
application/json on success
text/html on error
Body Ingestion entity

Example curl command:

curl -X POST \
	-H 'Content-Type: application/json' -H 'X-Api-Key: XXXXXX' -H 'X-Api-Secret: XXXXXX' \
	-d '{"url": "https://github.com/Financial-Times/example", "tag": "v1.0.0"}' \
	https://origami-repo-data.ft.com/v1/queue

GitHub Webhook

You can also automatically add repository/tag combinations to the queue via a GitHub webhook. This further automates the service. We recommend doing this as an organisation-wide hook.

  1. Create a new webhook on either your repository or origanisation

  2. Set the Payload URL to this endpoint. You'll need to include the apiKey and apiSecret query parameters rather than headers, as GitHub does not allow you to send custom headers with webhooks. See authentication docs for more detail

  3. Set the Content type to application/json

  4. Select the Let me select individual events radio button

  5. Uncheck all event types except Create – this should be the only one checked

  6. Click Add webhook

Delete an ingestion

Delete a single ingestion from the queue by ID, preventing that repo/tag combination from being ingested. This endpoint requires the ADMIN permission.

Request

Method DELETE
Path /v1/queue/:ingestion-id
(where :ingestion-id is the unique identifier for a Ingestion)
Headers
X-Api-Key
See authentication docs
X-Api-Secret
See authentication docs

Response

Status 204 on success
401 if authentication failed
403 if authorization failed
404 if an ingestion matching :ingestion-id does not exist
Headers
Content-Type
Empty on success
text/html on error
Body Empty

Example curl command:

curl -X DELETE \
	-H 'X-Api-Key: XXXXXX' -H 'X-Api-Secret: XXXXXX' \
	https://origami-repo-data.ft.com/v1/queue/XXXXXX