Firehose Authentication and CDN Endpoints

Each Firehose feed is served by two endpoints:

• The authentication endpoint checks your API key. It returns no data, only a link to the CDN endpoint.
• The CDN endpoint serves the pages of data.

The endpoints listed in Accessing the Firehose are authentication endpoints. They are the only URLs you need to know. Every response contains a next URL, and following it always takes you to the right place.

Danger

A page with zero items does not mean you have reached the end of the feed.

You have reached the end of the feed only when a page has zero items and its next URL is the same as the URL you just requested.

The Two Endpoints

	Authentication endpoint	CDN endpoint
URL starts with	https://firehose.imin.co/firehose/	https://firehose-cdn.imin.co/firehose-cdn/
What it returns	Zero items, and a next URL for the CDN endpoint	Pages of items
How you are authorised	Your API key, in the X-API-KEY request header	A token in the jwt query parameter. The authentication endpoint puts this in the next URL for you

Following a Feed

1. Request the authentication endpoint

GET https://firehose.imin.co/firehose/standard/v2/session-series
X-API-KEY: <your API key>

{
  "next": "https://firehose-cdn.imin.co/firehose-cdn/standard/v2/session-series?jwt=eyJhbGciOi...",
  "items": []
}

This page has zero items, but it is not the end of the feed. The next URL is different from the URL you requested, so follow it straight away.

2. Follow next to the CDN endpoint

GET https://firehose-cdn.imin.co/firehose-cdn/standard/v2/session-series?jwt=eyJhbGciOi...

{
  "next": "https://firehose-cdn.imin.co/firehose-cdn/standard/v2/session-series?jwt=eyJhbGciOi...&afterChangeNumber=21349940161",
  "items": [
    {
      "state": "updated",
      "kind": "SessionSeries",
      "id": "...",
      "modified": 21343296230,
      "data": { "@type": "SessionSeries", "...": "..." }
    }
  ]
}

Process the items, then follow next again. Keep doing this for as long as pages contain items.

3. Reach the end of the feed

GET https://firehose-cdn.imin.co/firehose-cdn/standard/v2/session-series?jwt=eyJhbGciOi...&afterChangeNumber=21349940161

{
  "next": "https://firehose-cdn.imin.co/firehose-cdn/standard/v2/session-series?jwt=eyJhbGciOi...&afterChangeNumber=21349940161",
  "items": []
}

This page has zero items and its next URL is the same as the URL you requested. This is the end of the feed for now. Wait at least 8 seconds, then request the same URL again to check for new items. See Rate Limits.

4. The token expires

The token in the CDN URL expires regularly. When it has expired, the CDN endpoint returns zero items and a next URL for the authentication endpoint:

{
  "next": "https://firehose.imin.co/firehose/standard/v2/session-series?afterChangeNumber=21349940161",
  "items": []
}

This is not the end of the feed either. Follow next straight away, with your API key. The authentication endpoint returns another zero-item page, with a next URL containing a new token. Follow that and you carry on from where you were.

The afterChangeNumber parameter keeps your place in the feed throughout, so this is not a resync.

Note

When a token expires you will receive two zero-item pages in a row before you receive items again. Neither is the end of the feed.

How to Read a Page

Items	next URL	What it means	What to do
One or more	Anything	There is more data	Process the items, then follow next straight away
Zero	Different from the URL you requested	You are being passed between the authentication endpoint and the CDN endpoint	Follow next straight away
Zero	The same as the URL you requested	You have reached the end of the feed for now	Wait at least 8 seconds, then request the same URL again

What Your Client Needs to Do

• Decide whether you have reached the end of the feed by comparing the next URL with the URL you requested. Never decide by counting items.
• Follow next exactly as it is returned. Do not build CDN URLs yourself, and do not change or remove query parameters.
• Send the X-API-KEY header to the authentication endpoint. The CDN endpoint ignores the header, so it does not matter if your client sends it there too, but we recommend that you do not send your key to the CDN endpoint.
• Store the most recent next URL as your place in the feed. If its token has expired by the time you use it, the CDN endpoint passes you back through the authentication endpoint and you continue from the same place.
• Update your stored next URL only after you have finished processing a page. If your client stops part-way through a page, it then requests that page again when it restarts.
• Keep CDN URLs private, as you would your API key. The token in the URL gives access to the feed until it expires.

Errors

The authentication endpoint returns these errors:

Status	Meaning
400	afterChangeNumber is not a number
401	The X-API-KEY header is missing
403	The API key is not valid
404	The feed type at the end of the URL is not recognised

The CDN endpoint does not return 401 or 403. If the token is missing, expired or invalid, it returns the zero-item page described in The token expires.