With the 2022.4 release we set the limits for all of the endpoints to 600 calls per 10 minutes for CDMS API. In the upcoming release (2022.5) we will start to enforce them.
Questions
What exactly is covered by a data request? We request the entire study per endpoint. Is that 1 request (per endpoint), or is it the number of pages within the request?
It is the number of pages in the request. So when sending a request to an endpoint, that will count as 1 request. When getting the next page, there will be another request. So if there are, for example, 10 pages, that will count as 10 requests.
What are the consequences if you go over the rate limit? What response do we get back? For example error message, delayed response?
When the rate limit is exceeded, an error response will be returned, with a 429 status code. The response headers will also contain information about the rate limit, e.g, the time until the next request can be made. Every request to the API (even when the rate limit is not exceeded) will contain information about the rate limit, e.g., the number of requests remaining before the limit is reached, the amount of time left before the current limit, etc. So when the rate limit is reached and the error response is shown, the user will just need to wait for the amount of time specified in the response header before continuing with the API calls.
How are the 10 minutes calculated exactly?
It is currently a fixed window policy, which means x amount of API calls within a fixed 10-minute window. If the first request happens at 09:00, then the rate is set until 09:10. So if all the API requests are used up in the first 1 minute, the user must wait 9 minutes for the next request.
A request only returns X results per page. What are my options for retrieving larger datasets if the number of requests is limited?
page_size can be passed by the client as a parameter to increase the amount of results per page, like so:
/api/study/{{study_id}}/data-points/study?page_size=1000
A max_page_size is configured for each endpoint, and requesting a page_size larger than it will return an informative error:
Page size is out of range, maximum page size is 5000
How are the rates tracked/grouped? For example, per study, per IP address, per user account?
It is per user, per endpoint. For example, if a rate limit is set to 600 requests for /api/study, then every user can send 600 requests in 10 minutes to that specific endpoint. But that limit won't count towards the limit for other endpoints for the same user.