Accessing Cursors via HTTP

Create cursor

create a cursor and return the first results

POST /_api/cursor

A JSON object describing the query and query parameters.

Request Body

  • query (string, required): contains the query string to be executed

  • count (boolean, optional): indicates whether the number of documents in the result set should be returned in the “count” attribute of the result. Calculating the “count” attribute might have a performance impact for some queries in the future so this option is turned off by default, and “count” is only returned when requested.

  • batchSize (integer, optional): maximum number of result documents to be transferred from the server to the client in one roundtrip. If this attribute is not set, a server-controlled default value will be used. A batchSize value of 0 is disallowed.

  • ttl (integer, optional): The time-to-live for the cursor (in seconds). If the result set is small enough (less than or equal to batchSize) then results are returned right away. Otherwise they are stored in memory and will be accessible via the cursor with respect to the ttl. The cursor will be removed on the server automatically after the specified amount of time. This is useful to ensure garbage collection of cursors that are not fully fetched by clients. If not set, a server-defined value will be used (default: 30 seconds).

  • cache (boolean, optional): flag to determine whether the AQL query results cache shall be used. If set to false, then any query cache lookup will be skipped for the query. If set to true, it will lead to the query cache being checked for the query if the query cache mode is either on or demand.

  • memoryLimit (integer, optional): the maximum number of memory (measured in bytes) that the query is allowed to use. If set, then the query will fail with error “resource limit exceeded” in case it allocates too much memory. A value of 0 indicates that there is no memory limit.

  • bindVars (array of objects, optional): key/value pairs representing the bind parameters.

  • options (object, optional): key/value object with extra options for the query.

    • fullCount (boolean, optional): if set to true and the query contains a LIMIT clause, then the result will have an extra attribute with the sub-attributes stats and fullCount, { ... , "extra": { "stats": { "fullCount": 123 } } }. The fullCount attribute will contain the number of documents in the result before the last top-level LIMIT in the query was applied. It can be used to count the number of documents that match certain filter criteria, but only return a subset of them, in one go. It is thus similar to MySQL’s SQL_CALC_FOUND_ROWS hint. Note that setting the option will disable a few LIMIT optimizations and may lead to more documents being processed, and thus make queries run longer. Note that the fullCount attribute may only be present in the result if the query has a top-level LIMIT clause and the LIMIT clause is actually used in the query.

    • fillBlockCache (boolean, optional): if set to true or not specified, this will make the query store the data it reads via the RocksDB storage engine in the RocksDB block cache. This is usually the desired behavior. The option can be set to false for queries that are known to either read a lot of data which would thrash the block cache, or for queries that read data which are known to be outside of the hot set. By setting the option to false, data read by the query will not make it into the RocksDB block cache if not already in there, thus leaving more room for the actual hot set.

    • maxPlans (integer, optional): Limits the maximum number of plans that are created by the AQL query optimizer.

    • maxNodesPerCallstack (integer, optional): The number of execution nodes in the query plan after that stack splitting is performed to avoid a potential stack overflow. Defaults to the configured value of the startup option --query.max-nodes-per-callstack.

      This option is only useful for testing and debugging and normally does not need any adjustment.

    • maxWarningCount (integer, optional): Limits the maximum number of warnings a query will return. The number of warnings a query will return is limited to 10 by default, but that number can be increased or decreased by setting this attribute.

    • failOnWarning (boolean, optional): When set to true, the query will throw an exception and abort instead of producing a warning. This option should be used during development to catch potential issues early. When the attribute is set to false, warnings will not be propagated to exceptions and will be returned with the query result. There is also a server configuration option --query.fail-on-warning for setting the default value for failOnWarning so it does not need to be set on a per-query level.

    • stream (boolean, optional): Can be enabled to execute the query lazily. If set to true, then the query is executed as long as necessary to produce up to batchSize results. These results are returned immediately and the query is suspended until the client asks for the next batch (if there are more results). Depending on the query this can mean that the first results will be available much faster and that less memory is needed because the server only needs to store a subset of results at a time. Read-only queries can benefit the most, unless SORT without index or COLLECT are involved that make it necessary to process all documents before a partial result can be returned. It is advisable to only use this option for queries without exclusive locks.

      Remarks:

      • The query will hold resources until it ends (such as RocksDB snapshots, which prevents compaction to some degree). Writes will be in memory until the query is committed.
      • If existing documents are modified, then write locks are held on these documents and other queries trying to modify the same documents will fail because of this conflict.
      • A streaming query may fail late because of a conflict or for other reasons after some batches were already returned successfully, possibly rendering the results up to that point meaningless.
      • The query options cache, count and fullCount are not supported for streaming queries.
      • Query statistics, profiling data and warnings are delivered as part of the last batch.

      If the stream option is false (default), then the complete result of the query is calculated before any of it is returned to the client. The server stores the full result in memory (on the contacted Coordinator if in a cluster). All other resources are freed immediately (locks, RocksDB snapshots). The query will fail before it returns results in case of a conflict.

    • optimizer (object, optional): Options related to the query optimizer.

      • rules (array of strings, optional): A list of to-be-included or to-be-excluded optimizer rules can be put into this attribute, telling the optimizer to include or exclude specific rules. To disable a rule, prefix its name with a -, to enable a rule, prefix it with a +. There is also a pseudo-rule all, which matches all optimizer rules. -all disables all rules.

    • profile (integer, optional): If set to true or 1, then the additional query profiling information will be returned in the sub-attribute profile of the extra return attribute, if the query result is not served from the query cache. Set to 2 the query will include execution stats per query plan node in sub-attribute stats.nodes of the extra return attribute. Additionally the query plan is returned in the sub-attribute extra.plan.

    • satelliteSyncWait (number, optional): This Enterprise Edition parameter allows to configure how long a DB-Server will have time to bring the SatelliteCollections involved in the query into sync. The default value is 60.0 (seconds). When the max time has been reached the query will be stopped.

    • maxRuntime (number, optional): The query has to be executed within the given runtime or it will be killed. The value is specified in seconds. The default value is 0.0 (no timeout).

    • maxTransactionSize (integer, optional): Transaction size limit in bytes.

    • intermediateCommitSize (integer, optional): Maximum total size of operations after which an intermediate commit is performed automatically.

    • intermediateCommitCount (integer, optional): Maximum number of operations after which an intermediate commit is performed automatically.

    • skipInaccessibleCollections (boolean, optional): AQL queries (especially graph traversals) will treat collection to which a user has no access rights as if these collections were empty. Instead of returning a forbidden access error, your queries will execute normally. This is intended to help with certain use-cases: A graph contains several collections and different users execute AQL queries on that graph. You can now naturally limit the accessible results by changing the access rights of users on collections. This feature is only available in the Enterprise Edition.

The query details include the query string plus optional query options and bind parameters. These values need to be passed in a JSON representation in the body of the POST request.

Responses

HTTP 201: is returned if the result set can be created by the server.

  • error (boolean): A flag to indicate that an error occurred (false in this case)

  • code (integer): the HTTP status code

  • result (array): an array of result documents (might be empty if query has no results)

  • hasMore (boolean): A boolean indicator whether there are more results available for the cursor on the server

  • count (integer): the total number of result documents available (only available if the query was executed with the count attribute set)

  • id (string): id of temporary cursor created on the server (optional, see above)

  • extra (object): An optional JSON object with extra information about the query result.

    • stats (object): An object with query statistics.

      • writesExecuted (number): The total number of data-modification operations successfully executed.

      • writesIgnored (number): The total number of data-modification operations that were unsuccessful, but have been ignored because of query option ignoreErrors.

      • scannedFull (number): The total number of documents iterated over when scanning a collection without an index. Documents scanned by subqueries will be included in the result, but operations triggered by built-in or user-defined AQL functions will not.

      • scannedIndex (number): The total number of documents iterated over when scanning a collection using an index. Documents scanned by subqueries will be included in the result, but operations triggered by built-in or user-defined AQL functions will not.

      • cursorsCreated (number): The total number of cursor objects created during query execution. Cursor objects are created for index lookups.

      • cursorsRearmed (number): The total number of times an existing cursor object was repurposed. Repurposing an existing cursor object is normally more efficient compared to destroying an existing cursor object and creating a new one from scratch.

      • cacheHits (number): The total number of index entries read from in-memory caches for indexes of type edge or persistent. This value will only be non-zero when reading from indexes that have an in-memory cache enabled, and when the query allows using the in-memory cache (i.e. using equality lookups on all index attributes).

      • cacheMisses (number): The total number of cache read attempts for index entries that could not be served from in-memory caches for indexes of type edge or persistent. This value will only be non-zero when reading from indexes that have an in-memory cache enabled, the query allows using the in-memory cache (i.e. using equality lookups on all index attributes) and the looked up values are not present in the cache.

      • filtered (number): The total number of documents that were removed after executing a filter condition in a FilterNode or another node that post-filters data. Note that IndexNodes can also filter documents by selecting only the required index range from a collection, and the filtered value only indicates how much filtering was done by a post filter in the IndexNode itself or following FilterNodes. EnumerateCollectionNodes and TraversalNodes can also apply filter conditions and can reported the number of filtered documents.

      • fullCount (number): The total number of documents that matched the search condition if the query’s final top-level LIMIT statement were not present. This attribute may only be returned if the fullCount option was set when starting the query and will only contain a sensible value if the query contained a LIMIT operation on the top level.

      • peakMemoryUsage (number): The maximum memory usage of the query while it was running. In a cluster, the memory accounting is done per shard, and the memory usage reported is the peak memory usage value from the individual shards. Note that to keep things lightweight, the per-query memory usage is tracked on a relatively high level, not including any memory allocator overhead nor any memory used for temporary results calculations (e.g. memory allocated/deallocated inside AQL expressions and function calls).

      • nodes (number): When the query was executed with the profile option set to at least 2, then this value contains runtime statistics per query execution node. This field contains the node id (in id), the number of calls to this node (calls), and the number of items returned by this node (items). Items are the temporary results returned at this stage. You can correlate this statistics with the plan returned in extra. For a human readable output you can execute db._profileQuery(<query>, <bind-vars>) in arangosh.

  • cached (boolean): a boolean flag indicating whether the query result was served from the query cache or not. If the query result is served from the query cache, the extra return attribute will not contain any stats sub-attribute and no profile sub-attribute.

HTTP 400: is returned if the JSON representation is malformed or the query specification is missing from the request.

If the JSON representation is malformed or the query specification is missing from the request, the server will respond with HTTP 400.

The body of the response will contain a JSON object with additional error details. The object has the following attributes:

  • error (boolean): boolean flag to indicate that an error occurred (true in this case)

  • code (integer): the HTTP status code

  • errorNum (integer): the server error number

  • errorMessage (string): a descriptive error message
    If the query specification is complete, the server will process the query. If an error occurs during query processing, the server will respond with HTTP 400. Again, the body of the response will contain details about the error.

HTTP 404: The server will respond with HTTP 404 in case a non-existing collection is accessed in the query.

HTTP 405: The server will respond with HTTP 405 if an unsupported HTTP method is used.

HTTP 410: The server will respond with HTTP 410 if a server which processes the query or is the leader for a shard which is used in the query stops responding, but the connection has not been closed.

HTTP 503: The server will respond with HTTP 503 if a server which processes the query or is the leader for a shard which is used in the query is down, either for going through a restart, a failure or connectivity issues.

Examples

Execute a query and extract the result in a single go

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 2 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 495
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Execute a query and extract a part of the result

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 5 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 507
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Using the query option “fullCount”

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR i IN 1..1000 FILTER i > 500 LIMIT 10 RETURN i", 
  "count" : true, 
  "options" : { 
    "fullCount" : true 
  } 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 401
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Enabling and disabling optimizer rules

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR i IN 1..10 LET a = 1 LET b = 2 FILTER a + b == 3 RETURN i", 
  "count" : true, 
  "options" : { 
    "maxPlans" : 1, 
    "optimizer" : { 
      "rules" : [ 
        "-all", 
        "+remove-unnecessary-filters" 
      ] 
    } 
  } 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 358
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Execute instrumented query and return result together with execution plan and profiling information

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "LET s = SLEEP(0.25) LET t = SLEEP(0.5) RETURN 1", 
  "count" : true, 
  "options" : { 
    "profile" : 2 
  } 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 2924
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Execute a data-modification query and retrieve the number of modified documents

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products REMOVE p IN products" 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 329
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Execute a data-modification query with option ignoreErrors

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "REMOVE 'bar' IN products OPTIONS { ignoreErrors: true }" 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 328
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Bad query - Missing body

shell> curl -X POST --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor

HTTP/1.1 400 Bad Request
content-type: application/json
connection: Keep-Alive
content-length: 73
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Bad query - Unknown collection

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR u IN unknowncoll LIMIT 2 RETURN u", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 404 Not Found
content-type: application/json
connection: Keep-Alive
content-length: 121
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Bad query - Execute a data-modification query that attempts to remove a non-existing document

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "REMOVE 'foo' IN products" 
}
EOF

HTTP/1.1 404 Not Found
content-type: application/json
connection: Keep-Alive
content-length: 100
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

return the next results from an existing cursor

POST /_api/cursor/{cursor-identifier}

Path Parameters

  • cursor-identifier (string, required): The name of the cursor

If the cursor is still alive, returns an object with the following attributes:

  • id: a cursor-identifier
  • result: a list of documents for the current batch
  • hasMore: false if this was the last batch
  • count: if present the total number of elements
  • code: an HTTP status code
  • error: a boolean flag to indicate whether an error occurred
  • errorNum: a server error number (if error is true)
  • errorMessage: a descriptive error message (if error is true)
  • extra: an object with additional information about the query result, with the nested objects stats and warnings. Only delivered as part of the last batch in case of a cursor with the stream option enabled.

Note that even if hasMore returns true, the next call might still return no documents. If, however, hasMore is false, then the cursor is exhausted. Once the hasMore attribute has a value of false, the client can stop.

Responses

HTTP 200: The server will respond with HTTP 200 in case of success.

HTTP 400: If the cursor identifier is omitted, the server will respond with HTTP 404.

HTTP 404: If no cursor with the specified identifier can be found, the server will respond with HTTP 404.

HTTP 410: The server will respond with HTTP 410 if a server which processes the query or is the leader for a shard which is used in the query stops responding, but the connection has not been closed.

HTTP 503: The server will respond with HTTP 503 if a server which processes the query or is the leader for a shard which is used in the query is down, either for going through a restart, a failure or connectivity issues.

Examples

Valid request for next batch

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 5 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

shell> curl -X POST --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/67441

HTTP/1.1 200 OK
content-type: application/json
connection: Keep-Alive
content-length: 507
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Missing identifier

shell> curl -X POST --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor

HTTP/1.1 400 Bad Request
content-type: application/json
connection: Keep-Alive
content-length: 73
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Unknown identifier

shell> curl -X POST --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/123123

HTTP/1.1 404 Not Found
content-type: application/json
connection: Keep-Alive
content-length: 75
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

return the next results from an existing cursor

PUT /_api/cursor/{cursor-identifier}

This endpoint is deprecated in favor its functionally equivalent POST counterpart.

Path Parameters

  • cursor-identifier (string, required): The name of the cursor

If the cursor is still alive, returns an object with the following attributes:

  • id: a cursor-identifier
  • result: a list of documents for the current batch
  • hasMore: false if this was the last batch
  • count: if present the total number of elements
  • code: an HTTP status code
  • error: a boolean flag to indicate whether an error occurred
  • errorNum: a server error number (if error is true)
  • errorMessage: a descriptive error message (if error is true)
  • extra: an object with additional information about the query result, with the nested objects stats and warnings. Only delivered as part of the last batch in case of a cursor with the stream option enabled.

Note that even if hasMore returns true, the next call might still return no documents. If, however, hasMore is false, then the cursor is exhausted. Once the hasMore attribute has a value of false, the client can stop.

Responses

HTTP 200: The server will respond with HTTP 200 in case of success.

HTTP 400: If the cursor identifier is omitted, the server will respond with HTTP 404.

HTTP 404: If no cursor with the specified identifier can be found, the server will respond with HTTP 404.

HTTP 410: The server will respond with HTTP 410 if a server which processes the query or is the leader for a shard which is used in the query stops responding, but the connection has not been closed.

HTTP 503: The server will respond with HTTP 503 if a server which processes the query or is the leader for a shard which is used in the query is down, either for going through a restart, a failure or connectivity issues.

Examples

Valid request for next batch

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 5 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

shell> curl -X PUT --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/67416

HTTP/1.1 200 OK
content-type: application/json
connection: Keep-Alive
content-length: 507
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Missing identifier

shell> curl -X PUT --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor

HTTP/1.1 400 Bad Request
content-type: application/json
connection: Keep-Alive
content-length: 97
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Unknown identifier

shell> curl -X PUT --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/123123

HTTP/1.1 404 Not Found
content-type: application/json
connection: Keep-Alive
content-length: 75
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
Show response body

Delete cursor

dispose an existing cursor

DELETE /_api/cursor/{cursor-identifier}

Path Parameters

  • cursor-identifier (string, required): The id of the cursor

Deletes the cursor and frees the resources associated with it.

The cursor will automatically be destroyed on the server when the client has retrieved all documents from it. The client can also explicitly destroy the cursor at any earlier time using an HTTP DELETE request. The cursor id must be included as part of the URL.

Note: the server will also destroy abandoned cursors automatically after a certain server-controlled timeout to avoid resource leakage.

Responses

HTTP 202: is returned if the server is aware of the cursor.

HTTP 404: is returned if the server is not aware of the cursor. It is also returned if a cursor is used after it has been destroyed.

Examples

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 5 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 201 Created
content-type: application/json
connection: Keep-Alive
content-length: 507
server: ArangoDB
x-arango-queue-time-seconds: 0.000000
x-content-type-options: nosniff
shell> curl -X DELETE --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/67363
Show response body
❮  Query Results

© ArangoDB - the native multi-model NoSQL database