ArangoDB v3.4 reached End of Life (EOL) and is no longer supported.

This documentation is outdated. Please see the most recent version here: Latest Docs

HTTP Interface for Bulk Imports

ArangoDB provides an HTTP interface to import multiple documents at once into a collection. This is known as a bulk import.

The data uploaded must be provided in JSON format. There are two mechanisms to import the data:

  • self-contained JSON documents: in this case, each document contains all attribute names and values. Attribute names may be completely different among the documents uploaded
  • attribute names plus document data: in this case, the first array must contain the attribute names of the documents that follow. The following arrays containing only the attribute values. Attribute values will be mapped to the attribute names by positions.

The endpoint address is /_api/import for both input mechanisms. Data must be sent to this URL using an HTTP POST request. The data to import must be contained in the body of the POST request.

The collection query parameter must be used to specify the target collection for the import. Importing data into a non-existing collection will produce an error.

The waitForSync query parameter can be set to true to make the import only return if all documents have been synced to disk.

The complete query parameter can be set to true to make the entire import fail if any of the uploaded documents is invalid and cannot be imported. In this case, no documents will be imported by the import run, even if a failure happens at the end of the import.

If complete has a value other than true, valid documents will be imported while invalid documents will be rejected, meaning only some of the uploaded documents might have been imported.

The details query parameter can be set to true to make the import API return details about documents that could not be imported. If details is true, then the result will also contain a details attribute which is an array of detailed error messages. If the details is set to false or omitted, no details will be returned.

imports document values

imports documents from JSON-encoded lists

POST /_api/import

Query Parameters

  • collection (string, required): The collection name.

  • fromPrefix (string, optional): An optional prefix for the values in _from attributes. If specified, the value is automatically prepended to each _from input value. This allows specifying just the keys for _from.

  • toPrefix (string, optional): An optional prefix for the values in _to attributes. If specified, the value is automatically prepended to each _to input value. This allows specifying just the keys for _to.

  • overwrite (boolean, optional): If this parameter has a value of true or yes, then all data in the collection will be removed prior to the import. Note that any existing index definitions will be preseved.

  • waitForSync (boolean, optional): Wait until documents have been synced to disk before returning.

  • onDuplicate (string, optional): Controls what action is carried out in case of a unique key constraint violation. Possible values are:

    • error: this will not import the current document because of the unique key constraint violation. This is the default setting.

    • update: this will update an existing document in the database with the data specified in the request. Attributes of the existing document that are not present in the request will be preseved.

    • replace: this will replace an existing document in the database with the data specified in the request.

    • ignore: this will not update an existing document and simply ignore the error caused by the unique key constraint violation.

    Note that update, replace and ignore will only work when the import document in the request contains the _key attribute. update and replace may also fail because of secondary unique key constraint violations.

  • complete (boolean, optional): If set to true or yes, it will make the whole import fail if any error occurs. Otherwise the import will continue even if some documents cannot be imported.

  • details (boolean, optional): If set to true or yes, the result will include an attribute details with details about documents that could not be imported.

Request Body

(string, required)

The body must consist of JSON-encoded arrays of attribute values, with one line per document. The first row of the request must be a JSON-encoded array of attribute names. These attribute names are used for the data in the subsequent lines.

NOTE Swagger examples won’t work due to the anchor.

Creates documents in the collection identified by collection-name. The first line of the request body must contain a JSON-encoded array of attribute names. All following lines in the request body must contain JSON-encoded arrays of attribute values. Each line is interpreted as a separate document, and the values specified will be mapped to the array of attribute names specified in the first header line.

The response is a JSON object with the following attributes:

  • created: number of documents imported.

  • errors: number of documents that were not imported due to an error.

  • empty: number of empty lines found in the input (will only contain a value greater zero for types documents or auto).

  • updated: number of updated/replaced documents (in case onDuplicate was set to either update or replace).

  • ignored: number of failed but ignored insert operations (in case onDuplicate was set to ignore).

  • details: if query parameter details is set to true, the result will contain a details attribute which is an array with more detailed information about which documents could not be inserted.

Responses

HTTP 201: is returned if all documents could be imported successfully.

HTTP 400: is returned if type contains an invalid value, no collection is specified, the documents are incorrectly encoded, or the request is malformed.

HTTP 404: is returned if collection or the _from or _to attributes of an imported edge refer to an unknown collection.

HTTP 409: is returned if the import would trigger a unique key violation and complete is set to true.

HTTP 500: is returned if the server cannot auto-generate a document key (out of keys error) for a document with no user-defined key.

Examples

Importing two documents, with attributes _key, value1 and value2 each. One line in the import data is empty

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]

[ "foo", "bar", "baz" ]
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Importing into an edge collection, with attributes _from, _to and name

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links <<EOF
[ "_from", "_to", "name" ]
[ "products/123","products/234", "some name" ]
[ "products/332", "products/abc", "other name" ]
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Importing into an edge collection, omitting _from or _to

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&details=true <<EOF
[ "name" ]
[ "some name" ]
[ "other name" ]
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Violating a unique constraint, but allow partial imports

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&details=true <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
["abc", "bar", "baz" ]
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Violating a unique constraint, not allowing partial imports

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&complete=true <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
["abc", "bar", "baz" ]
EOF

HTTP/1.1 Conflict
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Using a non-existing collection

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
[ "_key", "value1", "value2" ]
[ "abc", 25, "test" ]
["foo", "bar", "baz" ]
EOF

HTTP/1.1 Not Found
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Using a malformed body

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products <<EOF
{ "_key": "foo", "value1": "bar" }
EOF

HTTP/1.1 Bad Request
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

imports documents from JSON

imports documents from JSON

POST /_api/import

Query Parameters

  • type (string, required): Determines how the body of the request will be interpreted. type can have the following values:
    • documents: when this type is used, each line in the request body is expected to be an individual JSON-encoded document. Multiple JSON objects in the request body need to be separated by newlines.
    • list: when this type is used, the request body must contain a single JSON-encoded array of individual objects to import.
    • auto: if set, this will automatically determine the body type (either documents or list).

  • collection (string, required): The collection name.

  • fromPrefix (string, optional): An optional prefix for the values in _from attributes. If specified, the value is automatically prepended to each _from input value. This allows specifying just the keys for _from.

  • toPrefix (string, optional): An optional prefix for the values in _to attributes. If specified, the value is automatically prepended to each _to input value. This allows specifying just the keys for _to.

  • overwrite (boolean, optional): If this parameter has a value of true or yes, then all data in the collection will be removed prior to the import. Note that any existing index definitions will be preseved.

  • waitForSync (boolean, optional): Wait until documents have been synced to disk before returning.

  • onDuplicate (string, optional): Controls what action is carried out in case of a unique key constraint violation. Possible values are:

    • error: this will not import the current document because of the unique key constraint violation. This is the default setting.

    • update: this will update an existing document in the database with the data specified in the request. Attributes of the existing document that are not present in the request will be preseved.

    • replace: this will replace an existing document in the database with the data specified in the request.

    • ignore: this will not update an existing document and simply ignore the error caused by a unique key constraint violation.

    Note that that update, replace and ignore will only work when the import document in the request contains the _key attribute. update and replace may also fail because of secondary unique key constraint violations.

  • complete (boolean, optional): If set to true or yes, it will make the whole import fail if any error occurs. Otherwise the import will continue even if some documents cannot be imported.

  • details (boolean, optional): If set to true or yes, the result will include an attribute details with details about documents that could not be imported.

Request Body

(string, required)

The body must either be a JSON-encoded array of objects or a string with multiple JSON objects separated by newlines.

NOTE Swagger examples won’t work due to the anchor.

Creates documents in the collection identified by collection-name. The JSON representations of the documents must be passed as the body of the POST request. The request body can either consist of multiple lines, with each line being a single stand-alone JSON object, or a singe JSON array with sub-objects.

The response is a JSON object with the following attributes:

  • created: number of documents imported.

  • errors: number of documents that were not imported due to an error.

  • empty: number of empty lines found in the input (will only contain a value greater zero for types documents or auto).

  • updated: number of updated/replaced documents (in case onDuplicate was set to either update or replace).

  • ignored: number of failed but ignored insert operations (in case onDuplicate was set to ignore).

  • details: if query parameter details is set to true, the result will contain a details attribute which is an array with more detailed information about which documents could not be inserted.

Responses

HTTP 201: is returned if all documents could be imported successfully.

HTTP 400: is returned if type contains an invalid value, no collection is specified, the documents are incorrectly encoded, or the request is malformed.

HTTP 404: is returned if collection or the _from or _to attributes of an imported edge refer to an unknown collection.

HTTP 409: is returned if the import would trigger a unique key violation and complete is set to true.

HTTP 500: is returned if the server cannot auto-generate a document key (out of keys error) for a document with no user-defined key.

Examples

Importing documents with heterogenous attributes from a JSON array

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF
[ 
  { 
    "_key" : "abc", 
    "value1" : 25, 
    "value2" : "test", 
    "allowed" : true 
  }, 
  { 
    "_key" : "foo", 
    "name" : "baz" 
  }, 
  { 
    "name" : { 
      "detailed" : "detailed name", 
      "short" : "short name" 
    } 
  } 
]
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Importing documents from individual JSON lines

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF
{ "_key": "abc", "value1": 25, "value2": "test","allowed": true }
{ "_key": "foo", "name": "baz" }

{ "name": { "detailed": "detailed name", "short": "short name" } }

EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Using the auto type detection

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=auto <<EOF
[ 
  { 
    "_key" : "abc", 
    "value1" : 25, 
    "value2" : "test", 
    "allowed" : true 
  }, 
  { 
    "_key" : "foo", 
    "name" : "baz" 
  }, 
  { 
    "name" : { 
      "detailed" : "detailed name", 
      "short" : "short name" 
    } 
  } 
]
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Importing into an edge collection, with attributes _from, _to and name

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=documents <<EOF
{ "_from": "products/123", "_to": "products/234" }
{"_from": "products/332", "_to": "products/abc",   "name": "other name" }
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Importing into an edge collection, omitting _from or _to

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=links&type=list&details=true <<EOF
[ 
  { 
    "name" : "some name" 
  } 
]
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Violating a unique constraint, but allow partial imports

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&details=true <<EOF
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Violating a unique constraint, not allowing partial imports

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents&complete=true <<EOF
{ "_key": "abc", "value1": 25, "value2": "test" }
{ "_key": "abc", "value1": "bar", "value2": "baz" }
EOF

HTTP/1.1 Conflict
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Using a non-existing collection

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=documents <<EOF
{ "name": "test" }
EOF

HTTP/1.1 Not Found
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Using a malformed body

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/import?collection=products&type=list <<EOF
{ }
EOF

HTTP/1.1 Bad Request
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body
❮  Async Result Handling

© ArangoDB - the native multi-model NoSQL database