Bulk Data Export in the Epic R4 Gateway

You can use the bulk export process to initially populate data stores for Epic Locations, Patients and Practitioners. This is based on the standard as described here. This feature is used for requesting a bulk data export from Epic, querying the status of the progress of the bulk export process and downloading the exported files.

Setup

1. Use an InteropiO Epic R4 Gateway and copy the base FHIR URL from the Details tab on the Gateway page.
2. Place the identifiers for the resources to be exported, into text format files and upload them to the AWS S3 bucket at location /<Site identifier>/<Resource Type>/<Resource Type>.csv. For example the Patient ids will need to be placed in /<Site identifier>/Patient/Patient.csv. The <SiteIdentifer> is another name for the gatewayID.

The bulk export process will go through the identifiers and for each one, retrieve the resource from Epic. The retrieved resources will be put in an ndjson file. Any errors will be placed in another ndjson file. Each resource file will hold a maximum of 100 resources and the bulk export process will produce multiple resource ndjson files if needed. 

Bulk Export Kickoff

Start the bulk export process by issuing an http GET call to

[gateway base fhir url]/<Resource Type>/$export 

along with the header Prefer with a value respond-async to tell the server to process this request in a background process. The response will have an http status of 202 and a Content-Location header containing the status polling location url.

Bulk data status request

The status of the bulk export request can be retrieved by sending an http GET request to the polling location url.

In-Progress status

If the bulk export process is still in progress, the http response will have a status of 202 and will include a Retry-After header that indicates how many seconds to wait before submitting the next status request. A subsequent GET operation will have a larger value for this header. You should not send a status request before the number of seconds specified in this header have elapsed. An X-Progress header will tell you how many ids have been processed.

Complete status

When the bulk export process has finished, a GET request to the polling location url will return an http status of 200. An Expires header will tell you the time till when the output files will be available for download. The response body will be in json format and will contain an array called output with url(s) for the resource files. These files can be downloaded using the url(s). Any errors found during the process will be listed in a separate array called error and also will be url(s) for files containing details for the errors. Both these files will be in ndjson format.

Error status  

If the bulk export process cannot proceed because of an error it will terminate. A GET request to the polling location url will return an http status of 500. The response body will contain an OperationOutcome with details of the error. One such error condition could be if the file containing the resource identifiers was not found.

Bulk data cancel request

You can cancel an existing bulk export request by sending an http DELETE request to the status polling url. The response will have an http status of 202. You will not be able to see the status for a bulk export request that has been cancelled and such a request will return an error status.