Ingest by WebDAV

Ingest by WebDAV is currently the preferred way to ingest data into an OCFL repository.

Contents

Prequisites

WebDAV Clients

Choose and install one of the following WebDAV clients:

• Oxygen XML Editor

WebDAV Credentials

Prepare your WebDAV credentials:

1. Go to /account/resetPassword.
2. Login with your home institution's credentials.
3. Set the WebDAV credentials, respectively, the so called API password.
4. Remember the username shown on this page, which looks like an email address, the so called API username.

WebDAV Connection

Establish a WebDAV connection with Oxygen:

1. Open the Data Source Explorer view.
2. Right click on Connections.
3. Choose Configure Database Sources....
4. In section Connections, click the plus sign to start a new connection.
5. Choose a name for your connection, e.g., lac_prod or lac_test.
6. Choose WebDAV (S)FTP as Data Source.
7. Type in the WebDAV URL of your OCFL repository, e.g. https://dav.ka3.uni-koeln.de/lac/. Make sure that the URL ends with a slash. See WebDAV endpoints below for a complete list of available WebDAV URLs.
8. Type in the API username mentioned above.
9. Type in the API password mentioned above.

WebDAV Endpoints

This server configures the following WebDAV endpoints, each endpoint targeting to an OCFL repository:

• https://dav.ka3.uni-koeln.de/a5-1/
• https://dav.ka3.uni-koeln.de/a5-2/
• https://dav.ka3.uni-koeln.de/a5-3/
• https://dav.ka3.uni-koeln.de/lac/

WebDAV Directory

Each WebDAV endpoint targets to the following basic WebDAV directory:

{repository}
├── cache
├── config
├── cloud
├── data
└── log

The cache, config, and log directories contain files not relevant for the WebDAV ingest. The cloud and data directories are the relevant ones, where data contains the target OCFL directory and cloud is a mounted cloud directory containing the source files.

Introduction

Collaborative Cloud Authoring

The cloud storage is the place where authoring of research data takes place in collaboration between researchers and research data managers. The following cloud storage is configured for this server:

Host:

https://uni-koeln.sciebo.de

Data Box:

rrzk-ka3.pbox@uni-koeln.de

In order to get access to the data box, contact the cloud storage administrators.

Since fixity is crucial for research data management, it is highly recommended to use one of the desktop clients for data authoring since, in contrast to the web clients, they compute digests for each file upload.

Data Ingest by Research Data Managers

Ingest is done by copying files and directories from a cloud directory into an OCFL content directory with a WebDAV client. See a detailed description of all WebDAV functions in section OCFL ingest below.

It is best practice to make a cloud directory readonly for all researchers and research data managers as soon as a data bundle is considered to be ready for OCFL ingest into a production environment. There may be inconsistencies otherwise if data change in the cloud directory during OCFL ingest.

The WebDAV interface keeps the OCFL structure consistent. It does not allow any copying of data that would confuse the OCFL directory. See WebDAV API for a technical description.

As soon as data are copied, they are in an archived state but are not yet available via REST APIs. Reindex an OCFL repository with the Ingest API in order to make ingested data available via REST APIs.

OCFL Ingest

Create a new OCFL Object

Create two directories:

• Create an OCFL object path directory directly under the OCFL storage root directory: /data/{objectPath}.
• Create an OCFL object directory directly under the OCFL object path directory: /data/{objectPath}/{object}.

Creating the second directory will automatically trigger the creation of a new OCFL object including OCFL inventory files, inventory digest files, a first version directory, and so on. Deeper nested directory structures are not supported. The second directory is always expected to be the OCFL object directory.

Choose the names of the new directories carefully because, at least in production environments, they should not be renamed at a later time. Special chars such as german umlauts are supported but are not recommended.

Delete an OCFL Object

The WebDAV API supports deletion of OCFL objects.

• Delete a single OCFL object: /data/{objectPath}/{object}.
• Delete multiple OCFL objects: /data/{objectPath}.

OCFL object deletion is a convenience feature for test and development environments, where data collections are prepared for production.

It is not recommended to delete OCFL objects in production environments.

Add OCFL Content Files

Adding OCFL content files is done by moving files from the cloud directory to an OCFL content directory with drag-and-drop.

• Select one or more cloud files and cloud directories.
• Drag-and-drop the selected files and directories into the latest version of an OCFL content directory.
• (Reindex the OCFL repository in order to make data available via REST APIs)

As soon as the copying of the cloud files has started, the whole OCFL repository is locked in order to keep the OCFL structure consistent. Files are copied with guarantee of fixity. Copied files are registered in the OCFL inventory of an OCFL object. If the file already exists in a preceding OCFL version, the file is deduplicated and only a reference is stored in the OCFL inventory. The WebDAV interface makes sure that files can be copied only into the latest version of an OCFL object. Older versions are immutable.

It is not possible to author OCFL content files. In order to change OCFL content files, edit them with the cloud storage desktop client and repeat the OCFL ingest by means of the WebDAV client.

Delete OCFL Content Files

OCFL content files can be deleted in the following way:

• Select one or more OCFL content files or OCFL content directories.
• Delete the selected files and directories.
• (Reindex the OCFL repository in order to make the deletions effective for REST APIs)

The WebDAV interface makes sure that only files of the latest version of an OCFL object can be deleted. Older versions are immutable. If the file to be deleted is a deduplicated file, only the reference to the file is deleted in the OCFL inventory. The physical file is also deleted if it is not a deduplicated file.

Start a new OCFL Version

Start a new OCFL version in the following way:

• Select an OCFL object directory /data/{objectPath}/{object}.
• Create a new directory /data/{objectPath}/{object}/v{number} below the OCFL object directory, where {number} is higher by one than the latest OCFL version of the OCFL object.

OCFL version directories always start with a v.

The new OCFL version by default contains all OCFL content files of its preceding OCFL version. All OCFL content files in the new OCFL version are deduplicated.

Delete the latest OCFL Version

Delete the latest OCFL version in the following way:

• Select the latest OCFL version directory /data/{objectPath}/{object}/v{number}.
• Delete the selected directory.

The WebDAV interface makes sure that only the latest version of an OCFL object can be deleted.

ACLs

Edit the Default ACL

Edit the default ACL of an OCFL repository in the following way:

1. Open the defaul ACL file /data/acl.json.
2. Edit the ACL file according to File System ACLs.
3. (Reindex the OCFL repository in order to make the ACL effective for REST API access.)

The default ACL has effect on all OCFL content files of all versions of all OCFL objects of an OCFL repository.

Edit an OCFL Object's ACL

Edit an OCFL object's ACL in the following way:

1. Open the OCFL Object ACL file /data/{objectPath}/{object}/acl.json.
2. Edit the ACL file according to File System ACLs.
3. (Reindex the OCFL repository in order to make the ACL effective for REST API access.)

An OCFL object ACL has effect on all OCFL content files of all versions of an OCFL object. It overwrites the default ACL.