icatdump

Synopsis

icatdump [standard options] [-o FILE] [-f FORMAT]

Description

This script queries the content from an ICAT server and serializes it into a flat file. The format of that file depends on the version of the ICAT server and the backend that can be selected with the --format option.

Options

The configuration options may be set in the command line or in a configuration file. Some options may also be set in the environment.

Specific Options

The following options are specific to icatdump:

-o FILE, --outputfile FILE

Set the output file name. If the value - is used, the output will be written to standard output. This is also the default.

-f FORMAT, --format FORMAT

Select the backend to use and thus the output file format. XML and YAML backends are available.

Standard Options

The following options needed to connect the ICAT service are common for most python-icat scripts:

-h, --help

Display a help message and exit.

-c CONFIGFILE, --configfile CONFIGFILE

Name of a configuration file.

-s SECTION, --configsection SECTION

Name of a section in the configuration file. If set, the values in this configuration section will be applied to define other options.

-w URL, --url URL

URL of the ICAT server. This should point to the web service descriptions. If the URL has no path component, a default path will be added.

--no-check-certificate

Do not verify the ICAT server’s TLS certificate. This is only relevant if the URL set with --url uses HTTPS. It is mostly only useful for connecting a test server that does not have a trusted certificate.

--http-proxy HTTP_PROXY

Proxy to use for http requests.

--https-proxy HTTPS_PROXY

Proxy to use for https requests.

--no-proxy NO_PROXY

Comma separated list of exclusions for proxy use.

-a AUTH, --auth AUTH

Name of the authentication plugin to use for login to the ICAT server.

-u USERNAME, --user USERNAME

The ICAT user name.

-p PASSWORD, --pass PASSWORD

The user’s password. Will prompt for the password if not set.

-P, --prompt-pass

Prompt for the password. This is mostly useful to override a password set in the configuration file.

Known Issues and Limitations

  • IDS is not supported: the script only dumps the meta data stored in the ICAT, not the content of the files stored in the IDS.

  • The output will only contain objects that the user connecting ICAT has read permissions for. The script may need to connect as the ICAT root user in order to get the full content.

  • The following items are deliberately not included in the output:

    • Log objects (ICAT server versions older then 4.7.0),

    • The attributes id, createId, createTime, modId, and modTime of any object.

  • It is assumed that for each Dataset ds in the ICAT where ds.sample is not NULL, the condition ds.investigation = ds.sample.investigation holds. If this is not satisfied, the script will fail with a DataConsistencyError.

  • The partition of the data into chunks is static. It should rather be dynamic, e.g. chunks should be splitted if the number of objects in them grows too large.

  • The content in the ICAT server must not be modified while this script is retrieving it. Otherwise the script may fail or the dumpfile be inconsistent.

  • The script fails if the ICAT server is older then 4.6.0 and the data contains any Study. This is a bug in icat.server.

Environment Variables

ICAT_CFG

Name of a configuration file, see --configfile.

ICAT_CFG_SECTION

Name of a section in the configuration file, see --configsection.

ICAT_SERVICE

URL of the ICAT server, see --url.

http_proxy

Proxy to use for http requests, see --http-proxy.

https_proxy

Proxy to use for https requests, see --https-proxy.

no_proxy

Exclusions for proxy use, see --no-proxy.

ICAT_AUTH

Name of the authentication plugin, see --auth.

ICAT_USER

ICAT user name, see --user.

See also