icatdump
Synopsis
Description
This script queries the content from an ICAT server and serializes it
into a flat file. The format of that file depends on the version of
the ICAT server and the backend that can be selected with the
--format
option.
Options
The configuration options may be set in the command line or in a configuration file. Some options may also be set in the environment.
Specific Options
The following options are specific to icatdump:
- -o FILE, --outputfile FILE
Set the output file name. If the value - is used, the output will be written to standard output. This is also the default.
- -f FORMAT, --format FORMAT
Select the backend to use and thus the output file format. XML and YAML backends are available.
Standard Options
The following options needed to connect the ICAT service are common for most python-icat scripts:
- -h, --help
Display a help message and exit.
- -c CONFIGFILE, --configfile CONFIGFILE
Name of a configuration file.
- -s SECTION, --configsection SECTION
Name of a section in the configuration file. If set, the values in this configuration section will be applied to define other options.
- -w URL, --url URL
URL of the ICAT server. This should point to the web service descriptions. If the URL has no path component, a default path will be added.
- --no-check-certificate
Do not verify the ICAT server’s TLS certificate. This is only relevant if the URL set with
--url
uses HTTPS. It is mostly only useful for connecting a test server that does not have a trusted certificate.
- --http-proxy HTTP_PROXY
Proxy to use for http requests.
- --https-proxy HTTPS_PROXY
Proxy to use for https requests.
- --no-proxy NO_PROXY
Comma separated list of exclusions for proxy use.
- -a AUTH, --auth AUTH
Name of the authentication plugin to use for login to the ICAT server.
- -u USERNAME, --user USERNAME
The ICAT user name.
- -p PASSWORD, --pass PASSWORD
The user’s password. Will prompt for the password if not set.
- -P, --prompt-pass
Prompt for the password. This is mostly useful to override a password set in the configuration file.
Known Issues and Limitations
IDS is not supported: the script only dumps the meta data stored in the ICAT, not the content of the files stored in the IDS.
The output will only contain objects that the user connecting ICAT has read permissions for. The script may need to connect as the ICAT root user in order to get the full content.
The following items are deliberately not included in the output:
Log objects (ICAT server versions older then 4.7.0),
The attributes
id
,createId
,createTime
,modId
, andmodTime
of any object.
It is assumed that for each Dataset ds in the ICAT where ds.sample is not NULL, the condition ds.investigation = ds.sample.investigation holds. If this is not satisfied, the script will fail with a
DataConsistencyError
.The partition of the data into chunks is static. It should rather be dynamic, e.g. chunks should be splitted if the number of objects in them grows too large.
The content in the ICAT server must not be modified while this script is retrieving it. Otherwise the script may fail or the dumpfile be inconsistent.
The script fails if the ICAT server is older then 4.6.0 and the data contains any Study. This is a bug in icat.server.
Environment Variables
- ICAT_CFG
Name of a configuration file, see
--configfile
.
- ICAT_CFG_SECTION
Name of a section in the configuration file, see
--configsection
.
- ICAT_SERVICE
URL of the ICAT server, see
--url
.
- http_proxy
Proxy to use for http requests, see
--http-proxy
.
- https_proxy
Proxy to use for https requests, see
--https-proxy
.
- no_proxy
Exclusions for proxy use, see
--no-proxy
.
- ICAT_AUTH
Name of the authentication plugin, see
--auth
.
- ICAT_USER
ICAT user name, see
--user
.
See also
Section ICAT data files on the structure of the dump files.
Section Predefined configuration variables on the standard options.
The icatingest script.