dquality.check

This module contains command line interface to data-quality. To use Please make sure the installation Pre-requisites are met.

Functions:

hdf(conf, fname) HDF file structure verifier.
pv(conf) PV verifier.
monitor_quality
monitor(conf, fname, num_files) Data quality monitor verifier.
dquality
dependency
dquality.check.hdf(conf, fname)[source]

HDF file structure verifier.

Parameters:
  • conf (str) – configuration file name, including path
  • file (str) – File Name to verify including path
Returns:

boolean
dquality.check.pv(conf)[source]

PV verifier.

Parameters:conf (str) – configuration file name, including path
Returns:boolean
dquality.check.monitor(conf, fname, num_files)[source]

Data quality monitor verifier.

Parameters:
  • conf (str) – configuration file name including path
  • folder (str) – folder name to monitor
  • num_files (int) – expected number of files. This script will exit after detecting and processing given number of files.
Returns:

None
dquality.check.accumulator(conf, fname, dtype, num_files, report_by_file)[source]

Data Quality monitor.

Parameters:
  • conf (str) – configuration file name, including path
  • folder (str) – monitored directory
  • data_type (str) – defines which data type is being evaluated
  • num_files (int) – number of files that will be processed
  • report_by_files (boolean) – this variable directs how to present the bad indexes in a report. If True, the indexes are related to the files, and a filename is included in the report. Otherwise, the report contains a list of bad indexes.
Returns:

bad_indexes (dict) – a dictionary or list containing bad indexes
dquality.check.data(conf, fname)[source]

Data Quality verifier.

Parameters:
  • conf (str) – name of the configuration file including path. If contains only directory, ‘dqconfig_test.ini’ will a default file name.
  • file (str) – file name to do the quality checks on
Returns:

bad_indexes (Dict)
dquality.check.hdf_dependency(conf, fname)[source]

Dependency verifier.

Parameters:
  • conf (str) – configuration file name, including path
  • file (str) – File Name to verify including path
Returns:

boolean