marine_qc.quality_control package¶

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if aground check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

smooth_win = 41
min_win_period = 8
max_win_period = 10

marine_qc.quality_control.do_bayesian_buddy_check(lat, lon, date, value, climatology, stdev1, stdev2, stdev3, prior_probability_of_gross_error, quantization_interval, one_sigma_measurement_uncertainty, limits, noise_scaling, maximum_anomaly, fail_probability, ignore_indexes=None)¶

Do the Bayesian buddy check.

The bayesian buddy check assigns a probability of gross error to each observation, which is rounded down to the tenth and then multiplied by 10 to yield a flag between 0 and 9.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array.
lon (SequenceNumberType) – 1-dimensional longitude array.
date (SequenceDatetimeType) – 1-dimensional date array.
value (SequenceNumberType) – 1-dimensional anomaly array.
climatology (ClimArgType) – The climatological average(s) used to calculate anomalies. Can be a scalar, sequence, a one-dimensional NumPy array, a pandas Series, a Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
stdev1 (Climatology) – Field of standard deviations representing standard deviation of difference between target gridcell and complete neighbour average (grid area to neighbourhood difference).
stdev2 (Climatology) – Field of standard deviations representing standard deviation of difference between a single observation and the target gridcell average (point to grid area difference).
stdev3 (Climatology) – Field of standard deviations representing standard deviation of difference between random neighbour gridcell and full neighbour average (uncertainty in neighbour average).
prior_probability_of_gross_error (float) – Prior probability of gross error, which is the background rate of gross errors.
quantization_interval (float) – Smallest possible increment in the input values.
one_sigma_measurement_uncertainty (float) – Estimated one sigma measurement uncertainty.
limits (list[int]) – List with three members which specify the search range for the buddy check.
noise_scaling (float) – Tuning parameter used to multiply stdev2. This was determined to be approximately 3.0 by comparison with observed point data. stdev2 was estimated from OSTIA data and typically underestimates the point to area-average difference by this factor.
maximum_anomaly (float) – Largest absolute anomaly, assumes that the maximum and minimum anomalies have the same magnitude.
fail_probability (float) – Probability of gross error that corresponds to a failed test. Anything with a probability of gross error greater than fail_probability will be considered failing.
ignore_indexes (list[int], optional) – List of row numbers to be skipped.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 2s if there are no buddies in the specified limits
Returns array/sequence/Series of 1s if the bayesian buddy check fails
Returns or array/sequence/Series of 0s otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions the default values for the parameters were:

prior_probability_of_gross_error = 0.05
quantization_interval = 0.1
limits = [2, 2, 4]
noise_scaling = 3.0
one_sigma_measurement_uncertainty = 1.0
maximum_anomaly = 8.0
fail_probability = 0.3

marine_qc.quality_control.do_climatology_check(value, climatology, maximum_anomaly, standard_deviation='default', standard_deviation_limits=None, lowbar=None)¶

Climatology check to compare a value with a climatological average within specified anomaly limits.

This check supports optional parameters to customize the comparison.

If standard_deviation is provided, the value is converted into a standardised anomaly. Optionally, if standard deviation is outside the range specified by standard_deviation_limits then standard_deviation is set to whichever of the lower or upper limits is closest. If lowbar is provided, the anomaly must be greater than lowbar to fail regardless of standard_deviation.

Parameters:

value (ValueNumberType) – Value(s) to be compared to climatology. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
climatology (ClimArgType) – The climatological average(s) to which the values(s) will be compared. Can be a scalar, sequence, a one-dimensional NumPy array, a pandas Series, a Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
maximum_anomaly (float) – Largest allowed anomaly. If standard_deviation is provided, this is interpreted as the largest allowed standardised anomaly.
standard_deviation (ClimArgType, default: "default") – The standard deviation(s) used to standardise the anomaly If set to “default”, it is internally treated as 1.0. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series, a Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
standard_deviation_limits (tuple of float, optional) – A tuple of two floats representing the upper and lower limits for standard deviation used in the check.
lowbar (float, optional) – The anomaly must be greater than lowbar to fail regardless of standard deviation.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if standard_deviation_limits[1] is less than or equal to standard_deviation_limits[0], or if maximum_anomaly is less than or equal to 0, or if any of value, climate_normal, or standard_deviation is numerically invalid (None or NaN).
Returns 1 (or array/sequence/Series of 1s) if the difference is outside the specified range.
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

Notes

If either climatology or standard_deviation is a Climatology object, pass lon and lat and date, or month and day, as keyword arguments to extract the relevant climatological value(s).

marine_qc.quality_control.do_date_check(date=None, year=None, month=None, day=None, year_init=None, year_end=None)¶

Perform the date QC check on the report. Checks whether the given date or date components are valid.

Parameters:

date (ValueDatetimeType, optional) – Date(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
year (ValueIntType, optional) – Year(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
month (ValueIntType, optional) – Month(s) of observation (1-12). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
day (ValueIntType, optional) – Day(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
year_init (int, optional) – Initial valid year.
year_end (int, optional) – Last valid year.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if any of year, month, or day is numerically invalid or None,
Returns 1 (or array/sequence/Series of 1s) if date is not valid,
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.do_datetime_check(date=None, year=None, month=None, day=None, hour=None, year_init=None, year_end=None)¶

Perform the datetime QC check on the report.

Checks whether the given date or date components are valid.
Check that the time is valid i.e. in the range 0.0 to 23.99999…

Parameters:

date (ValueDatetimeType, optional) – Date(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
year (ValueIntType, optional) – Year(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
month (ValueIntType, optional) – Month(s) of observation (1-12). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
day (ValueIntType, optional) – Day(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
hour (ValueFloatType, optional) – Hour(s) of observation (minutes as decimal). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
year_init (int, optional) – Initial valid year.
year_end (int, optional) – Last valid year.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if any of year, month, day, or hour is numerically invalid or None,
Returns 1 (or array/sequence/Series of 1s) if date is not valid or hour is not a valid hour,
Returns 0 (or array/sequence/Series of 0s) otherwise.

marine_qc.quality_control.do_day_check(date=None, year=None, month=None, day=None, hour=None, lat=None, lon=None, time_since_sun_above_horizon=None)¶

Determine if the sun was above the horizon a specified time before the report.

This “day” test is used to classify Marine Air Temperature (MAT) measurements as either Night MAT (NMAT) or Day MAT, accounting for solar heating biases and a potential lag between sun rise and the onset of significant warming. The function calculates the sun’s elevation using the sunangle function, offset by the specified time_since_sun_above_horizon.

Parameters:

date (ValueDatetimeType, optional) – Date(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
year (ValueIntType, optional) – Year(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
month (ValueIntType, optional) – Month(s) of observation (1-12). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
day (ValueIntType, optional) – Day(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
hour (ValueFloatType, optional) – Hour(s) of observation (minutes as decimal). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lat (ValueNumberType, optional) – Latitude(s) of observation in degrees. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (ValueNumberType, optional) – Longitude() of observation in degree. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
time_since_sun_above_horizon (float) – Maximum time sun can have been above horizon (or below) to still count as night. Original QC test had this set to 1.0 i.e. it was night between one hour after sundown and one hour after sunrise.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if any of do_position_check, do_date_check, or do_time_check returns 2.
Returns 1 (or array/sequence/Series of 1s) if any of do_position_check, do_date_check, or do_time_check returns 1 or if it is night (sun below horizon an hour ago).
Returns 0 (or array/sequence/Series of 0s) if it is day (sun above horizon an hour ago).

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

See also

do_night_check: Determine if the sun was above the horizon an hour ago based on date, time, and position.

Notes

In previous versions, time_since_sun_above_horizon has the default value 1.0 as one hour is used as a definition of “day” for marine air temperature QC. Solar heating biases were considered to be negligible mmore than one hour after sunset and up to one hour after sunrise.

marine_qc.quality_control.do_few_check(value)¶

Check if number of observations is less than 3.

Parameters:

value (SequenceNumberType) – One-dimensional array of values to be analyzed. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if number of observations is less than 3.
Returns array/sequence/Series of 0s otherwise.

Raises:

ValueError – If either input is not 1-dimensional.
TypeError – If inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.do_hard_limit_check(value, limits)¶

Check if a value is outside specified limits.

Parameters:

value (ValueNumberType) – The value(s) to be tested against the limits. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
limits (tuple of float) – A tuple of two floats representing the lower and upper limit.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if the upper limit is less than or equal to the lower limit, or if the input is invalid (None or NaN).
Returns 1 (or array/sequence/Series of 1s) if value(s) are outside the specified limits.
Returns 0 (or array/sequence/Series of 0s) if value(s) are within limits.

Raises:

TypeError – If decorator inspect_arrays does not return np.ndarrays.

marine_qc.quality_control.do_iquam_track_check(lat, lon, date, speed_limit, delta_d, delta_t, n_neighbours)¶

Perform the IQUAM track check as detailed in Xu and Ignatov 2013.

The track check calculates speeds between pairs of observations and counts how many exceed a threshold speed. The ob with the most violations of this limit is flagged as bad and removed from the calculation. Then the next worst is found and removed until no violations remain.

Parameters:

lat (SequenceNumberType) – One-dimensional latitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (SequenceNumberType) – One-dimensional longitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
date (SequenceDatetimeType) – One-dimensional date array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
speed_limit (float) – Speed limit of platform in kilometers per hour. Typically, 60.0 for ships and 15.0 for drifting buoys.
delta_d (float) – Latitude tolerance in degrees.
delta_t (float) – Time tolerance in hundredths of an hour.
n_neighbours (int) – Number of neighbouring points considered in the analysis.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if the IQUAM QC fails.
Returns array/sequence/Series of 0s otherwise.

Raises:

ValueError – If either input is not 1-dimensional or if their lengths do not match.
TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

Previous versions had default values for the parameters of:

speed_limit = 60.0 for ships and 15.0 for drifting buoys
delta_d = 1.11
delta_t = 0.01
n_neighbours = 5

marine_qc.quality_control.do_landlocked_check(lat, lon, land_sea_mask, land_flag)¶

Check input position(s) to determine whether they correspond to a land point.

Parameters:

lat (ValueNumberType) – Latitude(s) of observation in degrees. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (ValueNumberType) – Longitude() of observation in degree. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
land_sea_mask (ClimArgType) – Land-sea classification value(s) to which the latitude and longitude values(s) will be compared. Can be a scalar, sequence, a one-dimensional NumPy array, a pandas Series, Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
land_flag (int) – Integer value in land_sea_mask that denotes a land point.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if either latitude or longitude is numerically invalid (None/NaN).
Returns 1 (or array/sequence/Series of 1s) if the position does not correspond to a land point
Returns 0 (or array/sequence/Series of 0s) otherwise

Raises:

ValueError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.do_maritime_check(lat, lon, sea_land_mask, sea_flag)¶

Check input position(s) to determine whether they correspond to a sea point.

Parameters:

lat (ValueNumberType) – Latitude(s) of observation in degrees. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (ValueNumberType) – Longitude() of observation in degree. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
sea_land_mask (ClimArgType) – Sea-land classification value(s) to which the latitude and longitude values(s) will be compared. Can be a scalar, sequence, a one-dimensional NumPy array, a pandas Series, a Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
sea_flag (int) – Integer value in sea_land_mask that denotes a sea point.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if either latitude or longitude is numerically invalid (None/NaN).
Returns 1 (or array/sequence/Series of 1s) if latitude and longitude denotes not a sea point
Returns 0 (or array/sequence/Series of 0s) otherwise

Raises:

ValueError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.do_mds_buddy_check(lat, lon, date, value, climatology, standard_deviation, limits, number_of_obs_thresholds, multipliers, ignore_indexes=None)¶

Do the old style buddy check.

The buddy check compares an observation to the average of its near neighbours (called the buddy mean). Depending on how many neighbours there are and their proximity to the observation being tested a multiplier is set. If the difference between the observation and the buddy mean is larger than the multiplier times the standard deviation then the observation fails the buddy check. If no buddy observations are found within the specified limits, then the limits are expanded until the check runs out of specified limits or observations are found within the limits.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array.
lon (SequenceNumberType) – 1-dimensional longitude array.
date (SequenceDatetimeType) – 1-dimensional date array.
value (SequenceNumberType) – 1-dimensional anomaly array.
climatology (ClimArgType) – The climatological average(s) used to calculate anomalies. Can be a scalar, sequence, a one-dimensional NumPy array, a pandas Series, a Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
standard_deviation (Climatology) – Field of standard deviations of 1x1xpentad standard deviations.
limits (list[list]) – Limits a list of lists. Each list member is a three-membered list specifying the longitudinal, latitudinal, and time range within which buddies are sought at each level of search.
number_of_obs_thresholds (list[list]) – Number of observations corresponding to each multiplier in multipliers. The initial list should be the same length as the limits list.
multipliers (list[list]) – Multiplier, x, used for buddy check mu +- x * sigma. The list should have the same structure as number_of_obs_threshold.
ignore_indexes (list[int], optional) – List of row numbers to be skipped.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if the MDS buddy check fails
Returns or array/sequence/Series of 0s otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

The limits, number_of_obs_thresholds, and multipliers parameters are rather complex. The buddy check basically looks within a lat-lon-time range specified by the first element in limits. If there are more than zero observations in the search range then a multiplier is chosen based on how many observations there are.

If the first element of limits is [1,1,2] then we first look within a distance equivalent to 1 degree latitude and longitude at the equator and 2 pentads in time. If there are more than zero observations then we calculate the buddy mean, and we consult the number_of_obs_threshold. If, for example, this is [0, 5, 15, 100] then we look for the first entry where the number of obs is greater than that threshold. We then look up the multiplier in the appropriate list (say [4, 3.5, 3.0, 2.5]). If the difference between an observation and the buddy mean is greater than the multiplier times the standard deviation at that point then it fails the buddy check. So, if there were 10 observations then the multiplier would be 3.5.

Previous versions had default values for the parameters of:

limits = [[1, 1, 2], [2, 2, 2], [1, 1, 4], [2, 2, 4]]
number_of_obs_thresholds = [[0, 5, 15, 100], [0], [0, 5, 15, 100], [0]]
multipliers = [[4.0, 3.5, 3.0, 2.5], [4.0], [4.0, 3.5, 3.0, 2.5], [4.0]]

marine_qc.quality_control.do_missing_value_check(value)¶

Check if a value is equal to None or numerically invalid (NaN).

Parameters:

value (ValueNumberType) – The input value(s) to be tested. Can be a scalar, sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 0 (or array/sequence/Series of 1s) if the input value is None or numerically invalid (NaN)
Returns 1 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays in value_check() does not return numpy ndarrays.

marine_qc.quality_control.do_missing_value_clim_check(climatology, **kwargs)¶

Check if a climatological value is equal to None or numerically invalid (NaN).

Parameters:

climatology (ClimArgType) – The input climatological value(s) to be tested. Can be a scalar, sequence, a one-dimensional NumPy array, a pandas Series, a Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
**kwargs (dict) – Additional keyword arguments passed by the decorator framework (unused).

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 0 (or array/sequence/Series of 1s) if the input value is None or numerically invalid (NaN)
Returns 1 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays in value_check() does not return numpy ndarrays.

Notes

If climatology is a Climatology object, pass lon and lat and date, or month and day, as keyword arguments to extract the relevant climatological value.

marine_qc.quality_control.do_multiple_grouped_check(data, qc_dict=None, preproc_dict=None, return_method='all')¶

Apply one or more buddy-check quality-control (QC) functions to a DataFrame or Series.

Parameters:

data (pandas.Series or pandas.DataFrame) – Hashable input data.
qc_dict (Mapping, optional) – Nested QC dictionary. Keys represent arbitrary user-specified names for the checks. The values are dictionaries which contain the keys “func” (name of the QC function), “names” (input data names as keyword arguments, that will be retrieved from data) and, if necessary, “arguments” (the corresponding keyword arguments). For more information see Examples.
preproc_dict (Mapping, optional) – Nested pre-processing dictionary. Keys represent variable names that can be used by qc_dict. The values are dictionaries which contain the keys “func” (name of the pre-processing function), “names” (input data names as keyword arguments, that will be retrieved from data), and “inputs” (list of input-given variables). For more information see Examples.
return_method ({"all", "passed", "failed"}, default: "all") – If “all”, return QC dictionary containing all requested QC check flags. If “passed”: return QC dictionary containing all requested QC check flags until the first check passes. Other QC checks are flagged as unstested (3). If “failed”: return QC dictionary containing all requested QC check flags until the first check fails. Other QC checks are flagged as unstested (3).

Return type:

DataFrame | Series

Returns:

pandas.DataFrame or pandas.Series – A DataFrame (or Series if the input was a Series) whose columns correspond to the QC names in qc_dict and whose values contain QC flags for each row. Flags depend on the QC functions used.

Raises:

NameError – If a function listed in qc_dict or preproc_dict is not defined. If columns listed in qc_dict or preproc_dict are not available in data.
ValueError – If return_method is not one of [“all”, “passed”, “failed”] If variable names listed in qc_dict or preproc_dict are not valid parameters of the QC function.

Notes

If a variable is pre-processed using preproc_dict, mark the variable name as “__preprocessed__” in qc_dict. For example: “climatology”: “__preprocessed__”.

For more information, see do_multiple_individual_checks().

marine_qc.quality_control.do_multiple_individual_check(data, qc_dict=None, preproc_dict=None, return_method='all')¶

Apply one or more quality-control (QC) functions independently to each row of a DataFrame or Series.

Parameters:

data (pandas.Series or pandas.DataFrame) – Hashable input data.
qc_dict (Mapping, optional) – Nested QC dictionary. Keys represent arbitrary user-specified names for the checks. The values are dictionaries which contain the keys “func” (name of the QC function), “names” (input data names as keyword arguments, that will be retrieved from data) and, if necessary, “arguments” (the corresponding keyword arguments). For more information see Examples.
preproc_dict (Mapping, optional) – Nested pre-processing dictionary. Keys represent variable names that can be used by qc_dict. The values are dictionaries which contain the keys “func” (name of the pre-processing function), “names” (input data names as keyword arguments, that will be retrieved from data), and “inputs” (list of input-given variables). For more information see Examples.
return_method ({"all", "passed", "failed"}, default: "all") – If “all”, return QC dictionary containing all requested QC check flags. If “passed”: return QC dictionary containing all requested QC check flags until the first check passes. Other QC checks are flagged as unstested (3). If “failed”: return QC dictionary containing all requested QC check flags until the first check fails. Other QC checks are flagged as unstested (3).

Return type:

DataFrame | Series

Returns:

Raises:

NameError – If a function listed in qc_dict or preproc_dict is not defined. If columns listed in qc_dict or preproc_dict are not available in data.
ValueError – If return_method is not one of [“all”, “passed”, “failed”] If variable names listed in qc_dict or preproc_dict are not valid parameters of the QC function.

Notes

If a variable is pre-processed using preproc_dict, mark the variable name as “__preprocessed__” in qc_dict. For example: “climatology”: “__preprocessed__”.

For more information, see Examples.

Examples

An example qc_dict for a hard limit test:

qc_dict = {
    "hard_limit_check": {
        "func": "do_hard_limit_check",
        "names": "ATEMP",
        "arguments": {"limits": [193.15, 338.15]},
    }
}

An example qc_dict for a climatology test. Variable “climatology” was previously defined:

qc_dict = {
    "climatology_check": {
        "func": "do_climatology_check",
        "names": {
            "value": "observation_value",
            "lat": "latitude",
            "lon": "longitude",
            "date": "date_time",
        },
        "arguments": {
            "climatology": climatology,
            "maximum_anomaly": 10.0,  # K
        },
    },
}

An example preproc_dict for extracting a climatological value:

preproc_dict = {
    "func": "get_climatological_value",
    "names": {
        "lat": "latitude",
        "lon": "longitude",
        "date": "date_time",
    },
    "inputs": climatology,
}

Make use of both dictionaries:

preproc_dict = {
    "func": "get_climatological_value",
    "names": {
        "lat": "latitude",
        "lon": "longitude",
        "date": "date_time",
    },
    "inputs": climatology,
}

qc_dict = {
    "climatology_check": {
        "func": "do_climatology_check",
        "names": {
            "value": "observation_value",
        },
        "arguments": {
            "climatology": "__preprocessed__",
            "maximum_anomaly": 10.0,  # K
        },
    },
}

Finally, run the function:

do_multiple_individual_check(
    data=df,
    qc_dict=qc_dict,
    preproc_dict=preproc_dict,
    return_method="failed",
)

marine_qc.quality_control.do_multiple_sequential_check(data, groupby=None, qc_dict=None, preproc_dict=None, return_method='all')¶

Apply one or more sequential quality-control (QC) functions to groups of a DataFrame or Series.

Typically for time-ordered or track-based checks.

Parameters:

data (pandas.Series or pandas.DataFrame) – Hashable input data.
groupby (str, iterable of str, or pandas GroupBy, optional) – Specifies how the data should be grouped before applying QC functions. If a string or iterable of strings, data.groupby is called on those keys. If a pandas.DataFrameGroupBy object is provided, its groups are used directly. Any groups that contain indices not present in data are automatically trimmed. If None, the entire input data is treated as a single group. For more information see Examples.
qc_dict (Mapping, optional) – Nested QC dictionary. Keys represent arbitrary user-specified names for the checks. The values are dictionaries which contain the keys “func” (name of the QC function), “names” (input data names as keyword arguments, that will be retrieved from data) and, if necessary, “arguments” (the corresponding keyword arguments).
preproc_dict (Mapping, optional) – Nested pre-processing dictionary. Keys represent variable names that can be used by qc_dict. The values are dictionaries which contain the keys “func” (name of the pre-processing function), “names” (input data names as keyword arguments, that will be retrieved from data), and “inputs” (list of input-given variables). For more information see Examples.
return_method ({"all", "passed", "failed"}, default: "all") – If “all”, return QC dictionary containing all requested QC check flags. If “passed”: return QC dictionary containing all requested QC check flags until the first check passes. Other QC checks are flagged as unstested (3). If “failed”: return QC dictionary containing all requested QC check flags until the first check fails. Other QC checks are flagged as unstested (3).

Return type:

DataFrame | Series

Returns:

Raises:

NameError – If a function listed in qc_dict or preproc_dict is not defined. If columns listed in qc_dict or preproc_dict are not available in data.
ValueError – If return_method is not one of [“all”, “passed”, “failed”] If variable names listed in qc_dict or preproc_dict are not valid parameters of the QC function.

Notes

If a variable is pre-processed using preproc_dict, mark the variable name as “__preprocessed__” in qc_dict. For example: “climatology”: “__preprocessed__”.

For more information, see do_multiple_individual_checks().

marine_qc.quality_control.do_new_aground_check(lat, lon, date, smooth_win, min_win_period)¶

Perform the new aground check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
smooth_win (int) – Length of window (odd number) in datapoints used for smoothing lon/lat.
min_win_period (int) – Minimum period of time in days over which position is assessed for no movement (see description).

Return type:

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if new aground check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

smooth_win = 41
min_win_period = 8

marine_qc.quality_control.do_new_speed_check(lat, lon, date, speed_limit, min_win_period, ship_speed_limit, delta_d, delta_t, n_neighbours)¶

Perform the new speed check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
speed_limit (float) – Maximum allowable speed for an in situ drifting buoy (metres per second).
min_win_period (int) – Minimum period of time in days over which position is assessed for speed estimates (see description).
ship_speed_limit (float) – Ship speed limit for the IQUAM track check.
delta_d (float) – The smallest increment in distance that can be resolved. For 0.01 degrees of lat-lon this is 1.11 km. Used in the IQUAM track check.
delta_t (float) – The smallest increment in time that can be resolved. For hourly data expressed as a float this is 0.01 hours. Used in the IQUAM track check.
n_neighbours (int) – Number of neighbours considered in the IQUAM track check.

Return type:

Returns:

array-like of int, shape (n,) – Array containing the QC outcomes for the new speed check.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

speed_limit = 3.0
min_win_period = 0.375

And, for the IQUAM-specific parameters:

ship_speed_limit = 60.0
delta_d = 1.11
delta_t = 0.01
n_neighbours = 5

marine_qc.quality_control.do_night_check(date=None, year=None, month=None, day=None, hour=None, lat=None, lon=None, time_since_sun_above_horizon=None)¶

Determine if the sun was below the horizon a specified time before the report.

This “night” test is used to classify Marine Air Temperature (MAT) measurements as either Night MAT (NMAT) or Day MAT, accounting for solar heating biases and a potential lag between sun rise and the onset of significant warming. The function calculates the sun’s elevation using the sunangle function, offset by the specified time_since_sun_above_horizon.

Parameters:

date (ValueDatetimeType, optional) – Date(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
year (ValueIntType, optional) – Year(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
month (ValueIntType, optional) – Month(s) of observation (1-12). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
day (ValueIntType, optional) – Day(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
hour (ValueFloatType, optional) – Hour(s) of observation (minutes as decimal). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lat (ValueNumberType, optional) – Latitude(s) of observation in degrees. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (ValueNumberType, optionalt) – Longitude() of observation in degree. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
time_since_sun_above_horizon (float) – Maximum time sun can have been above horizon (or below) to still count as night. Original QC test had this set to 1.0 i.e. it was night between one hour after sundown and one hour after sunrise.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if any of do_position_check, do_date_check, or do_time_check returns 2.
Returns 1 (or array/sequence/Series of 1s) if any of do_position_check, do_date_check, or do_time_check returns 1 or if it is day (sun above horizon an hour ago).
Returns 0 (or array/sequence/Series of 0s) if it is night (sun below horizon an hour ago).

Raises:

ValueError – If mode is not in valid list [“day”, “night”].
TypeError – If decorator inspect_arrays does not return numpy ndarrays.

See also

do_day_check: Determine if the sun was above the horizon an hour ago based on date, time, and position.

Notes

marine_qc.quality_control.do_position_check(lat, lon)¶

Perform the positional QC check on the report.

Simple check to make sure that the latitude and longitude are within specified bounds: - Latitude is between -90 and 90. - Longitude is between 180 and 360.

Parameters:

lat (ValueNumberType) – Latitude(s) of observation in degrees. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (ValueNumberType) – Longitude() of observation in degrees. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if either latitude or longitude is numerically invalid (None/NaN).
Returns 1 (or array/sequence/Series of 1s) if either latitude or longitude is out of the valid range.
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.do_speed_check(lat, lon, date, speed_limit, min_win_period, max_win_period)¶

Perform the Track QC speed check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
speed_limit (float) – Maximum allowable speed for an in situ drifting buoy (metres per second).
min_win_period (int) – Minimum period of time in days over which position is assessed for speed estimates (see description).
max_win_period (int) – Maximum period of time in days over which position is assessed for speed estimates (this should be greater than min_win_period and allow for some erratic temporal sampling e.g. min_win_period + 0.2 to allow for gaps of up to 0.2 - days in sampling).

Return type:

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if speed check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

speed_limit = 2.5
min_win_period = 0.8
max_win_perido = 1.8

marine_qc.quality_control.do_spike_check(value, lat, lon, date, max_gradient_space, max_gradient_time, delta_t, n_neighbours)¶

Perform IQUAM-like spike check.

Parameters:

value (SequenceNumberType) – One-dimensional array of values to be analyzed. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lat (SequenceNumberType) – One-dimensional array of latitudes in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (SequenceNumberType) – One-dimensional array of longitudes in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
date (SequenceDatetimeType) – One-dimensional array of datetime values. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
max_gradient_space (float, default: 0.5) – Maximum allowed spatial gradient. The unit is “units of value” per kilometer.
max_gradient_time (float, default: 1.0) – Maximum allowed temporal gradient. The unit is “units of value” per hour.
delta_t (float, default: 2.0) – Temperature delta used in the comparison. Typically set to 2.0 for ships and 1.0 for drifting buoys.
n_neighbours (int, default: 5) – Number of neighboring points considered in the analysis.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if the spike check fails.
Returns array/sequence/Series of 0s otherwise.

Raises:

ValueError – If either input is not 1-dimensional or if their lengths do not match.
TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

max_gradient_space: float = 0.5
max_gradient_time: float = 1.0
delta_t: float = 2.0
n_neighbours: int = 5

marine_qc.quality_control.do_sst_biased_check(lat, lon, date, sst, ostia, ice, bgvar, n_eval, bias_lim, drif_intra, drif_inter, err_std_n, n_bad, background_err_lim)¶

Perform the SST bias check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
sst (SequenceNumberType) – 1-dimensional array of sea surface temperatures in K.
ostia (SequenceNumberType) – 1-dimensional array of background field sea surface temperatures in K.
ice (SequenceNumberType) – 1-dimensional array of ice concentrations in the range 0.0 to 1.0.
bgvar (SequenceNumberType) – 1-dimensional array of background sea surface temperature fields variances in K^2.
n_eval (int) – The minimum number of drifter observations required to be assessed by the long-record check.
bias_lim (float) – Maximum allowable drifter-background bias, beyond which a record is considered biased (degC or K).
drif_intra (float) – Maximum random measurement uncertainty reasonably expected in drifter data (standard deviation, degC or K).
drif_inter (float) – Spread of biases expected in drifter data (standard deviation, degC or K).
err_std_n (float) – Number of standard deviations of combined background and drifter error, beyond which short-record data are deemed suspicious.
n_bad (int) – Minimum number of suspicious data points required for failure of short-record check.
background_err_lim (float) – Background error variance beyond which the SST background is deemed unreliable (degC squared or K squared).

Return type:

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if SST bias check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

n_eval = 30
bias_lim = 1.10
drif_intra = 1.0
drif_inter = 0.29
err_std_n = 3.0
n_bad = 2
background_err_lim = 0.3

marine_qc.quality_control.do_sst_biased_noisy_short_check(lat, lon, date, sst, ostia, ice, bgvar, n_eval, bias_lim, drif_intra, drif_inter, err_std_n, n_bad, background_err_lim)¶

Perform the SST short check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
sst (SequenceNumberType) – 1-dimensional array of sea surface temperatures in K.
ostia (SequenceNumberType) – 1-dimensional array of background field sea surface temperatures in K.
ice (SequenceNumberType) – 1-dimensional array of ice concentrations in the range 0.0 to 1.0.
bgvar (SequenceNumberType) – 1-dimensional array of background sea surface temperature fields variances in K^2.
n_eval (int) – The minimum number of drifter observations required to be assessed by the long-record check.
bias_lim (float) – Maximum allowable drifter-background bias, beyond which a record is considered biased (degC or K).
drif_intra (float) – Maximum random measurement uncertainty reasonably expected in drifter data (standard deviation, degC or K).
drif_inter (float) – Spread of biases expected in drifter data (standard deviation, degC or K).
err_std_n (float) – Number of standard deviations of combined background and drifter error, beyond which short-record data are deemed suspicious.
n_bad (int) – Minimum number of suspicious data points required for failure of short-record check.
background_err_lim (float) – Background error variance beyond which the SST background is deemed unreliable (degC squared or K squared).

Return type:

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if SST short check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

n_eval = 30
bias_lim = 1.10
drif_intra = 1.0
drif_inter = 0.29
err_std_n = 3.0
n_bad = 2
background_err_lim = 0.3

marine_qc.quality_control.do_sst_end_tail_check(lat, lon, date, sst, ostia, ice, bgvar, long_win_len, long_err_std_n, short_win_len, short_err_std_n, short_win_n_bad, drif_inter, drif_intra, background_err_lim)¶

Perform the SST Start Tail Check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
sst (SequenceNumberType) – 1-dimensional array of sea surface temperatures in K.
ostia (SequenceNumberType) – 1-dimensional array of background field sea surface temperatures in K.
ice (SequenceNumberType) – 1-dimensional array of ice concentrations in the range 0.0 to 1.0.
bgvar (SequenceNumberType) – 1-dimensional array of background sea surface temperature fields variances in K^2.
long_win_len (int) – Length of window (in data-points) over which to make long tail-check (must be an odd number).
long_err_std_n (float) – Number of standard deviations of combined background and drifter bias error, beyond which data fail bias check.
short_win_len (int) – Length of window (in data-points) over which to make the short tail-check.
short_err_std_n (float) – Number of standard deviations of combined background and drifter error, beyond which data are deemed suspicious.
short_win_n_bad (int) – Minimum number of suspicious data points required for failure of short check window.
drif_inter (float) – Spread of biases expected in drifter data (standard deviation, degC or K).
drif_intra (float) – Maximum random measurement uncertainty reasonably expected in drifter data (standard deviation, degC or K).
background_err_lim (float) – Background error variance beyond which the SST background is deemed unreliable (degC squared).

Return type:

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if SST start tail check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

long_win_len = 121
long_err_std_n = 3.0
short_win_len = 30
short_err_std_n = 3.0
short_win_n_bad = 2
drif_inter = 0.29
drif_intra = 1.00
background_err_lim = 0.3

marine_qc.quality_control.do_sst_freeze_check(sst, freezing_point, freeze_check_n_sigma='default', sst_uncertainty='default')¶

Check input sea-surface temperature(s) to see if it is above freezing.

This is a simple freezing point check made slightly more complex. We want to check if a measurement of SST is above freezing, but there are two problems. First, the freezing point can vary from place to place depending on the salinity of the water. Second, there is uncertainty in SST measurements. If we place a hard cut-off at -1.8C, then we are likely to bias the average of many measurements too high when they are near the freezing point - observational error will push the measurements randomly higher and lower, and this test will trim out the lower tail, thus biasing the result. The inclusion of an SST uncertainty parameter might mitigate that, and we allow that possibility here. Note also that many ships make sea-surface temperature measurements to the nearest whole degree, which in the case of water at or close to freezing would round to -2C and would fail a naive test.

Parameters:

sst (ValueNumberType) – Input sea-surface temperature value(s) to be checked. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
freezing_point (float, optional) – The freezing point of the water.
freeze_check_n_sigma (float, optional, default: "default") – Number of uncertainty standard deviations that sea surface temperature can be below the freezing point before the QC check fails.
sst_uncertainty (float, optional, default: "default") – The uncertainty in the SST value.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if any of sst, freezing_point, sst_uncertainty, or n_sigma is numerically invalid (None or NaN).
Returns 1 (or array/sequence/Series of 1s) if sst is below freezing_point by more than n_sigma times sst_uncertainty.
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, some parameters had default values:

sst_uncertainty: 0.0

freezing_point: -1.80

n_sigma: 2.0

marine_qc.quality_control.do_sst_noisy_check(lat, lon, date, sst, ostia, ice, bgvar, n_eval, bias_lim, drif_intra, drif_inter, err_std_n, n_bad, background_err_lim)¶

Perform the SST noise check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
sst (SequenceNumberType) – 1-dimensional array of sea surface temperatures in K.
ostia (SequenceNumberType) – 1-dimensional array of background field sea surface temperatures in K.
ice (SequenceNumberType) – 1-dimensional array of ice concentrations in the range 0.0 to 1.0.
bgvar (SequenceNumberType) – 1-dimensional array of background sea surface temperature fields variances in K^2.
n_eval (int) – The minimum number of drifter observations required to be assessed by the long-record check.
bias_lim (float) – Maximum allowable drifter-background bias, beyond which a record is considered biased (degC or K).
drif_intra (float) – Maximum random measurement uncertainty reasonably expected in drifter data (standard deviation, degC or K).
drif_inter (float) – Spread of biases expected in drifter data (standard deviation, degC or K).
err_std_n (float) – Number of standard deviations of combined background and drifter error, beyond which short-record data are deemed suspicious.
n_bad (int) – Minimum number of suspicious data points required for failure of short-record check.
background_err_lim (float) – Background error variance beyond which the SST background is deemed unreliable (degC squared or K squared).

Return type:

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if SST noise check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

n_eval = 30
bias_lim = 1.10
drif_intra = 1.0
drif_inter = 0.29
err_std_n = 3.0
n_bad = 2
background_err_lim = 0.3

marine_qc.quality_control.do_sst_start_tail_check(lat, lon, date, sst, ostia, ice, bgvar, long_win_len, long_err_std_n, short_win_len, short_err_std_n, short_win_n_bad, drif_inter, drif_intra, background_err_lim)¶

Perform the SST Start Tail Check.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
sst (SequenceNumberType) – 1-dimensional array of sea surface temperatures in K.
ostia (SequenceNumberType) – 1-dimensional array of background field sea surface temperatures in K.
ice (SequenceNumberType) – 1-dimensional array of ice concentrations in the range 0.0 to 1.0.
bgvar (SequenceNumberType) – 1-dimensional array of background sea surface temperature fields variances in K^2.
long_win_len (int) – Length of window (in data-points) over which to make long tail-check (must be an odd number).
long_err_std_n (float) – Number of standard deviations of combined background and drifter bias error, beyond which data fail bias check.
short_win_len (int) – Length of window (in data-points) over which to make the short tail-check.
short_err_std_n (float) – Number of standard deviations of combined background and drifter error, beyond which data are deemed suspicious.
short_win_n_bad (int) – Minimum number of suspicious data points required for failure of short check window.
drif_inter (float) – Spread of biases expected in drifter data (standard deviation, degC or K).
drif_intra (float) – Maximum random measurement uncertainty reasonably expected in drifter data (standard deviation, degC or K).
background_err_lim (float) – Background error variance beyond which the SST background is deemed unreliable (degC squared).

Return type:

Returns:

array-like of int, shape (n,) – 1-dimensional array containing QC flags. 1 if SST start tail check fails, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous versions, default values for the parameters were:

long_win_len = 121
long_err_std_n = 3.0
short_win_len = 30
short_err_std_n = 3.0
short_win_n_bad = 2
drif_inter = 0.29
drif_intra = 1.00
background_err_lim = 0.3

marine_qc.quality_control.do_supersaturation_check(dpt, at2)¶

Perform the super saturation check.

Check if a valid dewpoint temperature is greater than a valid air temperature.

Parameters:

dpt (ValueNumberType) – Dewpoint temperature value(s). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
at2 (ValueNumberType) – Air temperature values(s). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if either dpt or at2 is invalid (None or NaN).
Returns 1 (or array/sequence/Series of 1s) if supersaturation is detected,
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.do_time_check(date=None, hour=None)¶

Perform the date QC check on the report. Check that the time is valid i.e. in the range 0.0 to 23.99999…

Parameters:

date (ValueDatetimeType, optional) – Date(s) of observation. Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
hour (ValueFloatType, optional) – Hour(s) of observation (minutes as decimal). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if hour is numerically invalid or None,
Returns 1 (or array/sequence/Series of 1s) if hour is not a valid hour,
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.do_track_check(vsi, dsi, lat, lon, date, max_direction_change, max_speed_change, max_absolute_speed, max_midpoint_discrepancy)¶

Perform one pass of the track check.

This is an implementation of the MDS track check code which was originally written in the 1990s. I don’t know why this piece of historic trivia so exercises my mind, but it does: the 1990s! I wish my code would last so long.

Parameters:

vsi (SequenceNumberType) – One-dimensional reported speed array in km/h. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
dsi (SequenceNumberType) – One-dimensional reported heading array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lat (SequenceNumberType) – One-dimensional latitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (SequenceNumberType) – One-dimensional longitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
date (SequenceDatetimeType) – One-dimensional date array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
max_direction_change (float, default: 60.0) – Maximum valid direction change in degrees.
max_speed_change (float, default: 10.0) – Maximum valid speed change in km/h.
max_absolute_speed (float, default: 40.0) – Maximum valid absolute speed in km/h.
max_midpoint_discrepancy (float, default: 150.0) – Maximum valid midpoint discrepancy in meters.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if the track check fails.
Returns array/sequence/Series of 0s otherwise.

Raises:

ValueError – If either input is not 1-dimensional or if their lengths do not match.
TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

If number of observations is less than three, the track check always passes.

In previous versions, the default values of the parameters were:

max_direction_change = 60.0
max_speed_change = 10.0
max_absolute_speed = 40.0
max_midpoint_discrepancy = 150.0

marine_qc.quality_control.do_valid_value_check(value)¶

Check if a value is not equal to None or numerically valid.

Parameters:

value (ValueNumberType) – The input value(s) to be tested. Can be a scalar, sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 1 (or array/sequence/Series of 1s) if the input value is None or numerically invalid (NaN)
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays in value_check() does not return numpy ndarrays.

marine_qc.quality_control.do_valid_value_clim_check(climatology, **kwargs)¶

Check if a climatological value is not equal to None or numerically valid.

Parameters:

climatology (ClimArgType) – The input climatological value(s) to be tested. Can be a scalar, sequence, a one-dimensional NumPy array, a pandas Series, a Climatology, a path-like string on disk, a xarray Dataset or a xarray DataArray.
**kwargs (dict) – Additional keyword arguments passed by the decorator framework (unused).

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 1 (or array/sequence/Series of 1s) if the input value is None or numerically invalid (NaN)
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays in value_check() does not return numpy ndarrays.

Notes

If climatology is a Climatology object, pass lon and lat and date, or month and day, as keyword arguments to extract the relevant climatological value.

marine_qc.quality_control.do_wind_consistency_check(wind_speed, wind_direction)¶

Test to compare windspeed to winddirection to check if they are consistent.

Zero windspeed should correspond to no particular direction (variable) and wind speeds above a threshold should correspond to a particular direction.

Parameters:

wind_speed (ValueNumberType) – Wind speed value(s). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
wind_direction (ValueNumberType) – Wind direction value(s). Can be a scalar, a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 2 (or array/sequence/Series of 2s) if either wind_speed or wind_direction is invalid (None or NaN).
Returns 1 (or array/sequence/Series of 1s) if wind_speed and wind_direction are inconsistent,
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.find_multiple_rounded_values(value, min_count, threshold)¶

Find instances when more than “threshold” of the observations are whole numbers and set the ‘round’ flag.

Used in the humidity QC where there are times when the values are rounded and this may have caused a bias.

Parameters:

value (SequenceNumberType) – One-dimensional array of values. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
min_count (int, default: 20) – Minimum number of rounded figures that will trigger the test.
threshold (float, default: 0.5) – Minimum fraction of all observations that will trigger the test.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if the value is a whole number.
Returns array/sequence/Series of 0s otherwise.

Raises:

ValueError – If threshold is not between 0.0 and 1.0.
TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

Previous versions had default values for the parameters of:

min_count = 20
threshold = 0.5

marine_qc.quality_control.find_repeated_values(value, min_count, threshold)¶

Find cases where more than a given proportion of SSTs have the same value.

This function goes through a voyage and finds any cases where more than a threshold fraction of the observations have the same values for a specified variable.

Parameters:

value (SequenceNumberType) – One-dimensional array of values. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
min_count (int, default: 20) – Minimum number of repeated values that will trigger the test.
threshold (float, default: 0.7) – Smallest fraction of all observations that will trigger the test.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if the value is repeated.
Returns array/sequence/Series of 0s otherwise.

Raises:

ValueError –
- If threshold is not between 0.0 and 1.0.
TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

Previous versions had default values for the parameters of:

min_count = 20
threshold = 0.7

marine_qc.quality_control.find_saturated_runs(at, dpt, lat, lon, date, min_time_threshold, shortest_run)¶

Perform checks on persistence of 100% rh while going through the voyage.

While going through the voyage repeated strings of 100 %rh (AT == DPT) are noted. If a string extends beyond 20 reports and two days/48 hrs in time then all values are set to fail the repsat qc flag.

Parameters:

at (SequenceNumberType) – One-dimensional air temperature array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
dpt (SequenceNumberType) – One-dimensional dew point temperature array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lat (SequenceNumberType) – One-dimensional latitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (SequenceNumberType) – One-dimensional longitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
date (SequenceDatetimeType) – One-dimensional date array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
min_time_threshold (float, default: 48.0) – Minimum time threshold in hours.
shortest_run (int, default: 4) – Shortest number of observations.

Return type:

Returns:

SequenceIntType – Same type as input, but with integer values

Returns array/sequence/Series of 1s if a saturated run is found.
Returns array/sequence/Series of 0s otherwise.

Raises:

ValueError – If either input is not 1-dimensional or if their lengths do not match.
TypeError – If inspect_arrays does not return numpy ndarrays.

Notes

In previous version, default values for the parameters were:

min_time_threshold = 48.0
shortest_run = 4

Submodules¶

marine_qc.quality_control.qc_buoy_tracking module¶

Buoy tracking QC module.

Module containing QC functions for sequential reports from a single drifting buoy.

class marine_qc.quality_control.qc_buoy_tracking.AgroundChecker(lon, lat, date, smooth_win, min_win_period, max_win_period)[source]¶

Bases: object

Class used to carry out do_aground_check().

Check to see whether a drifter has run aground based on 1/100th degree precision positions. A flag is set for each input report: flag=1 for reports deemed aground, else flag=0.

Positional errors introduced by lon/lat ‘jitter’ and data precision can be of order several km’s. Longitude and latitude timeseries are smoothed prior to assessment to reduce position ‘jitter’. Some post-smoothing position ‘jitter’ may remain and its expected magnitude is set within the function by the ‘tolerance’ parameter. A drifter is deemed aground when, after a period of time, the distance between reports is less than the ‘tolerance’. The minimum period of time over which this assessment is made is set by ‘min_win_period’. This period must be long enough such that slow moving drifters are not falsely flagged as aground given errors in position (e.g. a buoy drifting at around 1 cm/s will travel around 1 km/day; given ‘tolerance’ and precision errors of a few km’s the ‘min_win_period’ needs to be several days to ensure distance-travelled exceeds the error so that motion is reliably detected and the buoy is not falsely flagged as aground). However, min_win_period should not be longer than necessary as buoys that run aground for less than min_win_period will not be detected.

Because temporal sampling can be erratic the time period over which an assessment is made is specified as a range (bound by ‘min_win_period’ and ‘max_win_period’) - assessment uses the longest time separation available within this range. If a drifter is deemed aground and subsequently starts moving (e.g. if a drifter has moved very slowly for a prolonged period) incorrectly flagged reports will be reinstated.

smooth_win: length of window (odd number) in datapoints used for smoothing lon/lat
min_win_period: minimum period of time in days over which position is assessed for no movement (see description)
max_win_period: maximum period of time in days over which position is assessed for no movement (this should be greater than min_win_period and allow for erratic temporal sampling e.g. min_win_period+2 to allow for gaps of up to 2-days in sampling).

Parameters:

lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
smooth_win (int) – Length of window (odd number) in datapoints used for smoothing lon/lat.
min_win_period (int) – Minimum period of time in days over which position is assessed for no movement (see description).
max_win_period (int or None) – Maximum period of time in days over which position is assessed for no movement (this should be greater than min_win_period and allow for erratic temporal sampling e.g. min_win_period+2 to allow for gaps of up to 2-days in sampling).

do_aground_check()[source]¶

Perform the actual aground check.

Return type:: None

get_qc_outcomes()[source]¶

Return the QC outcomes.

Return type:: ndarray
Returns:: array-like of int, shape (n,) – 1-dimensional array containing QC flags.

hrs_smooth: ndarray¶

lat_smooth: ndarray¶

lon_smooth: ndarray¶

smooth_arrays()[source]¶

Perform the preprocessing of the lat lon and time arrays.

Return type:: None

tolerance = 1.5725359013624185¶

valid_arrays()[source]¶

Check the input arrays are valid. Raises a warning and returns False if not valid.

Return type:: bool
Returns:: bool – True if array is valid, otherwise False.

valid_parameters()[source]¶

Check the parameters are valid. Raises a warning and returns False if not valid.

Return type:: bool
Returns:: bool – True if parameter is valid, otherwise False.

class marine_qc.quality_control.qc_buoy_tracking.NewSpeedChecker(lon, lat, date, speed_limit, min_win_period, ship_speed_limit, delta_d, delta_t, n_neighbours)[source]¶

Bases: object

Class used to carry out do_new_speed_check().

Check to see whether a drifter has been picked up by a ship (out of water) based on 1/100th degree precision positions. A flag is set for each input report: flag=1 for reports deemed picked up, else flag=0.

A drifter is deemed picked up if it is moving faster than might be expected for a fast ocean current (a few m/s). Unreasonably fast movement is detected when speed of travel between report-pairs exceeds the chosen ‘speed_limit’ (speed is estimated as distance between reports divided by time separation - this ‘straight line’ speed between the two points is a minimum speed estimate given a less-direct path may have been followed). Positional errors introduced by lon/lat ‘jitter’ and data precision can be of order several km’s. Reports must be separated by a suitably long period of time (the ‘min_win_period’) to minimise the effect of these errors when calculating speed e.g. for reports separated by 9 hours errors of order 10 cm/s would result which are a few percent of fast ocean current speed. Conversely, the period of time chosen should not be too long so as to resolve short-lived burst of speed on manouvering ships. Larger positional errors may also trigger the check.

For each report, speed is assessed over the shortest available period that exceeds ‘min_win_period’.

Prior to assessment the drifter record is screened for positional errors using the iQuam track check method (from ex.Voyage). When running the iQuam check the record is treated as a ship (not a drifter) so as to avoid accidentally filtering out observations made aboard a ship (which is what we are trying to detect). This iQuam track check does not overwrite any existing iQuam track check flags.

IMPORTANT - for optimal performance, drifter records with observations failing this check should be subsequently manually reviewed. Ships move around in all sorts of complicated ways that can readily confuse such a simple check (e.g. pausing at sea, crisscrossing its own path) and once some erroneous movement is detected it is likely a human operator can then better pick out the actual bad data. False fails caused by positional errors (particularly in fast ocean currents) will also need reinstating.

The class has the following class attributes which can be modified using the set_parameters method.

iquam_parameters: Parameter dictionary for Voyage.iquam_track_check() function.
speed_limit: maximum allowable speed for an in situ drifting buoy (metres per second)
min_win_period: minimum period of time in days over which position is assessed for speed estimates (see description)

Parameters:

lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
speed_limit (float) – Maximum allowable speed for an in situ drifting buoy (metres per second).
min_win_period (int) – Minimum period of time in days over which position is assessed for speed estimates (see description).
ship_speed_limit (float) – Ship speed limit for the IQUAM track check.
delta_d (float) – The smallest increment in distance that can be resolved. For 0.01 degrees of lat-lon this is 1.11 km. Used in the IQUAM track check.
delta_t (float) – The smallest increment in time that can be resolved. For hourly data expressed as a float this is 0.01 hours. Used in the IQUAM track check.
n_neighbours (int) – Number of neighbours considered in the IQUAM track check.

do_new_speed_check()[source]¶

Perform the actual new speed check.

Return type:: None

get_qc_outcomes()[source]¶

Retrieve the QC outcomes for all observations.

Return type:: ndarray
Returns:: numpy.ndarray – Array of QC flags for each observation (0 = valid, 1 = flagged, untested otherwise).

iquam_track_ship: ndarray¶

perform_iquam_track_check()[source]¶

Perform iQuam track check as if reports are from a ship.

A deep copy of reps is made so metadata can be safely modified ahead of iQuam check an array of qc flags (iquam_track_ship) is the result.

Return type:: None

valid_arrays()[source]¶

Validate the input arrays (longitude, latitude, and time differences).

Checks for: - NaN values in longitude, latitude, or time arrays. - Monotonicity of the time array (self.hrs).

Warnings are raised for any issues detected.

Return type:: bool
Returns:: bool – True if all arrays are valid, False otherwise.

valid_parameters()[source]¶

Validate the QC parameters to ensure they are sensible.

Checks that: - speed_limit is non-negative. - min_win_period is non-negative.

Warnings are raised for any invalid parameters.

Return type:: bool
Returns:: bool – True if all parameters are valid, False otherwise.

class marine_qc.quality_control.qc_buoy_tracking.SSTBiasedNoisyChecker(lat, lon, date, sst, ostia, bgvar, ice, n_eval, bias_lim, drif_intra, drif_inter, err_std_n, n_bad, background_err_lim)[source]¶

Bases: object

Class used to perform the do_sst_biased_check(), do_sst_noisy_check(), and do_sst_biased_noisy_short_check().

Check to see whether a drifter sea surface temperature record is unacceptably biased or noisy as a whole.

The check makes an assessment of the quality of data in a drifting buoy record by comparing to a background reference field. If the record is found to be unacceptably biased or noisy relative to the background all observations are flagged by the check. For longer records the flags ‘drf_bias’ and ‘drf_noise’ are set for each input report: flag=1 for records with erroneous data, else flag=0. For shorter records ‘drf_short’ is set for each input report: flag=1 for reports with erroneous data, else flag=0.

When making the comparison an allowance is made for background error variance and also normal drifter error (both bias and random measurement error). A background error variance limit is also specified, beyond which the background is deemed unreliable and is excluded from comparison. Observations made during the day, in icy regions or where the background value is missing are also excluded from the comparison.

The check has two separate streams; a ‘long-record check’ and a ‘short-record check’. Records with at least n_eval observations are passed to the long-record check, else they are passed to the short-record check. The long-record check looks for records that are too biased or noisy as a whole. The short record check looks for individual observations exceeding a noise limit within a record. The purpose of n_eval is to ensure records with too few observations for their bias and noise to be reliably estimated are handled separately by the short-record check.

The correlation of the background error is treated as unknown and handled differently for each assessment. For the long-record noise-check and the short-record check the background error is treated as uncorrelated, which maximises the possible impact of background error on these assessments. For the long-record bias-check a limit (bias_lim) is specified beyond which the record is considered biased. The default value for this limit was chosen based on histograms of drifter-background bias. An alternative approach would be to treat the background error as entirely correlated across a long-record, which maximises its possible impact on the bias assessment. In this case the histogram approach was used as the limit could be tuned to give better results.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
sst (SequenceNumberType) – 1-dimensional sea surface temperature array in K.
ostia (SequenceNumberType) – 1-dimensional background field sea surface temperature array in K.
bgvar (SequenceNumberType) – 1-dimensional background variance array in K^2.
ice (SequenceNumberType) – 1-dimensional ice concentration array in range 0,1.
n_eval (int) – The minimum number of drifter observations required to be assessed by the long-record check.
bias_lim (float) – Maximum allowable drifter-background bias, beyond which a record is considered biased (degC or K).
drif_intra (float) – Maximum random measurement uncertainty reasonably expected in drifter data (standard deviation, degC or K).
drif_inter (float) – Spread of biases expected in drifter data (standard deviation, degC or K).
err_std_n (float) – Number of standard deviations of combined background and drifter error, beyond which short-record data are deemed suspicious.
n_bad (int) – Minimum number of suspicious data points required for failure of short-record check.
background_err_lim (float) – Background error variance beyond which the SST background is deemed unreliable (degC squared or K squared).

bgerr: ndarray¶

bgvar_is_masked: bool¶

do_sst_biased_noisy_check()[source]¶

Perform the bias/noise check QC.

Return type:: None

get_qc_outcomes_bias()[source]¶

Return the QC outcomes for the bias check.

Return type:: ndarray
Returns:: array-like of int, shape (n,) – 1-dimensional array containing QC flags.

get_qc_outcomes_noise()[source]¶

Return the QC outcomes for the noisy check.

Return type:: ndarray
Returns:: array-like of int, shape (n,) – 1-dimensional array containing QC flags.

get_qc_outcomes_short()[source]¶

Return the QC outcomes for the short check.

Return type:: ndarray
Returns:: array-like of int, shape (n,) – 1-dimensional array containing QC flags.

set_all_qc_outcomes_to(input_state)[source]¶

Set all the QC outcomes to the specified input_state.

Parameters:: input_state (int) – QC flag to map to the QC outcomes.
Return type:: None

sst_anom: ndarray¶

valid_parameters()[source]¶

Check the parameters are valid. Raises a warning and returns False if not valid.

Return type:: bool
Returns:: bool – True if parameter is valid, otherwise False.

class marine_qc.quality_control.qc_buoy_tracking.SSTTailChecker(lat, lon, sst, ostia, ice, bgvar, date, long_win_len, long_err_std_n, short_win_len, short_err_std_n, short_win_n_bad, drif_inter, drif_intra, background_err_lim)[source]¶

Bases: object

Class used to carry out do_sst_start_tail_check() and do_sst_end_tail_check().

Check to see whether there is erroneous sea surface temperature data at the beginning or end of a drifter record (referred to as ‘tails’). Flags are set for each input report: flag=1 for reports with erroneous data, else flag=0, ‘drf_tail1’ is used for bad data at the beginning of a record, ‘drf_tail2’ is used for bad data at the end of a record.

The tail check makes an assessment of the quality of data at the start and end of a drifting buoy record by comparing to a background reference field. Data found to be unacceptably biased or noisy relative to the background are flagged by the check. When making the comparison an allowance is made for background error variance and also normal drifter error (both bias and random measurement error). The correlation of the background error is treated as unknown and takes on a value which maximises background error dependent on the assessment being made. A background error variance limit is also specified, beyond which the background is deemed unreliable. Observations made during the day, in icy regions or where the background value is missing are excluded from the comparison.

The check proceeds in two steps; a ‘long tail-check’ followed by a ‘short tail-check’. The idea is that the short tail-check has finer resolution but lower sensitivity than the long tail-check and may pick off noisy data not picked up by the long tail check. Only observations that pass the long tail-check are passed to the short tail-check. Both of these tail checks proceed by moving a window over the data and assessing the data in each window. Once good data are found the check stops and any bad data preceding this are flagged. If unreliable background data are encountered the check stops. The checks are run forwards and backwards over the record so as to assess data at the start and end of the record. If the whole record fails no observations are flagged as there are then no ‘tails’ in the data (this is left for other checks). The long tail check looks for groups of observations that are too biased or noisy as a whole. The short tail check looks for individual observations exceeding a noise limit within the window.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
sst (SequenceNumberType) – 1-dimensional array of sea surface temperatures in K.
ostia (SequenceNumberType) – 1-dimensional array of background field sea surface temperatures in K.
ice (SequenceNumberType) – 1-dimensional array of ice concentrations in the range 0.0 to 1.0.
bgvar (SequenceNumberType) – 1-dimensional array of background sea surface temperature fields variances in K^2.
date (SequenceDatetimeType) – 1-dimensional date array.
long_win_len (int) – Length of window (in data-points) over which to make long tail-check (must be an odd number).
long_err_std_n (float) – Number of standard deviations of combined background and drifter bias error, beyond which data fail bias check.
short_win_len (int) – Length of window (in data-points) over which to make the short tail-check.
short_err_std_n (float) – Number of standard deviations of combined background and drifter error, beyond which data are deemed suspicious.
short_win_n_bad (int) – Minimum number of suspicious data points required for failure of short check window.
drif_inter (float) – Spread of biases expected in drifter data (standard deviation, degC or K).
drif_intra (float) – Maximum random measurement uncertainty reasonably expected in drifter data (standard deviation, degC or K).
background_err_lim (float) – Background error variance beyond which the SST background is deemed unreliable (degC squared).

bgerr: ndarray¶

do_sst_tail_check(start_tail)[source]¶

Perform the actual SST tail check.

Parameters:: start_tail (bool) – If True flag the start of the record as failed, otherwise flag the end of the record as failed.
Return type:: None

end_tail_ind: int¶

get_qc_outcomes()[source]¶

Return the QC outcomes.

Return type:: ndarray
Returns:: array-like of int, shape (n,) – 1-dimensional array containing QC flags.

reps_ind: ndarray¶

sst_anom: ndarray¶

start_tail_ind: int¶

valid_parameters()[source]¶

Check the parameters are valid. Raises a warning and returns False if not valid.

Return type:: bool
Returns:: bool – True if parameter is valid, otherwise False.

class marine_qc.quality_control.qc_buoy_tracking.SpeedChecker(lon, lat, date, speed_limit, min_win_period, max_win_period)[source]¶

Bases: object

Class used to carry out do_speed_check().

The check identifies whether a drifter has been picked up by a ship (out of water) based on 1/100th degree precision positions. A flag is set for each input report: flag=1 for reports deemed picked up, else flag=0.

A drifter is deemed picked up if it is moving faster than might be expected for a fast ocean current (a few m/s). Unreasonably fast movement is detected when speed of travel between report-pairs exceeds the chosen ‘speed_limit’ (speed is estimated as distance between reports divided by time separation - this ‘straight line’ speed between the two points is a minimum speed estimate given a less-direct path may have been followed). Positional errors introduced by lon/lat ‘jitter’ and data precision can be of order several km’s. Reports must be separated by a suitably long period of time (the ‘min_win_period’) to minimise the effect of these errors when calculating speed e.g. for reports separated by 24 hours errors of several cm/s would result which are two orders of magnitude less than a fast ocean current which seems reasonable. Conversely, the period of time chosen should not be too long so as to resolve short-lived burst of speed on manoeuvring ships. Larger positional errors may also trigger the check. Because temporal sampling can be erratic the time period over which this assessment is made is specified as a range (bound by ‘min_win_period’ and ‘max_win_period’) - assessment uses the longest time separation available within this range.

Parameters:

lon (SequenceNumberType) – 1-dimensional longitude array in degrees.
lat (SequenceNumberType) – 1-dimensional latitude array in degrees.
date (SequenceDatetimeType) – 1-dimensional date array.
speed_limit (float) – Maximum allowable speed for an in situ drifting buoy (metres per second).
min_win_period (int) – Minimum period of time in days over which position is assessed for speed estimates (see description).
max_win_period (int) – Maximum period of time in days over which position is assessed for speed estimates (this should be greater than min_win_period and allow for some erratic temporal sampling e.g. min_win_period + 0.2 to allow for gaps of up to 0.2 - days in sampling).

do_speed_check()[source]¶

Perform the actual speed check.

Return type:: None

get_qc_outcomes()[source]¶

Retrieve the QC outcomes for all observations.

Return type:: ndarray
Returns:: numpy.ndarray – Array of QC flags for each observation (0 = passed, 1 = failed, untested otherwise).

valid_arrays()[source]¶

Validate the input observation arrays (longitude, latitude, and time differences).

Checks for: - NaN values in longitude, latitude, or time arrays. - Monotonicity of the time array (self.hrs).

Warnings are raised for any issues detected.

Return type:: bool
Returns:: bool – True if all arrays are valid, False otherwise.

valid_parameters()[source]¶

Validate the QC parameters to ensure they are sensible.

Checks that: - speed_limit is non-negative. - min_win_period is non-negative. - max_win_period is greater than or equal to min_win_period.

Warnings are raised for any invalid parameters.

Return type:: bool
Returns:: bool – True if all parameters are valid, False otherwise.

marine_qc.quality_control.qc_grouped_reports module¶

QC of grouped reports.

Module containing QC functions for quality control of grouped marine reports.

class marine_qc.quality_control.qc_grouped_reports.SuperObsGrid[source]¶

Bases: object

Class for gridding data in buddy check, based on numpy arrays.

add_multiple_observations(lat, lon, value, date=None, month=None, day=None)[source]¶

Add a series of observations to the grid and take the grid average.

Parameters:

lat (SequenceNumberType) – 1-dimensional latitude array.
lon (SequenceNumberType) – 1-dimensional longitude array.
value (SequenceNumberType) – 1-dimensional anomaly array.
date (SequenceDatetimeType, optional) – 1-dimensional datetime array.
month (SequenceIntType, optional) – 1-dimensional month array. Used if date is not provided.
day (SequenceIntType, optional) – 1-dimensional day array. Used if date is not provided.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

Return type:

Notes

The observations should be anomalies.

add_single_observation(lat, lon, month, day, anom)[source]¶

Add an anomaly to the grid from specified lat lon and date.

Parameters:

lat (float) – Latitude of the observation in degrees.
lon (float) – Longitude of the observation in degrees.
month (int) – Month of the observation.
day (int) – Day of the observation.
anom (float) – Value to be added to the grid.

Return type:

Returns:

None – The function performs its operations in-place and does not return anything.

get_buddy_limits_with_parameters(pentad_stdev, limits, number_of_obs_thresholds, multipliers)[source]¶

Get buddy limits with parameters.

Parameters:

pentad_stdev (Climatology) – Climatology containing the 3-dimensional latitude array containing the standard deviations.
limits (list[list[int]]) – List of the limits.
number_of_obs_thresholds (list[list[int]]) – List containing the number of obs thresholds.
multipliers (list[list[float]]) – List containing the multipliers to be applied.

Return type:

Returns:

None – The function performs its operations in-place and does not return anything.

get_buddy_mean(lat, lon, month, day)[source]¶

Get the buddy mean from the grid for a specified time and place.

Parameters:

lat (float) – Latitude of the location for which the buddy mean is desired.
lon (float) – Longitude of the location for which the buddy mean is desired.
month (int) – Month for which the buddy mean is desired.
day (int) – Day for which the buddy mean is desired.

Return type:

float

Returns:

float – Buddy mean at the specified location.

get_buddy_stdev(lat, lon, month, day)[source]¶

Get the buddy standard deviation from the grid for a specified time and place.

Parameters:

lat (float) – Latitude of the location for which the buddy standard deviation is desired.
lon (float) – Longitude of the location for which the buddy standard deviation is desired.
month (int) – Month for which the buddy standard deviation is desired.
day (int) – Day for which the buddy standard deviation is desired.

Return type:

float

Returns:

float – Buddy standard deviation at the specified location.

get_neighbour_anomalies(search_radius, xindex, yindex, pindex)[source]¶

Search within a specified search radius of the given point and extract the neighbours for buddy check.

Parameters:

search_radius (list[int]) – Three element array search radius [lon, lat, time].
xindex (int) – The xindex of the gridcell to start from.
yindex (int) – The yindex of the gridcell to start from.
pindex (int) – The pindex of the gridcell to start from.

Return type:

tuple[list[float], list[float]]

Returns:

tuple of list of float – Anomalies and numbers of observations in two lists.

get_new_buddy_limits(stdev1, stdev2, stdev3, limits, sigma_m, noise_scaling)[source]¶

Get buddy limits for new bayesian buddy check.

Parameters:

stdev1 (Climatology) – Field of standard deviations representing standard deviation of difference between target gridcell and complete neighbour average (grid area to neighbourhood difference).
stdev2 (Climatology) – Field of standard deviations representing standard deviation of difference between a single observation and the target gridcell average (point to grid area difference).
stdev3 (Climatology) – Field of standard deviations representing standard deviation of difference between random neighbour gridcell and full neighbour average (uncertainty in neighbour average).
limits (list[int, int, int]) – Three membered list of number of degrees in latitude and longitude and number of pentads.
sigma_m (float) – Estimated measurement error uncertainty.
noise_scaling (float) – Scale noise by a factor of noise_scaling used to match observed variability.

Return type:

Returns:

None – The function performs its operations in-place and does not return anything.

Notes

The original default values for limits, sigma_m, and noise_scaling originally defaulted to:

limits = (2, 2, 4)
sigma_m = 1.0
noise_scaling = 3.0

take_average()[source]¶

Take the average of a grid to which reps have been added using add_rep.

Return type:: None

marine_qc.quality_control.qc_grouped_reports.get_threshold_multiplier(total_nobs, nob_limits, multiplier_values)[source]¶

Find the highest value of i such that total_nobs is greater than nob_limits[i] and return multiplier_values[i].

This routine is used by the buddy check. It’s a bit niche.

Parameters:

total_nobs (int) – Total number of neighbour observations.
nob_limits (list[int]) – List containing the limiting numbers of observations in ascending order first element must be zero.
multiplier_values (list[float]) – List containing the multiplier values associated..

Return type:

float

Returns:

float – The multiplier value.

marine_qc.quality_control.qc_individual_reports module¶

QC of individual reports.

Module containing main QC functions which could be applied on a DataBundle.

marine_qc.quality_control.qc_individual_reports.value_check(value, valid_flag=0, invalid_flag=1)[source]¶

Check if a value is equal to None or numerically invalid (NaN).

Parameters:

value (ValueNumberType) – The input value(s) to be tested. Can be a scalar, sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
valid_flag (int, default: 0) – Integer value how to flag valid values.
invalid_flag (int, default: 1) – Integer value how to flag invalid values.

Return type:

Returns:

ValueIntType – Same type as input, but with integer values

Returns 1 (or array/sequence/Series of 1s) if the input value is None or numerically invalid (NaN)
Returns 0 (or array/sequence/Series of 0s) otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.qc_multiple_checks module¶

Module containing base QC which call multiple QC functions and could be applied on a DataBundle.

marine_qc.quality_control.qc_sequential_reports module¶

QC of sequential reports.

Module containing QC functions for track checking which could be applied on a DataBundle.

marine_qc.quality_control.track_check_utils module¶

The New Track Check QC module provides the functions needed to perform the track check.

The main routine is mds_full_track_check which takes a list of class`.MarineReport` from a single ship and runs the track check on them. This is an update of the MDS system track check in that it assumes the Earth is a sphere. In practice, it gives similar results to the cylindrical earth formerly assumed.

marine_qc.quality_control.track_check_utils.backward_discrepancy(lat, lon, date, vsi, dsi)[source]¶

Calculate the distance between the projected position and the actual position.

The projected position is based on the reported speed and heading at the current and previous time steps. The calculation proceeds from the final, later observation to the first (in contrast to distr1 which runs in time order)

This takes the speed and direction reported by the ship and projects it forwards half a time step, it then projects it forwards another half-time step using the speed and direction for the next report, to which the projected location is then compared. The distances between the projected and actual locations is returned

Parameters:

lat (SequenceNumberType) – One-dimensional latitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (SequenceNumberType) – One-dimensional longitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
date (SequenceDatetimeType) – One-dimensional date array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
vsi (SequenceNumberType) – One-dimensional reported speed array in km/h. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
dsi (SequenceNumberType) – One-dimensional reported heading array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

SequenceFloatType

Returns:

SequenceFloatType – Same type as input, but with float values, shape (n,)

One-dimensional array, sequence, or pandas Series containing distances from estimated positions.

Raises:

ValueError – If either input is not 1-dimensional or if their lengths do not match.
TypeError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.track_check_utils.calculate_course_parameters(lat_later, lat_earlier, lon_later, lon_earlier, date_later, date_earlier)[source]¶

Calculate course parameters.

Parameters:

lat_later (float) – Latitude in degrees of later timestamp.
lat_earlier (float) – Latitude in degrees of earlier timestamp.
lon_later (float) – Longitude in degrees of later timestamp.
lon_earlier (float) – Longitude in degrees of earlier timestamp.
date_later (datetime) – Date of later timestamp.
date_earlier (datetime) – Date of earlier timestamp.

Return type:

tuple[float, float, float, float]

Returns:

tuple of float – A tuple of four floats representing the speed, distance, course and time difference.

marine_qc.quality_control.track_check_utils.calculate_midpoint(lat, lon, timediff)[source]¶

Interpolate between alternate reports and compare the interpolated location to the actual location.

E.g. take difference between reports 2 and 4 and interpolate to get an estimate for the position at the time of report 3. Then compare the estimated and actual positions at the time of report 3.

The calculation linearly interpolates the latitudes and longitudes (allowing for wrapping around the dateline and so on).

Parameters:

lat (SequenceNumberType) – One-dimensional latitude array in degrees.
lon (SequenceNumberType) – One-dimensional longitude array in degrees.
timediff (SequenceNumberType) – One-dimensional time difference array.

Return type:

Returns:

1D numpy.ndarray of float – One-dimensional array of distances from estimated positions in kilometers.

Raises:

ValueError – If either input is not 1-dimensional or if their lengths do not match.

marine_qc.quality_control.track_check_utils.calculate_speed_course_distance_time_difference(lat, lon, date, alternating=False)[source]¶

Calculate speeds, courses, distances and time differences using consecutive reports.

Parameters:

lat (SequenceNumberType) – One-dimensional latitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (SequenceNumberType) – One-dimensional longitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
date (SequenceDatetimeType) – One-dimensional date array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
alternating (bool, default: False) – Whether to use alternating reports for calculation.

Return type:

tuple[ndarray, ndarray, ndarray, ndarray]

Returns:

tuple of numpy.ndarray, each with float values, shape (n,) – A tuple containing four one-dimensional arrays representing: speed, distance, course, and time difference.

marine_qc.quality_control.track_check_utils.check_distance_from_estimate(vsi, time_differences, fwd_diff_from_estimated, rev_diff_from_estimated, vsi_previous=None)[source]¶

Check that distances from estimated positions are less than calculated distance.

The estimated positions are calculated forward and backwards in time. The calculated distance is the time difference multiplied by the average reported speeds.

Parameters:

vsi (SequenceNumberType) – Reported speed in km/h at current time step.
time_differences (SequenceNumberType) – Calculated time differences between reports in hours.
fwd_diff_from_estimated (SequenceNumberType) – Distance in km from estimated position, estimates made forward in time.
rev_diff_from_estimated (SequenceNumberType) – Distance in km from estimated position, estimates made backward in time.
vsi_previous (SequenceNumberType, optional) – One-dimensional array of reported speed in km/h at previous time step. If None, get vsi_previous from vsi.

Return type:

Returns:

numpy.ndarray – Returned array elements set to 10 if estimated and reported positions differ by more than the reported speed multiplied by the calculated time difference, 0 otherwise.

Raises:

TypeError – If inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.track_check_utils.direction_continuity(dsi, directions, dsi_previous=None, max_direction_change=60.0)[source]¶

Check if reported and calculated directions are within the allowed change.

This function compares the heading at the previous time step with the calculated ship direction from reported positions, flagging differences that exceed the maximum allowed direction change.

Parameters:

dsi (SequenceNumberType) – Heading at current time step in degrees.
directions (SequenceNumberType) – Calculated ship direction from reported positions in degrees.
dsi_previous (SequenceNumberType, optional) – Heading at previous time step in degrees. If None, get dsi_previous from dsi.
max_direction_change (float) – Largest deviations that will not be flagged in degrees.

Return type:

Returns:

numpy.ndarray – Returned array elements are 10.0 if the difference between reported and calculated direction is greater than the max_direction_change (default, 60 degrees), 0.0 otherwise.

marine_qc.quality_control.track_check_utils.forward_discrepancy(lat, lon, date, vsi, dsi)[source]¶

Calculate the distance between the projected position and the actual position.

The projected position is based on the reported speed and heading at the current and previous time steps. The observations are taken in time order.

This takes the speed and direction reported by the ship and projects it forwards half a time step, it then projects it forwards another half time-step using the speed and direction for the next report, to which the projected location is then compared. The distances between the projected and actual locations is returned

Parameters:

lat (SequenceNumberType) – One-dimensional latitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
lon (SequenceNumberType) – One-dimensional longitude array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
date (SequenceDatetimeType) – One-dimensional date array. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
vsi (SequenceNumberType) – One-dimensional reported speed array in km/h. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.
dsi (SequenceNumberType) – One-dimensional reported heading array in degrees. Can be a sequence (e.g., list or tuple), a one-dimensional NumPy array, or a pandas Series.

Return type:

SequenceFloatType

Returns:

SequenceFloatType – Same type as input, but with float values, shape (n,)

One-dimensional array, sequence, or pandas Series containing distances from estimated positions.

Raises:

ValueError – If either input is not 1-dimensional or if their lengths do not match.
TypeError – If decorator inspect_arrays does not return numpy ndarrays.

marine_qc.quality_control.track_check_utils.increment_position(alat1, alon1, avs, ads, timediff)[source]¶

Compute latitude and longitude increments over half a time interval.

This function takes latitudes and longitude, a speed, a direction and a time difference and returns increments of latitude and longitude which correspond to half the time difference.

Parameters:

alat1 (SequenceNumberType) – One-dimensional array of Latitude at starting point in degrees.
alon1 (SequenceNumberType) – One-dimensional array of Longitude at starting point in degrees.
avs (SequenceNumberType) – One-dimensional array of speed of ship in km/h.
ads (SequenceNumberType) – One-dimensional array of heading of ship in degrees.
timediff (SequenceNumberType) – One-dimensional array of time difference between the points in hours.

Return type:

tuple[ndarray, ndarray]

Returns:

1D numpy.ndarray of float – Returns latitude and longitude increment or None and None if timediff is None.

marine_qc.quality_control.track_check_utils.is_monotonic(inarr)[source]¶

Test if elements in an array are increasing monotonically.

I.e. each element is greater than or equal to the preceding element.

Parameters:: inarr (array-like of datetime, shape (n,)) – 1-dimensional date array.
Return type:: bool
Returns:: bool – True if array is increasing monotonically, False otherwise.

marine_qc.quality_control.track_check_utils.modal_speed(speeds)[source]¶

Calculate the modal speed from the input array in 3 knot bins.

Returns thebin-centre for the modal group.

The data are binned into 3-knot bins with the first from 0-3 knots having a bin centre of 1.5 and the highest containing all speed in excess of 33 knots with a bin centre of 34.5. The bin with the most speeds in it is found. The higher of the modal speed or 8.5 is returned:

Bins- 0-3, 3-6, 6-9, 9-12, 12-15, 15-18, 18-21, 21-24, 24-27, 27-30, 30-33, 33-36 Centres-1.5, 4.5, 7.5, 10.5, 13.5, 16.5, 19.5, 22.5, 25.5, 28.5, 31.5, 34.5

Parameters:: speeds (list) – Input speeds in km/h.
Return type:: float
Returns:: float – Bin-centre speed (expressed in km/h) for the 3 knot bin which contains most speeds in input array, or 8.5, whichever is higher.

marine_qc.quality_control.track_check_utils.set_speed_limits(amode)[source]¶

Take a modal speed and calculate speed limits for the track checker.

Parameters:: amode (float) – Modal speed in km/h.
Return type:: tuple[float, float, float]
Returns:: (float, float, float) – Max speed, maximum max speed and min speed.

marine_qc.quality_control.track_check_utils.speed_continuity(vsi, speeds, vsi_previous=None, max_speed_change=10.0)[source]¶

Check if reported speeds are within the allowed change from calculated speeds.

This function compares the reported speed at the current and previous time steps with the speed calculated from positions. Flags positions where the change exceeds the maximum allowed speed change.

Parameters:

vsi (SequenceNumberType) – One-dimensional array of reported speed in km/h at current time step.
speeds (SequenceNumberType) – One-dimensional array of speed of ship calculated from locations at current and previous time steps in km/h.
vsi_previous (SequenceNumberType, optional) – One-dimensional array of reported speed in km/h at previous time step. If None, get vsi_previous from vsi.
max_speed_change (float, optional) – Largest change of speed that will not raise flag in km/h, default 10 (km/h).

Return type:

Returns:

numpy.ndarray – Returned array elements are 10 if the reported and calculated speeds differ by more than 10 knots, 0 otherwise.

marine_qc.quality_control.track_check_utils.track_day_test(year, month, day, hour, lat, lon, elevdlim=-2.5)[source]¶

Given date, time, lat and lon calculate if the sun elevation is > elevdlim.

This is the “day” test used by tracking QC to decide whether an SST measurement is night or day. This is important because daytime diurnal heating can affect comparison with an SST background. It uses the function sunangle to calculate the elevation of the sun. A default solar_zenith angle of 92.5 degrees (elevation of -2.5 degrees) delimits night from day.

Parameters:

year (int) – Year.
month (int) – Month.
day (int) – Day.
hour (float) – Hour expressed as decimal fraction (e.g. 20.75 = 20:45 pm).
lat (float) – Latitude in degrees.
lon (float) – Longitude in degrees.
elevdlim (float, default: -2.5) – Elevation day/night delimiter in degrees above horizon.

Return type:

Returns:

bool – True if daytime, else False.

Raises:

ValueError – If either year, month, day, hour, lat or lon is numerically invalid or None of if either month, day, hour or lat is not in valid range.

marine_qc.quality_control.validations module¶

Module containing base QC which call multiple QC functions and could be applied on a DataBundle.

marine_qc.quality_control.validations.is_func_param(func, param)[source]¶

Return True if param is the name of a parameter of function func.

Parameters:

func (Callable) – Function whose parameters are to be inspected.
param (str) – Name of the parameter.

Return type:

Returns:

bool – Returns True if param is one of the functions parameters or the function uses **kwargs.

marine_qc.quality_control.validations.is_in_data(name, data)[source]¶

Return True if named column or variable, name, is in data.

Parameters:

name (str) – Name of variable.
data (pandas.Series or pandas.DataFrame) – Pandas Series or DataFrame to be tested.

Return type:

Returns:

bool – Returns True if name is one of the columns or variables in data, False otherwise.

Raises:

TypeError – If data type is not pandas Series or pandas DataFrame.

marine_qc.quality_control.validations.validate_arg(key, value, func_name, parameters, type_hints, reserved_keys, has_arguments)[source]¶

Validate argument against a function’s signature, taking decorators into account.

Parameters:

key (str) – The name of the argument to validate.
value (Any) – The value of the argument to validate.
func_name (str) – The name of the function (used in error message).
parameters (Mapping[str, inspect.Parameter]) – A mapping of parameter names to inspect.Parameter objects, typically from inspect.signature(func).parameters.
type_hints (Mapping[str, type]) – A mapping of parameter names to expected types, typically from typing.get_type_hints(func).
reserved_keys (set[str]) – Argument names that are considered reserved and should nor raise errors.
has_arguments (bool) – Whether the function accepts arbitrary arguments.

Return type:

marine_qc.quality_control.validations.validate_args(func, args=None, kwargs=None)[source]¶

Validate positional and keyword arguments against a function’s signature, taking decorators into account.

This function checks that: - All provided keyword arguments correspond to valid parameters of the given function. - All required parameters of the function (i.e., parameters without default values) are present in the provided keyword arguments.

Parameters:

func (Callable[..., Any]) – The function whose signature is used for validation.
args (Sequence[Any], optional) – Sequence of arguments intended to be passed to func.
kwargs (Mapping[str, Any], optional) – Dictionary of keyword arguments intended to be passed to func.

Raises:

ValueError – If kwargs contains a key that is not a parameter of func.
TypeError – If a required parameter of func is missing from kwargs.

Return type:

marine_qc.quality_control.validations.validate_dict(input_dict)[source]¶

Validate that the input is a dictionary with string keys and dictionary values.

This function checks that: - input_dict is a dictionary. - All keys in the dictionary are strings. - All top-level values in the dictionary are themselves dictionaries.

Parameters:: input_dict (Mapping[str, Mapping[str, Any]]) – The object to validate.
Raises:: TypeError – If input_dict is not a dictionary, if any key is not a string, or if any value is not a dictionary.
Return type:: None

marine_qc.quality_control.validations.validate_type(value, expected)[source]¶

Recursively validate that a value matches the expected type hint.

Parameters:

value (Any) – The value to validate.
expected (Any) – The expected value type for validation.

Return type: