Customizations#
SaQC comes with a continuously growing number of pre-implemented quality-checking and processing routines as well as flagging schemes. For a sufficiently large use case, however, it might be necessary to extend the system anyhow. The main extension options, namely quality check routines and the flagging scheme. Both of these mechanisms are described within this document.
Custom Quality Check Routines#
In case you are missing quality check routines, you are, of course, very welcome to file a feature request issue on the project’s GitLab repository. However, if you are more the “I-get-this-done-by-myself” type of person, SaQC offers the possibility to directly extend its functionality using its interface to the evaluation machinery.
In order to make a function usable within the evaluation framework of SaQC, it needs to implement the following function interface:
import saqc
def yourTestFunction(qc: SaQC, field: str | list[str], *args, **kwargs) -> SaQC:
# your code
return qc
with the following parameters
Name |
Description |
---|---|
|
An instance of |
|
The field(s)/column(s) of |
|
Any number of named arguments needed to parameterize the function. |
|
Any number of named keyword arguments needed to parameterize the function. |
Integrate into SaQC#
SaQC provides two decorators, @flagging()
and @register()
, to integrate custom functions
into its workflow. The choice between them depends on the nature of your algorithm. @register()
is a more versatile decorator, allowing you to handle masking, demasking, and squeezing of data and flags, while
@flagging()
is simpler and suitable for univariate flagging functions without the need for complex
data manipulations.
Use @flagging()
for simple univariate flagging tasks without the need for complex data manipulations.
@flagging()
is especially suitable when your algorithm operates on a single column
from saqc import SaQC
from saqc.core.register import flagging
@flagging()
def simpleFlagging(saqc: SaQC, field: str | list[str], param1: ..., param2: ..., **kwargs) -> SaQC:
"""
Your simple univariate flagging logic goes here.
Parameters
----------
saqc : SaQC
The SaQC instance.
field : str
The field or fields on which to apply anomaly detection.
param1 : ...
Additional parameters needed for your algorithm.
param2 : ...
Additional parameters needed for your algorithm.
Returns
-------
SaQC
The modified SaQC instance.
"""
# Your flagging logic here
# Modify saqc._flags as needed
return saqc
Use @register()
when your algorithm needs to handle multiple columns simultaneously (multivariate=True
)
and or you need explicit control over masking, demasking, and squeezing of data and flags.
register()
is especially for complex algorithms that involve interactions between different columns.
from saqc import SaQC
from saqc.core.register import register
@register(
mask=["field"], # Parameter(s) of the decorated functions giving the names of columns in SaQC._data to mask before the call
demask=["field"], # Parameter(s) of the decorated functions giving the names of columns in SaQC._data to unmask after the call
squeeze=["field"], # Parameter(s) of the decorated functions giving the names of columns in SaQC._flags to squeeze into a single flags column after the call
multivariate=True, # Set to True to handle multiple columns
handles_target=False,
)
def complexAlgorithm(
saqc: SaQC, field: str | list[str], param1: ..., param2: ..., **kwargs
) -> SaQC:
"""
Your custom anomaly detection logic goes here.
Parameters
----------
saqc : SaQC
The SaQC instance.
field : str or list of str
The field or fields on which to apply anomaly detection.
param1 : ...
Additional parameters needed for your algorithm.
param2 : ...
Additional parameters needed for your algorithm.
Returns
-------
SaQC
The modified SaQC instance.
"""
# Your anomaly detection logic here
# Modify saqc._flags and saqc._data as needed
return saqc
Custom flagging schemes#
Sorry for the inconvenience! Coming soon…