quantiles#

class pdcleaner.detection.basic.quantiles(obj, detector=None, lowerq=0, upperq=1, inclusive='both')[source]#

Detect errors values in a Series using quantiles.

Intended to be used by the detect method with the keyword ‘quantiles’

>>> series.cleaner.detect.quantiles(...)
>>> series.cleaner.detect('quantiles',...)

This detection method flags values as errors wherever the corresponding Series element is outside the range between the values at given quantiles

Notes

NA values are not treated as errors.

Parameters:

lowerq (float (Default = 0)) – The lower quantile, which can lie in range: 0 <= lowerq <= 1.
upperq (float (Default = 1)) – The upper quantile, which can lie in range: 0 <= upperq <= 1.
inclusive ({“both”, “neither”, “left”, “right”}, default "both") – Include boundaries. Whether to set each bound as closed or open.

Raises:

Examples

>>> s = pd.Series([0, 0, 0, 0, -1, 1, -1, 1, -5, 5])
>>> q_errors = s.cleaner.detect.quantiles(lowerq=.1, upperq=.9)
>>> q_errors.n_errors
2

Attributes Summary

`inclusive`	Keyword to indicate if boundaries are included {“both”, “neither”, “left”, “right”}
`index`	Indices of the rows detected as errors
`lower`	Lower bound
`lowerq`	Lower quantile value
`n_errors`	Number of rows detected as errors
`name`
`obj`	The object (Series or DataFrame) containing the data to which the detection is applied
`sided`	Keyword to indicate if detection is one side or both {"both", "right", "left"}
`upper`	Upper bound
`upperq`	Upper quantile value

Methods Summary

`detected`()	Series or DataFrame containing only the detected errors
`has_errors`()	Returns True if any error has been detected, False otherwise
`is_error`()	Return a boolean same-sized object indicating if the values are flagged as errors
`not_error`()	Return a boolean same-sized object indicating if the values are NOT flagged as errors
`plot`([color, errors_color, compact, limits, ...])	plot a visualization representing an overview of the treated data and colored according to the validity of the values:
`report`()	prints a detection report
`valid`()	Series or DataFrame containing only the valid values

Attributes Documentation

inclusive#: Keyword to indicate if boundaries are included {“both”, “neither”, “left”, “right”}

obj#: The object (Series or DataFrame) containing the data to which the detection is applied

sided#: Keyword to indicate if detection is one side or both {“both”, “right”, “left”}

Methods Documentation

has_errors() → bool#: Returns True if any error has been detected, False otherwise

is_error() → Series#: Return a boolean same-sized object indicating if the values are flagged as errors

not_error() → Series#: Return a boolean same-sized object indicating if the values are NOT flagged as errors

plot(color='green', errors_color='red', compact=False, limits=True, figsize=None)#

plot a visualization representing an overview of the treated data and colored according to the validity of the values:

Parameters:

color (palette name (Default: "green")) – Color associated to legitimate values. Should be something that can be interpreted by seaborn’s color_palette()
errors_color (palette name (Default: "red")) – Color associated to erroneous values. Should be something that can be interpreted by seaborn’s color_palette()
compact (Bool (Default: False)) – If True, compact the plots around valid values and show the number of erroneous values on the scatter plot
limits (Bool (Default: True)) – If True, draw horizontal lines showing the lower and upper values delimiting the allowed values
figsize ((float, float) (Default: None)) – width and height of the figure.

Returns:

axs – an array of length 4 containing the matplotlib axes representing the plots

Return type:

array of matplotlib.axes._subplots.AxesSubplot

Examples

>>> series = pd.Series([-5, 1, 2 , 3, 8, 12])
>>> detector = series.cleaner.detect.bounded(lower=0, upper=10)
>>> detector.plot()