StatsLibX logoStatsLibX

Preprocessing

v0.3.0

A class for data preprocessing and cleaning. Provides methods for null detection, missing value handling, scaling, standardization, filtering, outlier detection, type conversion, and comprehensive data quality reporting.

Class Overview

The Preprocessing class is the core module for data cleaning and transformation. It accepts a pandas.DataFrame as input and provides a rich set of methods for inspecting, describing, transforming, filtering, and cleaning your data.

Constructor

Constructor signature
Preprocessing(data: pd.DataFrame)
data : pd.DataFrame — Input data for preprocessing

Inspection

detect_nulls
detect_nulls(columns: str | list[str] | None = None) -> pd.DataFrame
check_uniqueness
check_uniqueness() -> pd.DataFrame
preview_data
preview_data(n: int = 5)

Description

describe_numeric
describe_numeric()
describe_categorical
describe_categorical()

Transformations

fill_nulls
fill_nulls(fill_with: Any, columns: str | list[str] | None = None)
normalize
normalize(column: str)
standardize
standardize(column: str)

Filtering

filter_rows
filter_rows(condition)
filter_columns
filter_columns(columns: list[str])
rename_columns
rename_columns(mapping: dict[str, str])

Outliers

detect_outliers
detect_outliers(column: str, method: str = 'iqr') -> pd.DataFrame

Data Quality

data_quality
data_quality() -> pd.DataFrame
change_dtypes
change_dtypes(columns: list[str] | str | None = None, from_type: str | None = None, to_type: str | None = None)
clean_data
clean_data(handle_missing: bool = False, missing_strategy: str = 'mean', fill_value=None, remove_duplicates: bool = False, convert_dtypes: bool = False, detect_outliers: bool = False, remove_outliers: bool = False, outlier_method: str = 'iqr', z_thresh: float = 3.0, scale: bool = False, scaling_method: str = 'standard', log_transform: bool = False, sqrt_transform: bool = False, drop_columns: list = None, keep_columns: list = None, analizer: bool = True, text_analizer: bool = False)