parq_tools.parq_concat

parq_concat.py

Utilities for concatenating Parquet files, supporting both row-wise (tall) and column-wise (wide) operations, with optional filtering, column selection, and progress tracking.

Main APIs:

  • concat_parquet_files: Concatenate multiple Parquet files into a single file, with flexible options for axis, filtering, and batching.

  • ParquetConcat: Class for advanced concatenation workflows, supporting batch processing, index alignment, and metadata handling.

Functions

concat_parquet_files

Concatenate multiple Parquet files into a single file, supporting both row-wise and column-wise concatenation.

Classes

ParquetConcat(files[, axis, index_columns, ...])

A utility for concatenating Parquet files while supporting axis-based merging, filtering, and progress tracking.