User Guide

This guide provides detailed information on how to use df-eval for various data transformation tasks.

Getting Started

If you haven’t installed df-eval yet, see the Installation section in the main documentation.

Guide Contents

• Basic Usage
  • Creating an Engine
  • Simple Expression Evaluation
  • Schema-Driven Derived Columns
  • Using Built-in Functions
  • Batch Evaluation
  • Specifying Data Types
  • Error Handling
  • Next Steps
• Advanced Usage
  • Dependency Management
  • Provenance Tracking
  • Custom Functions (UDFs)
  • Custom Constants
  • Out-of-Memory Parquet Workflows
  • Pandera Integration and Schema IO
• Lookups
  • Overview
  • Basic Lookup Usage
  • Resolver Types
  • Caching
  • Missing Value Handling
  • Using Lookups in Pipelines
  • Chaining Lookups
  • Batch Lookups
  • Error Handling
  • Best Practices
  • Next Steps

Featured Examples

The following gallery examples provide end-to-end, runnable walkthroughs of common df-eval workflows:

• Engine Lookups with Resolvers
• Pandera Schema Integration
• Lookup and Function Pipeline via Pandera
• Out-of-Memory Parquet Processing

What You’ll Learn

• Basic Usage: Learn the fundamentals of expression evaluation and schema-driven transformations

• Advanced Usage: Explore dependency management, provenance tracking, custom functions, and Parquet streaming

• Lookups: Master external data lookups and resolver patterns

Quick Reference

Creating an Engine

from df_eval import Engine

engine = Engine()

Basic Evaluation

result = engine.evaluate(df, "a + b")

Schema Application

schema = {"derived_col": "a * 2"}
result = engine.apply_schema(df, schema)

Next Steps

Start with Basic Usage to learn the fundamentals, then progress through the guide at your own pace.

For runnable examples, see Examples.