System Overview

How CRUNCH takes messy data
and ships something clean.

Each row is a processing layer. Raw data enters at the top in any format and flows down through AI-powered analysis, a human review gate, distributed Databricks processing, and clean output delivery. Colored labels on the left identify each layer's role.

Ingest
📂
File Upload
Drop any CSV, Excel, or
Google Sheet up to 500MB
CSV XLSX Google Sheets
🔍
Schema Snapshot
Auto-detect encoding,
delimiter, and column count
UTF-8 Detect Auto Delimit
📊
Data Profile
Null count, type guesses,
and anomaly flags on load
Pre-Analysis Anomaly Flags
✅
Input Validator
Confirms file integrity
before AI processing begins
Integrity Check Error Report
AI Plan
🧠
Claude AI Planner
Reads every column header and samples each field. Maps data types, formats, and semantic intent. Generates a full transformation plan before a single cell is touched.
Anthropic API Claude Sonnet Structured Output Prompt Chaining
🏷️
Column Classifier
Maps semantic intent per
column: date, currency, name,
ID, category, free text
Semantic Map Type Inference
⚠️
Issue Detector
Flags nulls, mixed types,
bad formatting, casing
inconsistencies, and duplicates
Null Detection Type Conflicts
Review
👁️
Human Review Gate
Every proposed change is shown before execution. Approve the full plan, adjust individual suggestions, or override per column. You stay in control the whole time.
Dashboard UI Per-Column Control Approve / Override Plan Preview
🎛️
Override Controls
Set custom defaults, null
handling policy, and
per-column transformation rules
Custom Rules Null Policy
Process
⚡
Databricks Compute Engine
Distributed processing at enterprise scale. Executes the approved plan across the full dataset with parallel workloads, fault tolerance, and sub-minute completion for most files.
Databricks One Apache Spark Distributed Fault Tolerant
🔧
Normalize
Dates to ISO 8601. Currencies
to float. Names title-cased.
Casing and abbreviations unified.
ISO 8601 Title Case
🧩
Resolve
Nulls and blanks handled
by context. Inferred values
flagged and logged separately.
Context Fill Flagged Infer
Output
📄
CSV Export
Clean, UTF-8 encoded
comma-separated file
📊
Excel Export
Formatted .xlsx with
column type headers
🗂️
JSON Export
Structured records, ready
for APIs and pipelines
📋
Audit Trail
Every transformation logged
in a human-readable change log