Transform a CSV

The reshaping tax nobody budgets for

Data rarely arrives in the shape you need it. A supplier's price list has unit_cost where your system wants unit_price; a marketing export has first_name and last_name split across two columns when your CRM wants one full_name field; a raw export includes a dozen internal columns your import script has to strip before it will even run. None of this is complicated individually, but multiplied across every file, every partner, and every schema change, it becomes a quiet, recurring tax on anyone who touches the data pipeline.

What a transform rule actually does

Call POST /data/transform-csv with your file and a list of rules, and each rule performs one operation: rename a column, compute a new column from an expression referencing existing ones (like price times quantity to produce a line total), drop a column entirely, or reorder the output columns. Rules apply in the order you specify, so a computed column can reference a column renamed by an earlier rule, letting you chain several small operations into one coherent reshape instead of writing a custom script for each step.

Computed columns without a spreadsheet

The most useful transforms are often the simplest arithmetic a spreadsheet formula would do — margin as (price minus cost) divided by price, a full address concatenated from street, city and postal code, a boolean flag derived from a threshold. Expressing that as a rule instead of a formula in cell Z2 means it runs the same way every time, on every row, without a formula reference breaking when someone inserts a column upstream.

Why structure and meaning are separate problems

Renaming and reshaping is a different problem from validating or filtering data, even though spreadsheets tend to blur the two into one tangled set of formulas. Keeping structural transformation as its own explicit step — separate from deciding which rows qualify or what a value means — makes each part of a pipeline easier to reason about and to debug when a downstream system rejects a file: you know immediately whether the shape or the content is at fault.

Composing it into a pipeline

Because the task is asynchronous and returns a task_id immediately, transforming a CSV fits naturally as one stage among several — pull a file, transform its columns to match a target schema, then filter or merge it with another source. Partner integrations, recurring imports and schema migrations all benefit from expressing the reshape as declarative rules rather than a script that has to be rewritten every time a source format changes slightly.