Pipelines

A pipeline connects inputs, processing steps, and outputs. Use it to repeat imports, data preparation, model training, prediction, or exports. A pipeline definition describes the work. A run is one execution of that definition. Inspect the run that produced a result when investigating errors or changes.

Understand the canvas

Node	Purpose
Dataset	Supplies input data or receives a step’s output.
Action	Performs an operation such as importing, transforming, training, or predicting.
Model	Holds a trained model that later steps can use.

Connections between nodes establish the flow of data. Check the configured dataset and model names as well as the visible node labels.

Create a pipeline

Use Create → Pipeline in Workspace, or ask Dawn to create one from a conversation. Attach the relevant datasets and connections first.

Build a pipeline that imports this month's orders, checks for duplicate order
IDs, and creates a daily_sales dataset. Keep the source data separate.
Explain the steps and output names before running it.

Review the generated canvas and each action’s configuration. For manual editing, use the available node controls to add datasets, actions, and model outputs, then connect the required inputs. See data engineering for a preparation walkthrough and modelling for a training workflow.

Choose the right operation

Goal	Pipeline operation
Bring in source data	Integration import using a connection and selection settings
Clean, join, or reshape data	Data transformation using SQL or Python
Predict a category or number	Model training, followed by inference
Compare predictions with actual outcomes	Model evaluation
Understand model behaviour	Model explanation, where supported
Classify or enrich text	LLM inference or embeddings, depending on the task
Write results to an external destination	Export using appropriate destination permissions

An import and an export require different access. Review the destination and write behaviour before running an export.

Save and run

Use Save before Run. The pipeline also has Code, Info, MLOps, History, and Logs tabs. Open a step to inspect its Configuration and Logs. Use Run this step or Run from this step when you need a smaller rerun, and watch its status. For a subset run, make sure any upstream datasets and models already exist and contain the versions you intend to use. A completed run means the configured steps finished. It does not establish that the business logic is correct. Open the output datasets or models and compare their contents with your expectations.

Troubleshoot a failed run

Open the run history or logs for the failed run.
Find the first failed step; downstream steps may not have run because of it.
Check that step’s configuration, inputs, and error message.
Add the pipeline and relevant error details to a conversation for help.
Review the proposed change, save it, and rerun the appropriate steps.

Explain this pipeline failure using the attached logs. Identify the first
failed step and suggest the smallest correction. Check whether any outputs
were produced before recommending a rerun.

Error pattern	Likely starting point
Authentication or permission failure	Connection credential and source-side permissions
Missing column or table	Source schema, dataset names, and selected inputs
Type conversion failure	Unexpected values, nulls, dates, or numeric formats
Empty result	Filters, source date range, and source availability
Model input mismatch	Feature names, types, and preparation used for training

Schedule recurring work

Open Info and use Schedule to configure recurring runs after a successful manual run and validation of the output. Save the schedule with Save changes. Check its timezone, intended input period, and expected duration. Avoid overlapping runs unless the workflow is designed for them. Use a Job when the recurring work is an instruction to an agent, such as reviewing results and writing commentary. Use an Automation when you need a visual workflow of actions. These are distinct from the data-processing steps in a pipeline.

Get started

Build with Dawn

Manage Dawn

Understand the canvas

Create a pipeline

Choose the right operation

Save and run

Troubleshoot a failed run

Schedule recurring work

​Understand the canvas

​Create a pipeline

​Choose the right operation

​Save and run

​Troubleshoot a failed run

​Schedule recurring work

Understand the canvas

Create a pipeline

Choose the right operation

Save and run

Troubleshoot a failed run

Schedule recurring work