dvc.yaml

dvc.yaml file

Type object
File match dvc.yaml
Schema URL https://catalog.lintel.tools/schemas/schemastore/dvc-yaml/latest.json
Source https://raw.githubusercontent.com/iterative/dvcyaml-schema/master/schema.json

Validate with Lintel

npx @lintel/lintel check
Type: object

Properties

vars

List of values for substitution.

May include any dict or a path to a params file which may be a string or a dict to params in the file).

Use elsewhere in dvc.yaml with the ${} substitution expression.

All of: Vars VarPath | VarDecl[]
stages Record<string, ForeachDo | Matrix | Stage>

List of stages that form a pipeline.

plots TopLevelPlots | TopLevelPlotsList

Top level plots definition.

params string[]

List of parameter files

uniqueItems=true
metrics string[]

List of metric files

uniqueItems=true
artifacts

Top level artifacts definition.

All of: TopLevelArtifacts object

Definitions

VarPath string

Path to params file with values for substitution.

VarDecl object

Dict of values for substitution.

Vars VarPath | VarDecl[]
DepModel string

Path to a dependency (input) file or directory for the stage.

Dependencies DepModel[]
ParamKey string
CustomParamFileKeys Record<string, ParamKey[]>
EmptyParamFileKeys Record<string, null>
Param ParamKey | CustomParamFileKeys | EmptyParamFileKeys
Params Param[]
OutFlags object
cache boolean

Cache output by DVC

Default: true
persist boolean

Persist output between runs

Default: false
checkpoint boolean

Indicate that the output is associated with in-code checkpoints

Default: false
desc string

User description for the output

type string

User assigned type of the output

labels string[]

User assigned labels of the output

uniqueItems=true
meta object

Custom metadata of the output.

remote string

Name of the remote to use for pushing/fetching

push boolean

Whether the output should be pushed to remote during dvc push

Default: true
Out string | object

Path to an output file or dir of the stage.

Outs Out[]
PlotFlags object
cache boolean

Cache output by DVC

Default: true
persist boolean

Persist output between runs

Default: false
checkpoint boolean

Indicate that the output is associated with in-code checkpoints

Default: false
desc string

User description for the output

type string

User assigned type of the output

labels string[]

User assigned labels of the output

uniqueItems=true
meta object

Custom metadata of the output.

remote string

Name of the remote to use for pushing/fetching

push boolean

Whether the output should be pushed to remote during dvc push

Default: true
x string

Default field name to use as x-axis data

y string

Default field name to use as y-axis data

x_label string

Default label for the x-axis

y_label string

Default label for the y-axis

title string

Default plot title

header boolean

Whether the target CSV or TSV has a header or not

Default: false
template string

Default plot template

Plot string | object

Path to plots file or dir of the stage.

Data files may be JSON/YAML/CSV/TSV.

Image files may be JPEG/GIF/PNG.

Plots Plot[]
Stage object

A named stage of a pipeline.

cmd string | string[] required

(Required) Command to run (anything your system terminal can run).

Can be a string or a list of commands.

wdir string

Working directory for the cmd, relative to dvc.yaml

deps

List of the dependencies for the stage.

All of: Dependencies DepModel[]
params

List of dot-separated parameter dependency keys to track from params.yaml.

May contain other YAML/JSON/TOML/Python parameter file names, with a sub-list of the param names to track in them (leave empty to include all).

All of: Params Param[]
outs

List of the outputs of the stage.

All of: Outs Out[]
metrics

List of metrics of the stage written to JSON/TOML/YAML.

All of: Outs Out[]
plots

List of plots of the stage for visualization.

Plots may be written to JSON/YAML/CSV/TSV for data or JPEG/GIF/PNG for images.

All of: Plots Plot[]
frozen boolean

Assume stage as unchanged

Default: false
always_changed boolean

Assume stage as always changed

Default: false
vars

List of stage-specific values for substitution.

May include any dict or a path to a params file.

Use in the stage with the ${} substitution expression.

All of: Vars VarPath | VarDecl[]
desc string

Description of the stage

meta

Additional information/metadata

ForeachDo object
foreach string | array | object required

Iterable to loop through in foreach. Can be a parametrized string, list or a dict.

The stages will be generated by iterating through this data, by substituting data in the do block.

do required

Parametrized stage definition that'll be substituted over for each of the value from the foreach data.

All of: Stage object
Matrix object

A named stage of a pipeline.

cmd string | string[] required

(Required) Command to run (anything your system terminal can run).

Can be a string or a list of commands.

matrix Record<string, array | string> required

Generate stages based on combination of variables.

The variable can be a list of values, or a parametrized string referencing a list.

wdir string

Working directory for the cmd, relative to dvc.yaml

deps

List of the dependencies for the stage.

All of: Dependencies DepModel[]
params

List of dot-separated parameter dependency keys to track from params.yaml.

May contain other YAML/JSON/TOML/Python parameter file names, with a sub-list of the param names to track in them (leave empty to include all).

All of: Params Param[]
outs

List of the outputs of the stage.

All of: Outs Out[]
metrics

List of metrics of the stage written to JSON/TOML/YAML.

All of: Outs Out[]
plots

List of plots of the stage for visualization.

Plots may be written to JSON/YAML/CSV/TSV for data or JPEG/GIF/PNG for images.

All of: Plots Plot[]
frozen boolean

Assume stage as unchanged

Default: false
always_changed boolean

Assume stage as always changed

Default: false
vars

List of stage-specific values for substitution.

May include any dict or a path to a params file.

Use in the stage with the ${} substitution expression.

All of: Vars VarPath | VarDecl[]
desc string

Description of the stage

meta

Additional information/metadata

X Record<string, string>
Y Record<string, string | string[]>
TopLevelPlotFlags object
x string | X

A single column name, or a dictionary of data-source and column pair

y string | string[] | Y

A single column name, list of columns, or a dictionary of data-source and column pair

x_label string

Default label for the x-axis

y_label string

Default label for the y-axis

title string

Default plot title

template string

Default plot template

Default: "linear"
EmptyTopLevelPlotFlags null
TopLevelPlots Record<string, TopLevelPlotFlags | EmptyTopLevelPlotFlags>
TopLevelPlotsList string | TopLevelPlots[]
TopLevelArtifactFlags object
path string required

Path to the artifact

type string

Type of the artifact

desc string

Description for the artifact

meta object

Custom metadata for the artifact

labels string[]

Labels for the artifact

uniqueItems=true
TopLevelArtifacts Record<string, object>