DataYoga Job

Declarative definition of sequential pipeline steps within a DataYoga job

Type object
File match **/jobs/**/*.dy.yaml
Schema URL https://catalog.lintel.tools/schemas/schemastore/datayoga-job/latest.json
Source https://raw.githubusercontent.com/datayoga-io/datayoga/main/schemas/job.schema.json

Validate with Lintel

npx @lintel/lintel check
Type: object

Job descriptor

Properties

error_handling string

Error handling strategy: abort - terminate job, ignore - skip

Default: "ignore"
Values: "abort" "ignore"
input object
All of: variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant, variant
1 nested properties
uses enum
Values: "add_field" "azure.read_event_hub" "cassandra.write" "files.read_csv" "filter" "http.receiver" "http.write" "jinja_template" "map" "parquet.read" "parquet.write" "redis.lookup" "redis.read_stream" "redis.write" "relational.lookup" "relational.read" "relational.write" "remove_field" "rename_field" "sequence" "std.read" "std.write"
steps block[]

Definitions

block object
uses enum
Values: "add_field" "azure.read_event_hub" "cassandra.write" "files.read_csv" "filter" "http.receiver" "http.write" "jinja_template" "map" "parquet.read" "parquet.write" "redis.lookup" "redis.read_stream" "redis.write" "relational.lookup" "relational.read" "relational.write" "remove_field" "rename_field" "sequence" "std.read" "std.write"