Ray

Ray autocluster configuration file

Type object
File match ray-*-cluster.yaml
Schema URL https://catalog.lintel.tools/schemas/schemastore/ray/latest.json
Source https://raw.githubusercontent.com/ray-project/ray/master/python/ray/autoscaler/ray-schema.json

Validate with Lintel

npx @lintel/lintel check
Type: object

Ray autoscaler schema

Properties

cluster_name string required

A unique identifier for the head node and workers of this cluster.

provider object required

Cloud-provider specific configuration.

24 nested properties
type string required

e.g. aws, azure, gcp,...

region string

e.g. us-east-1

module string

module, if using external node provider

head_ip string

gcp project id, if using gcp

worker_ips array

local cluster head node

use_internal_ips boolean

don't require public ips

namespace string

k8s namespace, if using k8s

location string

Azure location

resource_group string

Azure resource group

tags object

Azure user-defined tags

subscription_id string

Azure subscription id

msi_identity_id string

User-defined managed identity (generated by config)

msi_identity_principal_id string

User-defined managed identity principal id (generated by config)

subnet_id string

Network subnet id

autoscaler_service_account object

k8s autoscaler permissions, if using k8s

autoscaler_role object

k8s autoscaler permissions, if using k8s

autoscaler_role_binding object

k8s autoscaler permissions, if using k8s

cache_stopped_nodes boolean

Whether to try to reuse previously stopped nodes instead of launching nodes. This will also cause the autoscaler to stop nodes instead of terminating them. Only implemented for AWS.

availability_zone string

GCP availability zone

project_id string | null

GCP globally unique project id

security_group object

AWS security group

2 nested properties
GroupName string

Security group name

IpPermissions array

Security group in bound rules

disable_node_updaters boolean

Disables node updaters if set to True. Default is False. (For Kubernetes operator usage.)

gcp_credentials object

Credentials for authenticating with the GCP client

2 nested properties
type string required

Credentials type: either temporary OAuth 2.0 token or permanent service account credentials blob.

Values: "credentials_token" "service_account"
credentials string

Oauth token or JSON string constituting service account credentials

cloudwatch
max_workers integer

The maximum number of workers nodes to launch in addition to the head node. This should be no larger than the sum of min_workers for all available node types.

min=0
upscaling_speed number

The autoscaler will scale up the cluster faster with higher upscaling speed. E.g., if the task requires adding more nodes then autoscaler will gradually scale up the cluster in chunks of upscaling_speed*currently_running_nodes. This number should be > 0.

min=0
idle_timeout_minutes number

If a node is idle for this many minutes, it will be removed.

min=0
auth object

How Ray will authenticate with newly launched nodes.

4 nested properties
ssh_user string
Default: "ubuntu"
ssh_public_key string
ssh_private_key string
ssh_proxy_command string

A value for ProxyCommand ssh option, for connecting through proxies. Example: nc -x proxy.example.com:1234 %h %p

docker object

Docker configuration. If this is specified, all setup and start commands will be executed in the container.

11 nested properties
image string

the docker image name

Default: "rayproject/ray:latest"
container_name string
Default: "ray_docker"
pull_before_run boolean

run docker pull first

run_options array

shared options for starting head/worker docker

head_image string

image for head node, takes precedence over 'image' if specified

head_run_options array

head specific run options, appended to run_options

worker_image string

analogous to head_image

worker_run_options array

analogous to head_run_options

disable_automatic_runtime_detection boolean

disable Ray from automatically using the NVIDIA runtime if available

Default: false
disable_shm_size_detection boolean

disable Ray from automatically detecting /dev/shm size for the container

Default: false
use_podman boolean

Use 'podman' command in place of 'docker'

Default: false
head_node_type string

If using multiple node types, specifies the head node type.

file_mounts object

Map of remote paths to local paths, e.g. {"/tmp/data": "/my/local/data"}

cluster_synced_files array

List of paths on the head node which should sync to the worker nodes, e.g. ["/some/data/somehwere"]

file_mounts_sync_continuously boolean

If enabled, file mounts will sync continously between the head node and the worker nodes. The nodes will not re-run setup commands if only the contents of the file mounts folders change.

rsync_exclude array

File pattern to not sync up or down when using the rsync command. Matches the format of rsync's --exclude param.

rsync_filter array

Pattern files to lookup patterns to exclude when using rsync up or rsync down. This file is checked for recursively in all directories. For example, if .gitignore is provided here, the behavior will match git's .gitignore behavior.

metadata object

Metadata field that can be used to store user-defined data in the cluster config. Ray does not interpret these fields.

initialization_commands string[]
setup_commands string[]
head_setup_commands string[]
worker_setup_commands string[]
head_start_ray_commands string[]
worker_start_ray_commands string[]
no_restart

Whether to avoid restarting the cluster during updates. This field is controlled by the ray --no-restart flag and cannot be set by the user.

available_node_types object

A list of node types for multi-node-type autoscaling.

Definitions

commands string[]