Nomad Cluster nomad_cluster
The nomad_cluster
resource allows you to create Nomad clusters as Docker containers. Clusters
can either be a single node combined server and client, or comprised of a dedicated server
and client nodes.
Properties
- Name
network
- Type
- (network_attachment: {})
- Required
- required
- Readonly
- Description
Network attaches the container to an existing network defined in a separate stanza. This block can be specified multiple times to attach the container to multiple networks.
- Name
client_nodes
- Type
- (int: 0)
- Required
- Readonly
- Description
Number of client nodes to create, if
0
Jumpppad will create a combined server and client. If greater than 0, Jumppad will creeat a dedicated server withn
clients.client_nodes
can be updated, if the value changes and the user runsjumppad up
, jumppad will attempt to nondestructively scale the cluster.
- Name
datacenter
- Type
- (string: dc1)
- Required
- Readonly
- Description
Nomad datacenter for the clients, defaults to
dc1
- Name
image
- Type
- (image: {})
- Required
- Readonly
- Description
Image defines a Docker image to use when creating the container. By default the nomad cluster resource will be created using the latest Jumppad container image.
- Name
environment
- Type
- (map[string]string: map[]{})
- Required
- Readonly
- Description
An env stanza allows you to set environment variables in the container. This stanza can be specified multiple times.
env { key = "PATH" value = "/usr/local/bin" }
- Name
server_config
- Type
- (string: "")
- Required
- Readonly
- Description
Path to a file containing custom Nomad server config to use when creating the server. Note: Only added to server nodes
This file extends the default server configuration and is mounted at the path
/etc/nomad.d/server_user_config.hcl
on server nodes.server { enabled = true bootstrap_expect = 1 } client { enabled = true meta { node_type = "server" } %s } plugin "raw_exec" { config { enabled = true } }
- Name
client_config
- Type
- (string: "")
- Required
- Readonly
- Description
Path to a file containing custom Nomad client config to use when creating the server. Note: This file is added to both server and clients nodes.
This file extends the default client config and is mounted at the path /etc/nomad.d/client_user_config.hcl
client { enabled = true server_join { retry_join = ["%s"] } } plugin "raw_exec" { config { enabled = true } }
- Name
consul_config
- Type
- (string: "")
- Required
- Readonly
- Description
Path to a file containing custom Consul agent config to use when creating the client.
- Name
volume
- Type
- (volume: {})
- Required
- Readonly
- Description
Additional volume to mount to the server and client nodes.
volume { source = "./mydirectory" destination = "/path_in_container" }
- Name
copy_image
- Type
- (image: {})
- Required
- Readonly
- Description
Docker image in the local Docker image cache to copy to the cluster on creation. This image is added to the Nomad clients docker cache enabling jobs to use images that may not be in the local registry.
Jumppad tracks changes to copied images, should the image change running
jumppad up
would push any changes to the cluster automatically.copy_image { name = "mylocalimage:versoin" }
- Name
port
- Type
- (port: {})
- Required
- Readonly
- Description
A port stanza allows you to expose container ports on the local network or host. This stanza can be specified multiple times.
port { local = 80 host = 8080 }
- Name
port_range
- Type
- (port_range: {})
- Required
- Readonly
- Description
A port_range stanza allows you to expose a range of container ports on the local network or host. This stanza can be specified multiple times.
The following example would create 11 ports from 80 to 90 (inclusive) and expose them to the host machine.
port { range = "80-90" enable_host = true }
- Name
api_port
- Type
- (int: 4646)
- Required
- Readonly
- Description
Port to expose the Nomad API on the host. By default this uses the standard nomad port
4646
; however, if you are running multiple nomad instances you will need to override this value.
- Name
connector_port
- Type
- (int: )
- Required
- Readonly
- readonly
- Description
The port where the Jumppad connector is exposed to the host, this property is requied by the
ingress
resource and is not generally needed when building blueprints.
- Name
config_dir
- Type
- (string: "")
- Required
- Readonly
- readonly
- Description
Local directory where the server and client configuration is stored.
- Name
server_fqrn
- Type
- (string: "")
- Required
- Readonly
- readonly
- Description
The fully qualified resource name for the Nomad server, this value can be used to address the server from the Docker network. It is also the name of the Docker container.
server.name.nomad-cluster.jumpad.dev
- Name
client_fqrn
- Type
- ([]string: [])
- Required
- Readonly
- readonly
- Description
The fully qualified resource names for the Nomad clients, this value can be used to address the client from the Docker network. It is also the name of the Docker container.
When
client_nodes
is set to0
this property will have no value.[ abse42wsdff.client.name.nomad-cluster.jumpad.dev, kjdf23123.client.name.nomad-cluster.jumpad.dev, 123dfkjs.client.name.nomad-cluster.jumpad.dev, ]
- Name
external_ip
- Type
- (string: "")
- Required
- Readonly
- readonly
- Description
Local IP address of the Nomad server, this property can be used to set the
NOAMD_ADDR
on the Jumppad client.output "NOMAD_ADDR" { value = "http://${resource.nomad_cluster.dev.external_ip}:${resource.nomad_cluster.dev.api_port}" }
network_attachment
Network attachment defines a network to which the container is attached.
- Name
id
- Type
- (string: "")
- Required
- required
- Readonly
- Description
ID of the network to attach the container to, specified in reference format. e.g. to attach to a network called
cloud
network { id = "network.cloud" }
- Name
ip_address
- Type
- (string: "")
- Required
- Readonly
- Description
Static IP address to assign container for the network, the ip address must be within range defined by the network subnet. If this parameter is omitted an IP address will be automatically assigned.
- Name
aliases
- Type
- ([]string: [])
- Required
- Readonly
- Description
Aliases allow alternate names to specified for the container. Aliases can be used to reference a container across the network, the container will respond to ping and other network resolution using the primary assigned name
[name].container.shipyard.run
and the aliases.network { name = "network.cloud" aliases = [ "alt1.container.local.jmpd.in", "alt2.container.local.jmpd.in" ] }
- Name
name
- Type
- (string: "")
- Required
- Readonly
- readonly
- Description
Name will equal the name of the network as created by jumppad.
- Name
assigned_address
- Type
- (string: "")
- Required
- Readonly
- readonly
- Description
assigned_address will equal the assigned IP address for the network. This will equal ip_address if set; otherwise, this is the automatically assigned ip_address.
image
Image defines a Docker image used when creating this container. An Image can be stored in a public or a private repository.
- Name
name
- Type
- (string: "")
- Required
- required
- Readonly
- Description
Name of the image to use when creating the container, can either be the full canonical name or short name for Docker official images. e.g.
consul:v1.6.1
ordocker.io/consul:v1.6.1
.
- Name
username
- Type
- (string: "")
- Required
- Readonly
- Description
Username to use when connecting to a private image repository
- Name
password
- Type
- (string: "")
- Required
- Readonly
- Description
Password to use when connecting to a private image repository, for both username and password interpolated environment variables can be used in place of static values.
image { name = "myregistry.io/myimage:latest" username = env("REGISTRY_USERNAME") password = env("REGISTRY_PASSWORD") }
volume
A volume type allows the specification of an attached volume.
- Name
source
- Type
- (string: "")
- Required
- required
- Readonly
- Description
The source volume to mount in the container, can be specified as a relative
./
or absolute path/usr/local/bin
. Relative paths are relative to the file declaring the container.
- Name
destination
- Type
- (string: "")
- Required
- required
- Readonly
- Description
The destination in the container to mount the volume to, must be an absolute path.
- Name
type
- Type
- (string: bind)
- Required
- Readonly
- Description
The type of the mount, can be one of the following values:
- bind: bind the source path to the destination path in the container
- volume: source is a Docker volume
- tmpfs: create a temporary filesystem
- Name
bind_propagation
- Type
- (string: rprivate)
- Required
- Readonly
- Description
Configures bind propagation for Docker volume mounts, only applies to bind mounts, can be one of the following values:
- shared
- slave
- private
- rslave
- rprivate
For more information please see the Docker documentation https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation
- Name
bind_propagation_non_recursive
- Type
- (boolean: false)
- Required
- Readonly
- Description
Configures recursiveness of the bind mount. By default Docker mounts with the equivalent of
mount --rbind
meaning that mounts below the the source directory are visible in the container. For instance runningdocker run --rm --mount type=bind,src=/,target=/host,readonly busybox
will make/run
of the host available as/host/run
in the container -- and to make matters even worse it will be writable (since only the toplevel bind is set readonly, not the children). Ifbind_propagation_non_recursive
is set totrue
then the container will only see an empty/host/run
, meaning thetmpfs
which is typically mounted to/run
on the host is not propagated into the container.
- Name
selinux_relabel
- Type
- (string: "")
- Required
- Readonly
- Description
Configures Selinux relabeling for the container (usually specified as :z or :Z) and can be one of the following values:
- shared (Equivalent to :z)
- private (Equivalent to :Z)
port
A port stanza defines host to container communications
- Name
local
- Type
- (string: "")
- Required
- required
- Readonly
- Description
The local port in the container.
- Name
host
- Type
- (string: "")
- Required
- required
- Readonly
- Description
The host port to map the local port to.
- Name
protocol
- Type
- (string: "")
- Required
- Readonly
- Description
The protocol to use when exposing the port, can be "tcp", or "udp".
- Name
open_in_browser
- Type
- (string: /)
- Required
- Readonly
- Description
Should a browser window be automatically opened when this resource is created. Browser windows will open at the path specified by this property.
port_range
A port_range stanza defines host to container communications by exposing a range of ports for the container.
- Name
range
- Type
- (string: "")
- Required
- required
- Readonly
- Description
The port range to expose, e.g,
8080-8082
would expose the ports8080
,8081
,8082
.
- Name
enable_host
- Type
- (bool: false)
- Required
- Readonly
- Description
The host port to map the local port to.
- Name
protocol
- Type
- (string: tcp)
- Required
- Readonly
- Description
The protocol to use when exposing the port, can be "tcp", or "udp".
Meta Properties
In addition to the main properties, all resources have meta
properties, such
as the id
of the resource. To see the list of these properties please see the
Meta Properties
section in the documentation /docs/resources/meta.
Examples
Minimal example
The following example creates a combined client and server running in a single container.
resource "network" "cloud" {
subnet = "10.10.0.0/16"
}
resource "nomad_cluster" "dev" {
network {
id = resource.network.cloud.meta.id
}
}
Full Example
The following example creates a multi_node nomad cluster with a job and ingress.
resource "network" "cloud" {
subnet = "10.10.0.0/16"
}
resource "nomad_cluster" "dev" {
client_nodes=3
network {
id = resource.network.cloud.meta.id
}
}
resource "nomad_job" "example_1" {
cluster = resource.nomad_cluster.dev
paths = ["./app_config/example1.nomad"]
health_check {
timeout = "60s"
nomad_jobs = ["example_1"]
}
}
resource "ingress" "fake_service_1" {
port = 19090
target {
resource = resource.nomad_cluster.dev
named_port = "http"
config = {
job = "example_1"
group = "fake_service"
task = "fake_service"
}
}
}
Image Caching
Nomad clusters do not share the local machines Docker image cache. Each node in a cluster has it's own unqiue cache.
To save bandwidth all containers launched in the Nomad cluster pulled through an image cache that runs in Docker. After the first pull all images are subsequently pulled from the image cache not the public internet. This cache is global to all Nomad and Kubernetes clusters created with Jumppad.
For more information on the image cache see the container_registry resource.