Nomad Cluster nomad_cluster

The nomad_cluster resource allows you to create Nomad clusters as Docker containers. Clusters can either be a single node combined server and client, or comprised of a dedicated server and client nodes.

Properties

  • Name
    network
    Type
    (network_attachment: {})
    Required
    required
    Readonly
    Description

    Network attaches the container to an existing network defined in a separate stanza. This block can be specified multiple times to attach the container to multiple networks.

  • Name
    client_nodes
    Type
    (int: 0)
    Required
    Readonly
    Description

    Number of client nodes to create, if 0 Jumpppad will create a combined server and client. If greater than 0, Jumppad will creeat a dedicated server with n clients. client_nodes can be updated, if the value changes and the user runs jumppad up, jumppad will attempt to nondestructively scale the cluster.

  • Name
    datacenter
    Type
    (string: dc1)
    Required
    Readonly
    Description

    Nomad datacenter for the clients, defaults to dc1

  • Name
    image
    Type
    (image: {})
    Required
    Readonly
    Description

    Image defines a Docker image to use when creating the container. By default the nomad cluster resource will be created using the latest Jumppad container image.

  • Name
    environment
    Type
    (map[string]string: map[]{})
    Required
    Readonly
    Description

    An env stanza allows you to set environment variables in the container. This stanza can be specified multiple times.

    env {
      key   = "PATH"
      value = "/usr/local/bin"
    }
    
  • Name
    server_config
    Type
    (string: "")
    Required
    Readonly
    Description

    Path to a file containing custom Nomad server config to use when creating the server. Note: Only added to server nodes

    This file extends the default server configuration and is mounted at the path /etc/nomad.d/server_user_config.hcl on server nodes.

    server {
      enabled = true
      bootstrap_expect = 1
    }
    
    client {
    	enabled = true
    	meta {
    		node_type = "server"
    	}
    	%s
    }
    
    plugin "raw_exec" {
      config {
    		enabled = true
      }
    }
    
  • Name
    client_config
    Type
    (string: "")
    Required
    Readonly
    Description

    Path to a file containing custom Nomad client config to use when creating the server. Note: This file is added to both server and clients nodes.

    This file extends the default client config and is mounted at the path /etc/nomad.d/client_user_config.hcl

    client {
    	enabled = true
    
    	server_join {
    		retry_join = ["%s"]
    	}
    }
    
    plugin "raw_exec" {
      config {
    		enabled = true
      }
    }
    
  • Name
    consul_config
    Type
    (string: "")
    Required
    Readonly
    Description

    Path to a file containing custom Consul agent config to use when creating the client.

  • Name
    volume
    Type
    (volume: {})
    Required
    Readonly
    Description

    Additional volume to mount to the server and client nodes.

    volume {
      source = "./mydirectory"
      destination = "/path_in_container"
    }
    
  • Name
    copy_image
    Type
    (image: {})
    Required
    Readonly
    Description

    Docker image in the local Docker image cache to copy to the cluster on creation. This image is added to the Nomad clients docker cache enabling jobs to use images that may not be in the local registry.

    Jumppad tracks changes to copied images, should the image change running jumppad up would push any changes to the cluster automatically.

    copy_image {
      name = "mylocalimage:versoin"
    }
    
  • Name
    port
    Type
    (port: {})
    Required
    Readonly
    Description

    A port stanza allows you to expose container ports on the local network or host. This stanza can be specified multiple times.

    port {
      local = 80
      host  = 8080
    }
    
  • Name
    port_range
    Type
    (port_range: {})
    Required
    Readonly
    Description

    A port_range stanza allows you to expose a range of container ports on the local network or host. This stanza can be specified multiple times.

    The following example would create 11 ports from 80 to 90 (inclusive) and expose them to the host machine.

    port {
      range       = "80-90"
      enable_host = true
    }
    
  • Name
    api_port
    Type
    (int: 4646)
    Required
    Readonly
    Description

    Port to expose the Nomad API on the host. By default this uses the standard nomad port 4646; however, if you are running multiple nomad instances you will need to override this value.

  • Name
    connector_port
    Type
    (int: )
    Required
    Readonly
    readonly
    Description

    The port where the Jumppad connector is exposed to the host, this property is requied by the ingress resource and is not generally needed when building blueprints.

  • Name
    config_dir
    Type
    (string: "")
    Required
    Readonly
    readonly
    Description

    Local directory where the server and client configuration is stored.

  • Name
    server_fqrn
    Type
    (string: "")
    Required
    Readonly
    readonly
    Description

    The fully qualified resource name for the Nomad server, this value can be used to address the server from the Docker network. It is also the name of the Docker container.

      server.name.nomad-cluster.jumpad.dev
    
  • Name
    client_fqrn
    Type
    ([]string: [])
    Required
    Readonly
    readonly
    Description

    The fully qualified resource names for the Nomad clients, this value can be used to address the client from the Docker network. It is also the name of the Docker container.

    When client_nodes is set to 0 this property will have no value.

    [
      abse42wsdff.client.name.nomad-cluster.jumpad.dev,
      kjdf23123.client.name.nomad-cluster.jumpad.dev,
      123dfkjs.client.name.nomad-cluster.jumpad.dev,
    ]
    
  • Name
    external_ip
    Type
    (string: "")
    Required
    Readonly
    readonly
    Description

    Local IP address of the Nomad server, this property can be used to set the NOAMD_ADDR on the Jumppad client.

      output "NOMAD_ADDR" {
        value = "http://${resource.nomad_cluster.dev.external_ip}:${resource.nomad_cluster.dev.api_port}"
      }
    

network_attachment

Network attachment defines a network to which the container is attached.

  • Name
    id
    Type
    (string: "")
    Required
    required
    Readonly
    Description

    ID of the network to attach the container to, specified in reference format. e.g. to attach to a network called cloud

    network {
      id = "network.cloud"
    }
    
  • Name
    ip_address
    Type
    (string: "")
    Required
    Readonly
    Description

    Static IP address to assign container for the network, the ip address must be within range defined by the network subnet. If this parameter is omitted an IP address will be automatically assigned.

  • Name
    aliases
    Type
    ([]string: [])
    Required
    Readonly
    Description

    Aliases allow alternate names to specified for the container. Aliases can be used to reference a container across the network, the container will respond to ping and other network resolution using the primary assigned name [name].container.shipyard.run and the aliases.

    network {
      name    = "network.cloud"
      aliases = [
        "alt1.container.local.jmpd.in", 
        "alt2.container.local.jmpd.in"
      ]
    }
    
  • Name
    name
    Type
    (string: "")
    Required
    Readonly
    readonly
    Description

    Name will equal the name of the network as created by jumppad.

  • Name
    assigned_address
    Type
    (string: "")
    Required
    Readonly
    readonly
    Description

    assigned_address will equal the assigned IP address for the network. This will equal ip_address if set; otherwise, this is the automatically assigned ip_address.


image

Image defines a Docker image used when creating this container. An Image can be stored in a public or a private repository.

  • Name
    name
    Type
    (string: "")
    Required
    required
    Readonly
    Description

    Name of the image to use when creating the container, can either be the full canonical name or short name for Docker official images. e.g. consul:v1.6.1 or docker.io/consul:v1.6.1.

  • Name
    username
    Type
    (string: "")
    Required
    Readonly
    Description

    Username to use when connecting to a private image repository

  • Name
    password
    Type
    (string: "")
    Required
    Readonly
    Description

    Password to use when connecting to a private image repository, for both username and password interpolated environment variables can be used in place of static values.

    image {
      name = "myregistry.io/myimage:latest"
      username = env("REGISTRY_USERNAME")
      password = env("REGISTRY_PASSWORD")
    }
    

volume

A volume type allows the specification of an attached volume.

  • Name
    source
    Type
    (string: "")
    Required
    required
    Readonly
    Description

    The source volume to mount in the container, can be specified as a relative ./ or absolute path /usr/local/bin. Relative paths are relative to the file declaring the container.

  • Name
    destination
    Type
    (string: "")
    Required
    required
    Readonly
    Description

    The destination in the container to mount the volume to, must be an absolute path.

  • Name
    type
    Type
    (string: bind)
    Required
    Readonly
    Description

    The type of the mount, can be one of the following values:

    • bind: bind the source path to the destination path in the container
    • volume: source is a Docker volume
    • tmpfs: create a temporary filesystem
  • Name
    bind_propagation
    Type
    (string: rprivate)
    Required
    Readonly
    Description

    Configures bind propagation for Docker volume mounts, only applies to bind mounts, can be one of the following values:

    • shared
    • slave
    • private
    • rslave
    • rprivate

    For more information please see the Docker documentation https://docs.docker.com/storage/bind-mounts/#configure-bind-propagation

  • Name
    bind_propagation_non_recursive
    Type
    (boolean: false)
    Required
    Readonly
    Description

    Configures recursiveness of the bind mount. By default Docker mounts with the equivalent of mount --rbind meaning that mounts below the the source directory are visible in the container. For instance running docker run --rm --mount type=bind,src=/,target=/host,readonly busybox will make /run of the host available as /host/run in the container -- and to make matters even worse it will be writable (since only the toplevel bind is set readonly, not the children). If bind_propagation_non_recursive is set to true then the container will only see an empty /host/run, meaning the tmpfs which is typically mounted to /run on the host is not propagated into the container.

  • Name
    selinux_relabel
    Type
    (string: "")
    Required
    Readonly
    Description

    Configures Selinux relabeling for the container (usually specified as :z or :Z) and can be one of the following values:

    • shared (Equivalent to :z)
    • private (Equivalent to :Z)

port

A port stanza defines host to container communications

  • Name
    local
    Type
    (string: "")
    Required
    required
    Readonly
    Description

    The local port in the container.

  • Name
    host
    Type
    (string: "")
    Required
    required
    Readonly
    Description

    The host port to map the local port to.

  • Name
    protocol
    Type
    (string: "")
    Required
    Readonly
    Description

    The protocol to use when exposing the port, can be "tcp", or "udp".

  • Name
    open_in_browser
    Type
    (string: /)
    Required
    Readonly
    Description

    Should a browser window be automatically opened when this resource is created. Browser windows will open at the path specified by this property.


port_range

A port_range stanza defines host to container communications by exposing a range of ports for the container.

  • Name
    range
    Type
    (string: "")
    Required
    required
    Readonly
    Description

    The port range to expose, e.g, 8080-8082 would expose the ports 8080, 8081, 8082.

  • Name
    enable_host
    Type
    (bool: false)
    Required
    Readonly
    Description

    The host port to map the local port to.

  • Name
    protocol
    Type
    (string: tcp)
    Required
    Readonly
    Description

    The protocol to use when exposing the port, can be "tcp", or "udp".

Meta Properties

In addition to the main properties, all resources have meta properties, such as the id of the resource. To see the list of these properties please see the Meta Properties section in the documentation /docs/resources/meta.

Examples

Minimal example

The following example creates a combined client and server running in a single container.

resource "network" "cloud" {
  subnet = "10.10.0.0/16"
}
resource "nomad_cluster" "dev" {
  network {
    id = resource.network.cloud.meta.id
  }
}

Full Example

The following example creates a multi_node nomad cluster with a job and ingress.

resource "network" "cloud" {
  subnet = "10.10.0.0/16"
}

resource "nomad_cluster" "dev" {
  client_nodes=3

  network {
    id = resource.network.cloud.meta.id
  }
}

resource "nomad_job" "example_1" {
  cluster = resource.nomad_cluster.dev

  paths = ["./app_config/example1.nomad"]

  health_check {
    timeout    = "60s"
    nomad_jobs = ["example_1"]
  }
}

resource "ingress" "fake_service_1" {
  port = 19090

  target {
    resource   = resource.nomad_cluster.dev
    named_port = "http"

    config = {
      job   = "example_1"
      group = "fake_service"
      task  = "fake_service"
    }
  }
}

Image Caching

Nomad clusters do not share the local machines Docker image cache. Each node in a cluster has it's own unqiue cache.

To save bandwidth all containers launched in the Nomad cluster pulled through an image cache that runs in Docker. After the first pull all images are subsequently pulled from the image cache not the public internet. This cache is global to all Nomad and Kubernetes clusters created with Jumppad.

For more information on the image cache see the container_registry resource.