Nomad Enterprise HVD on Azure VMs

Terraform module aligned with HashiCorp Validated Designs (HVD) to deploy Nomad Enterprise on Microsoft Azure using Azure Virtual Machines.

Prerequisites

General

• Knowledge of Terraform and Azure
• Terraform CLI >= 1.9 installed on client/workstations
• An Azure account with permissions to provision resources via Terraform
• git CLI and Visual Studio Code recommended on workstations

Networking

• Azure Virtual Network (VNet) and subnets:
  • A subnet for the Virtual Machines (compute)
  • Optional subnet for the Load Balancer

📝 Note: Specify at least two subnets for high availability.

Security groups

• This module will create network security groups (NSGs) for Nomad Virtual Machines.
• Define CIDR ranges for Nomad access using the input variable cidr_allow_ingress_nomad.

TLS certificates

TLS certificates for Nomad have very specific requirements for server and client nodes. Nomad Agent Certificates. Let's Encrypt certificates cannot be used for Nomad due to the specific requirements. This module has the option to not enable TLS with nomad_tls_enabled; however, this should only be used in lab environments.

• TLS certificate (e.g. cert.pem) and private key (e.g. privkey.pem) for the Nomad web UI, in PEM format.
  • TLS private key must not be password-protected.
• TLS certificate authority (CA) bundle (e.g. ca_bundle.pem) in PEM format.

📝 Store these TLS files as Azure Key Vault secrets to securely manage your certificates.

Gossip encryption

Gossip encryption is required for this module. A Gossip encryption key can be generated by the Nomad CLI with nomad operator gossip keyring generate. Create this before deploying the module and store it in Azure Key Vault.

Secrets management

The following secrets must be stored in Azure Key Vault to bootstrap the Nomad server deployment:

• Nomad license - text string used for Nomad Enterprise licensing - This is only needed for Nomad server deployment
• Nomad Gossip encryption key - text string generated by nomad operator gossip keyring generate - This is only needed for Nomad server deployment
• Nomad TLS certificate and private key - stored as base64-encoded PEM secrets in Azure Key Vault

📝 See the Nomad documentation for more details on securing these secrets in Azure Key Vault.

Deployment Options

see Deployment customizations

Usage

1. Configure the prerequisites.

2. In the examples directory, you will find subdirectories with ready-made Terraform configurations for deploying this module. Select an example that matches your use case, and copy its contents to a new directory.

   📝 Example structure for managing multiple Nomad deployments:

   .
   └── environments
       ├── production
       │   ├── backend.tf
       │   ├── main.tf
       │   ├── outputs.tf
       │   ├── terraform.tfvars
       │   └── variables.tf
       └── sandbox
           ├── backend.tf
           ├── main.tf
           ├── outputs.tf
           ├── terraform.tfvars
           └── variables.tf
   

   📝 This example has two separate Nomad deployments: one for a sandbox environment and one for a production environment.

3. (Optional) If using Azure Blob Storage for remote state, configure the backend.tf file with custom values.

4. Update the terraform.tfvars file with your custom values, then run terraform init, terraform plan, and terraform apply.

5. After terraform apply completes successfully, connect to the Nomad VM instance shell using SSH or Azure Bastion to monitor the cloud-init logs:

   Connecting to the VM instance:

   ssh -i /path/to/ssh_private_key azureuser@<vm-public-ip>
   

   Viewing logs:

   tail -f /var/log/nomad-cloud-init.log
   

6. Once the installation finishes, verify the health of the Nomad cluster by checking the Nomad Web UI or by running:

   nomad server members
   

7. If nomad_acl_enabled is true, the Nomad cluster will need to be bootstrapped with the command nomad acl bootstrap. This will generate a bootstrap token that can be used to log in to the CLI or UI.

Docs

Additional documentation for managing and customizing your Nomad deployment is available in the docs folder:

• Nomad version upgrades
• Nomad TLS certificate rotation
• Nomad configuration settings
• Nomad deployment customizations

Module support

This open source software is maintained by the HashiCorp Technical Field Organization, independently of our enterprise products. While our Support Engineering team provides dedicated support for our enterprise offerings, this open source software is not included.

• For help using this open source software, please engage your account team.
• To report bugs/issues with this open source software, please open them directly against this code repository using the GitHub issues feature.

Please note that there is no official Service Level Agreement (SLA) for support of this software as a HashiCorp customer. This software falls under the definition of Community Software/Versions in your Agreement. We appreciate your understanding and collaboration in improving our open source projects.

Requirements

Name	Version
terraform	>= 1.9
azurerm	>= 3.70.0

Providers

Name	Version
azurerm	>= 3.70.0

Resources

Name	Type
azurerm_dns_a_record.nomad	resource
azurerm_key_vault_access_policy.nomad_vmss_keyvault_access	resource
azurerm_lb.nomad	resource
azurerm_lb_backend_address_pool.nomad_backend_pool	resource
azurerm_lb_probe.nomad_probe	resource
azurerm_lb_rule.nomad_rule_4646	resource
azurerm_linux_virtual_machine_scale_set.nomad	resource
azurerm_network_interface.nomad_nic	resource
azurerm_network_interface_security_group_association.lb_nsg_association	resource
azurerm_network_security_group.lb_nsg	resource
azurerm_network_security_group.nomad	resource
azurerm_private_dns_a_record.nomad	resource
azurerm_private_dns_zone_virtual_network_link.nomad	resource
azurerm_public_ip.nomad_frontend_ip	resource
azurerm_resource_group.nomad	resource
azurerm_role_assignment.nomad_vm_reader_role	resource
azurerm_user_assigned_identity.nomad_vm_identity	resource
azurerm_client_config.current	data source
azurerm_dns_zone.nomad	data source
azurerm_image.custom	data source
azurerm_key_vault.nomad_keyvault	data source
azurerm_platform_image.latest_os_image	data source
azurerm_private_dns_zone.nomad	data source
azurerm_resource_group.nomad	data source
azurerm_resource_group.nomad_rg	data source
azurerm_subnet.lb_subnet	data source
azurerm_subscription.primary	data source

Inputs

Name	Description	Type	Default	Required
friendly_name_prefix	Friendly name prefix used for uniquely naming Azure resources.	string	n/a	yes
location	Azure region to use for this deployment.	string	n/a	yes
nomad_client	Enable the Nomad client agent.	bool	n/a	yes
nomad_datacenter	Specifies the data center of the local agent.	string	n/a	yes
nomad_server	Enable the Nomad server agent.	bool	n/a	yes
resource_group_name	Name of Azure resource group to create or name of existing resource group to use (if create_resource_group is false).	string	n/a	yes
subnet_id	Azure subnet ID for Nomad instance network interface.	string	n/a	yes
vnet_id	ID of the Azure VNet where resources are deployed.	string	n/a	yes
vnet_name	Name of the Azure VNet where resources are deployed.	string	n/a	yes
additional_package_names	List of additional repository package names to install on the VMs	set(string)	[]	no
admin_username	Admin username for VM instance.	string	"ubuntu"	no
associate_public_ip	Whether to associate public IPs with the Nomad cluster VMs.	bool	false	no
autopilot_health_enabled	Perform autopilot health checks on Nomad server nodes at boot.	bool	true	no
availability_zones	List of Azure Availability Zones to spread nomad resources across.	set(string)	[ "1", "2", "3" ]	no
cidr_allow_ingress_nomad	CIDR ranges allowed ingress on port 443/80 for Nomad server/load balancer.	list(string)	[ "0.0.0.0/0" ]	no
cni_version	Version of CNI plugin to install.	string	"1.6.0"	no
common_tags	Map of common tags for taggable Azure resources.	map(string)	{}	no
create_load_balancer	Boolean to create an Azure Load Balancer for Nomad.	bool	true	no
create_nomad_private_dns_record	Boolean to create a DNS record for nomad in a private Azure DNS zone. private_dns_zone_name must also be provided when true.	bool	false	no
create_nomad_public_dns_record	Boolean to create a DNS record for nomad in a public Azure DNS zone. public_dns_zone_name must also be provided when true.	bool	false	no
create_resource_group	Boolean to create a new Azure resource group for this deployment. Set to false if you want to use an existing resource group.	bool	false	no
custom_startup_script_template	Name of custom startup script template file. File must exist within a directory named ./templates within your current working directory.	string	null	no
disk_size_gb	Size of OS disk for Nomad VMs in GB.	number	50	no
disk_type	Disk type for Nomad VMs. Options: Standard_LRS, Premium_LRS, etc.	string	"Standard_LRS"	no
frontend_ip_config_name	The name of the frontend IP configuration to which the rule is associated.	string	"PublicIPAddress"	no
lb_is_internal	Create an internal (private) Azure Load Balancer.	bool	true	no
lb_private_ip	Private IP address for internal Azure Load Balancer. Only valid when lb_is_internal is true.	string	null	no
lb_subnet_name	Name of the Azure lb subnet where the lb resources should be deployed too.	string	null	no
nomad_acl_enabled	Enable ACLs for Nomad.	bool	true	no
nomad_architecture	Architecture of the Nomad binary to install.	string	"amd64"	no
nomad_fqdn	Fully qualified domain name of the Nomad Cluster. This name should resolve to the load balancer IP address.	string	null	no
nomad_gossip_encryption_key_secret_id	ID of Azure Key Vault secret for Nomad gossip encryption key.	string	null	no
nomad_key_vault_name	ID of Azure Key Vault secret for Nomad license file.	string	null	no
nomad_license_secret_id	ID of Azure Key Vault secret for Nomad license file.	string	null	no
nomad_location	Specifies the region of the local agent. Defaults to the Azure region if null.	string	null	no
nomad_nodes	Number of Nomad nodes to deploy.	number	2	no
nomad_tls_ca_bundle_secret_id	ID of Azure Key Vault secret for private/custom TLS Certificate Authority (CA) bundle in PEM format. Secret must be stored as a base64-encoded string.	string	null	no
nomad_tls_cert_secret_id	ID of Azure Key Vault secret for Nomad TLS certificate in PEM format. Secret must be stored as a base64-encoded string.	string	null	no
nomad_tls_enabled	Enable TLS for Nomad.	bool	true	no
nomad_tls_privkey_secret_id	ID of Azure Key Vault secret for Nomad TLS private key in PEM format. Secret must be stored as a base64-encoded string.	string	null	no
nomad_ui_enabled	Enable the Nomad UI.	bool	true	no
nomad_upstream_servers	List of Nomad server addresses to join the Nomad client with.	list(string)	null	no
nomad_upstream_tag_key	String of the tag key the Nomad client should look for in Azure to join with. Only needed for auto-joining the Nomad client.	string	null	no
nomad_upstream_tag_value	String of the tag value the Nomad client should look for in Azure to join with. Only needed for auto-joining the Nomad client.	string	null	no
nomad_version	Version of Nomad to install.	string	"1.9.0+ent"	no
permit_all_egress	Allow unrestricted egress on cluster nodes. Additional rules may be required if disabled.	bool	true	no
private_dns_zone_name	Name of existing private Azure DNS zone to create DNS record in. Required when create_nomad_private_dns_record is true.	string	null	no
private_dns_zone_rg	Name of Resource Group where private_dns_zone_name resides. Required when create_nomad_private_dns_record is true.	string	null	no
public_dns_zone_name	Name of existing public Azure DNS zone to create DNS record in. Required when create_nomad_public_dns_record is true.	string	null	no
public_dns_zone_rg	Name of Resource Group where public_dns_zone_name resides. Required when create_nomad_public_dns_record is true.	string	null	no
vm_custom_image_name	Name of custom VM image to use for VMSS. If not using a custom image, leave this blank.	string	null	no
vm_custom_image_rg_name	Name of Resource Group where vm_custom_image_name image resides. Only valid if vm_custom_image_name is not null.	string	null	no
vm_enable_boot_diagnostics	Boolean to enable boot diagnostics for VMSS.	bool	true	no
vm_image_version	Version of the VM image.	string	"latest"	no
vm_os_image	The OS image to use for the VM. Options are: redhat8, redhat9, ubuntu2204, ubuntu2404.	string	"ubuntu2404"	no
vm_size	Azure VM size for Nomad VMs.	string	"Standard_D2s_v3"	no
vm_ssh_public_key	SSH public key for VMs in VMSS.	string	null	no

Outputs

Name	Description
nomad_cli_config	Environment variables to configure the nomad CLI
nomad_url	URL to access Nomad application based on value of nomad_fqdn input.