Exploring Ansible via Setting Up a WireGuard VPN
Learn Ansible and use it to automate setting up server and client sides of a WireGuard VPN.
Mar 4, 2021
Greg Schaberg
Senior Infrastructure and Web
In my previous blogpost, we set up a WireGuard VPN server and client and learned about various configuration options for WireGuard, how to improve VPN server uptime, how to relay traffic, and more. Setting up a server and client like that is a lot of work! If the server dies or you want to set up a new server (maybe for a friend or family member this time), you have to go back to the walk-through and follow all the steps, remembering if you deviated from those instructions at any point.
There's a better way — automation! If you're only going to do a thing once (e.g. set up a VPN), investing in automation probably doesn't make sense. But if you anticipate doing a thing repeatedly, automating it frees up your time to learn and accomplish more in the future. You can also share your automation, empowering others to build and achieve more, faster.
Automation is the heart of computing, and many different automation tools and approaches have sprung up over time. For our project of automating VPN server setup, we can consider a variety of tools:
Shell scripts
The simplest approach from a tooling perspective, writing shell scripts would involve running the commands from the previous WireGuard tutorial blogpost, using
sshfor the commands that run on the server andrsyncto copy configurations files to the server.
SSH scripting libraries like Capistrano or Fabric
If shell scripting isn't ideal, there are libraries that expose similar scripting functionality in a more ergonomic interface for developers familiar with higher-level languages like Ruby and Python.
Infrastructure/configuration automation tools like Puppet, Chef, or Ansible
Tools in this category are even more specialized for automating server infrastructure and configuration, often including an ecosystem of packages and plugins to automatically set up or configure nearly anything you can think of.
Infrastructure-as-code tools like Terraform
Infrastructure-as-code (IaC) tools have a lot of overlap with the above category, but support provisioning cloud resources in a more first-class/native way.
Containers like Docker
For this tutorial, I'm going to focus on the middle category above — infrastructure/configuration automation tools — and specifically, I'll focus on Ansible. There is a great comparison of different tools in this area by Gruntwork and, even though that article favors Terraform, Ansible is still a useful general-purpose tool, especially if you're working with servers that aren't "in the cloud", such as a Raspberry Pi at home.
Let's get started with automating VPN setup with Ansible! By the end of this article, we'll be able to set up a VPN server and client with a single command. Similar to the previous blogpost, I'll use Ubuntu 20.04 and DigitalOcean droplets.
Setting up Ansible
Ansible can be installed via an OS package manager like apt , but I prefer to use pip so I can get the latest updates and avoid cluttering system package management with third-party PPAs (Personal Package Archives). We'll also use pyenv (as suggested by Hypermodern Python) to make sure we're not breaking or cluttering the system Python installation. Install pyenv with the following:
It's a good habit when a tutorial gives you curl <url> | bash to open up that URL and see what it's going to do. In this case, you'll see that it'll download and execute a shell script on GitHub that will clone 6 repos from GitHub to your ~/.pyenv folder and prompt you to add a few lines to your shell's initialization script.
Follow the output prompt from above, which asks you to put lines like the below in your shell initialization script (e.g. ~/.bashrc if you use the bash shell). Make sure to fill in your own username!
Install a recent python version:
If you want, you can also create a [virtualenv](https://virtualenv.pypa.io/en/latest/) to further isolate the Ansible installation, and make that virtualenv automatically activate when you're in a particular folder/repo. That would look like:
Install the ansible pip package, which will install various command-line tools, including ansible-playbook, which we'll use to run a "playbook" of commands that will set up a VPN server and client for us.
Get a Server
To use Ansible for a VPN server, we need... a server! Ansible could provision a server from a cloud provider for us (and I'll touch on this briefly later), but we'll keep our playbook hardware-provider-agnostic for now, so you can run it as easily against a cloud server as a Raspberry Pi on your home network. I'm going to [create a $5/month DigitalOcean droplet](https://www.digitalocean.com/docs/droplets/how-to/create/) to test against, but you could also [use Vagrant](https://docs.ansible.com/ansible/latest/scenario_guides/guide_vagrant.html) (to test against a local VM) or any server you can SSH to.
Testing Ansible playbooks against VMs, rather than a bare-metal machine, comes with an advantage — after you've written the playbook, you can start a new, empty VM and test the whole playbook start to finish to ensure that it works consistently.
Connecting to the Server with Ansible
Once you have your server or VM, take note of its IP address use it to create an inventory.ini file like the below:
An inventory file tells Ansible what servers it can act upon and how to access them. Let's use the above inventory file as an example. When we run Ansible and target the vpn group of servers or the vpn_server host, it will try to connect to the server using a command like:
So, if you can't SSH to the server, then Ansible won't be able to connect either!
Connecting to the server with an SSH key is strongly recommended! Add your SSH key to your server to connect without needing a password. If you must connect with a password, you can sudo apt install sshpass and then provide your SSH password when using Ansible by adding the --ask-pass flag to all ansible commands.
Let's test to make sure that Ansible can connect to the server:
This runs the ping Ansible module, targeting the vpn group of servers. You should see "pong" in the output, meaning that Ansible could connect to the server and the server has a Python installation that Ansible can use.
Ansible's Built-in Variables and Facts
There are other useful Ansible modules that we can use with the ansible command:
The setup module fetches system information, also known as "facts", about the server. You can use these facts as variables in Ansible commands and playbooks.
The debug module can evaluate variables, which is useful for... well, debugging!
Try running both of these modules with your server so you can see what facts and information Ansible makes available:
This was one of the most confusing parts for me when learning Ansible — figuring out what all these built-in variables and facts (like groups , inventory_dir , and ansible_distribution ) were and how to find them.
Writing an Ansible Playbook
The ansible command lets you run ad-hoc commands across groups of servers. This is powerful, but we probably shouldn't try to automate server setup and configuration in a single ansible command... probably. 🤔 Instead, we can organize multiple tasks in one or multiple YAML files, which we will run with the ansible-playbook command.
Let's write a playbook.yml file In the same folder as inventory.ini . Here are its contents:
If you're not familiar with YAML, the above is equivalent to this JSON structure:
Breaking down the above:
The top-level structure is a "play" in Ansible lexicon. Our play above has a
name, ahostspattern which describes which servers the play will run against, and a list oftasks.We have 2 tasks, each has a
nameand the name of an Ansible module that will do something.
Run the playbook...
... and you'll see that it gathers facts from the server (just like the ansible -m setup command above did), and then runs the "ping" task and the "debug" task to show all the gathered facts and variables defined for vpn_server .
There are tons of built-in Ansible modules, even more curated Ansible community modules, and even more published to Ansible Galaxy (an open repository for Ansible collections and roles).
WireGuard Server Setup
There's much more to learn about Ansible! But let's stop here and apply what we've learned in order to set up a WireGuard server.
Referring to the steps we took in the previous tutorial, we want to:
Install the
wireguardsystem packageCreate public and private keys with correct permissions
Create the server's WireGuard configuration file
(Optionally) Enable IP forwarding for relaying traffic
Start the VPN
Managing the Keys
As hinted at in the previous tutorial, if we want to repeatably deploy the VPN server without needing to reconfigure all VPN clients, we need to use the same private key every time.
Put another way: if we generated a private key while deploying the server and used the corresponding public key on various clients, and the server ends up dying, we could deploy it again by generating a new private key. However, all of our VPN clients would then need to update to the new public key to be able to connect to the new VPN server. This would be inconvenient!
Instead, we'll generate the server keys once by hand and use them in the playbook so they're consistent between every deploy. This means we won't include step #2 from above in the Ansible playbook.
Generate the keys with wg genkey and wg pubkey commands. You can output both with the following command:
Copy the output lines and add them to a new vars mapping under the play in playbook.yml . Here's what mine looks like now (your keys will be different):
Encrypting the Private Key
It's a good practice to AVOID having secrets in plaintext (like the VPN private key above). This is especially true if those secrets will be shared with anyone else, like via a git repo. Let's prevent this by using Ansible Vault. Vault is a tool for encrypting secret values and using them in playbooks. Encrypt the private key with:
You'll be prompted twice for a Vault encryption password, after which you'll paste your privkey value and hit Ctrl+d twice. If the command completed after a single Ctrl+d , try again and make sure you're not copy-pasting an invisible newline character at the end of the privkey value. Copy the output into your playbook, which will now look like:
Make sure to remember your encryption password (and save it in a password manager); you'll need to enter it every time you run the playbook.
Installing and Configuring WireGuard
Next, we'll remove our testing ping and debug tasks and write tasks for steps 1, 3, 4, and 5 from the above list. These steps translate neatly into Ansible tasks in our updated playbook.yml:
Ok ok, yes, this is a bit like drawing an owl.
Source: https://29.media.tumblr.com/tumblr_l7iwzq98rU1qa1c9eo1_500.jpg
...but usually an ansible playbook like the above can be written quickly. I follow a cycle:
Type "ansible module install package" into a search engine
Open the docs.ansible.com result that looks most helpful
Read through available parameters and the (often helpful) examples at the bottom
Copy an example into my playbook and modify parameters as needed
Go back to step 1, searching for the next task (e.g. "ansible module template file")
I've included a comment line linking to the Ansible docs page for each module used in the playbook.yml above, in case you want to read about the parameters.
Testing our First Attempt
Let's test our playbook.
Oh no! Installing WireGuard was successful, but creating the config failed. Ansible's error messages are usually helpful, and this one indicates that the template file (server_wg0.conf.j2) we're trying to use to create the server's configuration couldn't be found. Let's create it at templates/server_wg0.conf.j2:
A few notes about the above:
- Ansible automatically searches in relative paths like templates/ and files/ when running Ansible modules that have a src parameter. Our template task has a parameter src: server_wg0.conf.j2, so Ansible will search for it in the templates/ folder.
- It's convention to suffix template files with .j2, to indicate that the file will be templated with Jinja2.
- In Jinja2, values inside double curly braces ({{ variable }}) will be replaced with the value of the variable. In this template, the server_privkey variable will be decrypted and its value inserted into the resulting file in place of {{ server_privkey }}.
- The {{ ansible_managed }} text is replaced with the string "Ansible managed". It's a good convention to put this in a comment at the top of templated files, because it signals to anyone reading the file on the server that the file is managed by Ansible — any edits they make could be overwritten when Ansible next runs, so they should find and make edits in the corresponding Ansible playbook and template files instead.
Let's run the test again:
It succeeded! The WireGuard interface is now running on the server.
Notice that the "install wireguard package" step shows ok instead of changed this time. The apt module (and most modules) detect that the server is already in the desired state (the wireguard package was installed last time we ran the playbook, so it satisfies state=present ) and perform no actions. The task is idempotent, meaning you can run it repeatedly and the outcome is the same. Idempotent tasks make it easy to see what changed and what didn't each time a playbook is run.
WireGuard Client Setup
Ansible can also operate on the local machine. To set up our local machine as a client, we want to:
Install the
wireguardsystem packageCreate public and private keys with correct permissions
Create the client's WireGuard configuration file, which must include the server's public key
Start the VPN
We also need to update the server's configuration file with a [Peer] section including the client's public key, so the client can connect to the server. The client's public key isn't known until after we create it — we could create client keys manually like we did for the server's keys, but then the playbook wouldn't be able to set up multiple clients without having to manually edit the keys for each client.
Acting on Localhost
Because we're targeting a new host (localhost ), we need to write a new play in playbook.yml . We can put it above the existing play (which targets vpn_server ), so the client's keys are generated before the server config is templated.
Lots of new things here!
We target the local machine with using
[localhost](http://localhost)for the hosts pattern.We "connect" locally by using the
localconnection plugin.The
become: yesline indicates that the play will run as root, which we need to be able to install thewireguardpackage. Ansible will effectively runsudo apt-get install wireguard, rather than justapt-get install wireguard(which would fail). Because of this setting, we'll need to run the playbook with the--ask-become-passflag. We didn't need this line for the server setup play, because we're already connecting as root via theansible_user=rootconnection variable.With the
ansible_python_interpretervar, we tell Ansible to use the system python (which includes theaptpython package). Alternatively, we could install that package for our current python 3.9.2 installation. If you get aNo such file or directoryerror, you may need to change the line frompythontopython3.
Client Setup Tasks and Config
Writing the Ansible tasks for the client-side VPN setup is similar to the server side.
Breaking this down:
Installing the
wireguardpackage should look very familiar!We generate keys with the
shellmodule so we can use pipes and file redirection. The keys are only generated if thepublickeyfile doesn't already exist, thanks to thecreatesparameter.Next, we need to save the public key so we can add it as a
[Peer]section in the server config. Normally, we'd use{{ lookup('file', '/etc/wireguard/publickey') }}to look up a value from a file, but the file lookup modules seems not to respectbecome: yes; it tries to read the file without escalating to root privileges and fails as a result. So, we insteadcatthe file and save the resulting output as a fact.Finally, template the client config file. Its contents closely match the previous tutorial's, but we use the
ansible_hostIP address of the VPN server frominventory.inito set the server's endpoint.
Managing Variables
If we run the playbook now, it will fail with a 'server_pubkey' is undefined error. That's because server_pubkey is defined for the play that targets the server, it's not available for the play targeting the client. We need to move the variable somewhere so that it's readable by the entire playbook. Ansible looks for YAML files in a group_vars/ folder where the filename matches server groups in the inventory file. So, we could create a group_vars/vpn.yml file and declare variables in it, which would be directly usable when running a play against any servers in the vpn group. We don't include localhost as a host in the vpn group (though we could). We'll instead use the special group_vars/all.yml file, which makes variables available to all hosts.
Move the server keys' variables from playbook.yml to group_vars.all.yml :
Your directory should now look like this:
Run the playbook and the client should run all its tasks successfully:
The VPN client is now set up. The only remaining step for the client is to start the VPN after the server is running and configured to accept connections from the client (so the client's PostUp ping will succeed).
Adding a Peer to the Server Config
Add a [Peer] section to the server template at templates/server_wg0.conf.j2 :
We read the {{ server_privkey }} from group_vars/all.yml and we read {{ hostvars['localhost'].pubkey }} from the set_fact module that runs during the client-targeted play in the playbook.
Reloading the Server Config
If we run the playbook, the config file on the server will be updated with the new [Peer] section, but the WireGuard interface is already running and configured based on the old file contents. We need to reload the configuration when it changes. Handlers are the Ansible-provided mechanism for this, and they trigger when a task referencing them changes. Handlers run at the end of the play in which they're notified, so many tasks could notify a "reload config" handler, but the handler would only run once at the end. Let's create a couple handlers in a handlers list after the tasks lists in playbook.yml and notify them from the create client wireguard config and create server wireguard config tasks:
The template Ansible module only performs an action and marks the task as changed if the config file changes — it is idempotent. Idempotence is valuable when used with handlers, because the handler will only run when the task changes. Notifying a handler on a task that isn't idempotent may result in the handler always running (e.g. a service is unnecessarily restarted everytime the playbook is run).
Start the VPN Client
Add one final play to the end of the playbook to start the client VPN now that the server is configured to accept its connection:
Automation Complete!
Now we can run the whole playbook and — whether the server and client are brand-new or in some intermediate state — this single command will set up a WireGuard VPN server and client!
The complete Ansible code can be found at: https://gitlab.com/tangram-vision-oss/tangram-visions-blog
There are many improvements that could be made:
Provision a cloud server automatically, using an Ansible module such as community.digitalocean.digital_ocean_droplet.
Automatically update a floating IP address when provisioning a new cloud VPN server.
Configure multiple clients automatically. One approach is to add a
vpn_clientsgroup to the inventory, define VPN IPs in the inventory (e.g.vpn_ip=10.0.0.8), and use those host variables in the config templates. When templating the server config, loop over hostnames in the clients group, adding a new[Peer]block for each.Organize the playbook as roles, one for the server and one for the client. Roles are more reusable and shareable than playbooks.
Test and lint with molecule and ansible-lint.
Thanks for joining me on this Ansible-learning journey! If you have any suggestions or corrections, please let me know or send us a tweet, and if you’re curious to learn more about how we improve perception sensors, visit us at Tangram Vision.
