Using Lando with many client sites

The problem

Working in Support requires setting up many different client sites, and the need to switch between them quickly is key, as it must be done many times per day. Lando solves this problem very well. It is a wrapper around the Docker API used to quickly set up many different types of sites. If you have used docker-compose before you will feel at home in a .lando.yml file.

However, Docker itself has some limitations in its default configuration. You are limited to what feels like a very small amount of networks. I came across this blog post that states the limitation is 31 networks by default, but I was hitting numbers even smaller than that — somewhere around 12. The issue with only 12 networks is that I can only have 12 client sites ready to go at any given time because each Lando site must reserve one of those network IP address ranges.

The solution

Since Lando uses Docker under the hood, dive into Docker networking! I found many good resources describing how Docker containers connect to networks that I link as resources below for further information. The solution ended up being very simple since Docker supports changing the default network pool.

NOTE: If you have existing networks you will need to docker network prune them away before you can change your existing available IP pool. This requires Docker to be running BEFORE you make any changes to your daemon.json file. It also requires your active containers to be stopped. If you are using Lando, do a lando poweroff to shut everything down. After the prune you should only see the default 3 local Docker networks.

First I added the /etc/docker/daemon.json file via my local text editor. This location is on Ubuntu; location may vary on other OSs. The file should contain these contents to change the default address pool range:

{
  "default-address-pools": [{"base":"192.168.129.0/17","size":26}],
}

The size option is a netmask, so starting with a base of 32 bits for IPv4 addresses and subtracting the netmask size, 26, we are left with 6 bits. 2^6 is 64, but we cannot use the network identifying address or the broadcast address, leaving us 62 containers per network. Netmask binary example shown below:

11000000.10101001.00000000.00 000000
11111111.11111111.11111111.11 000000
00000000.00000000.00000000.00 111111

The base option provides the base address pool that can be used. We will use the netmask from the base minus the netmask from the size to calculate the number of networks we can set up. We know that each network has a netmask size of 26, so 26 minus the netmask of the pool, 17, equals 9 bits to be used for networks. 2^9 = 512 networks.

So in total, this configuration allows me to have 62 containers per network, 512 networks, and a whopping 31,744 Docker containers. That also means I can run 512 Lando instances/client sites on my machine, fixing the annoying issue of bumping into network limits. Now I get to worry about other resource bottlenecks like CPU and memory.

Tips

• Make sure the address pool has a base IP range that isn’t already claimed by your router or your VPN, etc.

• This solution assumes that you are using named Docker networks, which are more secure since they limit network traffic to the containers connected to the network. If you are using the default network for some containers that don’t need contained traffic, another helpful setting to tweak in your /etc/docker/daemon.json file is the bip key:

  "bip": "192.168.127.1/24",

In my case, this is ideal for services that I want to be available on my local network. I could expand the number of hosts and subnet here if needed.  It is likely that this is my actual maximum number of networks since Lando sets up named networks for each instance. The theoretical maximum here would be 256 since I am using the /24 netmask. 

The DNS key can also be set up if you wish for your Docker containers to connect to a specific DNS server (local or external):

  "dns": ["8.8.8.8","8.8.4.4"],

The insecure registries key is used if you wish to set up the equivalent of Docker hub locally (a place where you can push custom Docker images):

 "insecure-registries" : ["192.168.99.0/24"]

This key is important if you are attempting to pull Docker images from a local Docker repository that uses HTTP (insecure) instead of HTTPS.

References

1. https://docs.docker.com/network/bridge/
2. https://docs.docker.com/config/daemon/
3. https://docs.docker.com/network/network-tutorial-standalone/
4. https://success.docker.com/article/how-do-i-configure-the-default-bridge-docker0-network-for-docker-engine-to-a-different-subnet
5. http://jodies.de/ipcalc?host=192.168.129.0&mask1=17&mask2=26
6. https://medium.com/@loomchild/docker-can-only-create-31-default-networks-e7f98f778626