Did you know that the city with the most McDonald’s restaurants per capita is Orlando, Florida? With almost 20 McDonald’s per 100k people, finding a McDonald’s depends entirely on where you are. It doesn’t matter if you’re on International Drive or Thornton Park, you’ll get your Big Mac; it’ll just be served by different staff, in a different franchise, in a different neighborhood. If there’s not a McDonald’s close enough to you, you’ll need to wait longer before you get your meal.

When we talk about Anycast, it’s pretty much the same. You can request your favorite website, and it’s within that website’s best interest to serve you from a location as close to you as possible for the best possible experience.

To dig into the fundamental advantages of Anycast before we dive into the more technical side of how it works, check out “Leverage BGP and Bare Metal To Build An Anycast Network“. Let’s start with a quick example of how the internet traditionally works with unicast routing. With unicast, every public IP address maps to zero or one machine. The machines that have these addresses have their IP address advertised by an autonomous system (AS). There are many autonomous systems on the internet, and they all share information with each other about the IP addresses they advertise.

When you request 1.2.3.4 from your laptop, you’re routed through a local AS. It’s unlikely the local AS is advertising 1.2.3.4, so it needs to route you through another AS. In our diagram above, it’s possible to get to 1.2.3.4 via multiple paths within the network, and your local AS will do its best to route you through the shortest, which is through the Amsterdam AS to the 1.2.3.4 machine.

With anycast routing, we can mix things up a little and switch from having a single machine advertising an IP address to one machine per AS advertising the IP.

Anycast with Equinix Metal

In-order to leverage the anycast pattern with Equinix Metal, we can utilize Global IPs. Global IPs are what Equinix Metal call IP addresses that can be advertised from multiple AS’s within network.

You can order Global IPs from the “IPs & Networks” page on your Equinix Metal console. They are available as 1 (/32), 2 (/31), or 4 (/30) IP blocks.

Deploying Our Workload

For this example, we are going to keep things nice and simple. We will deploy nginx to two machines with Equinix Metal: one in Amsterdam (AM6) and one in Singapore (SG1). On Ubuntu, we can install nginx with the following command:

apt update && apt install -y nginx

If we ask curl to give us some performance metrics from our laptop, we will see how much the requests differ; due to the distance traveled for each request. To have curl give us performance metrics, we need to create a small helper script.

#!/usr/bin/env sh
curl -w @- -o /dev/null -s "$@" <<'EOF' 
time_namelookup:  %{time_namelookup}\n
time_connect:  %{time_connect}\n 
time_appconnect:  %{time_appconnect}\n
time_pretransfer:  %{time_pretransfer}\n 
time_redirect:  %{time_redirect}\n 
time_starttransfer: %{time_starttransfer}\n
                  ----------\n
        time_total:  %{time_total}\n
EOF

These requests are being made in Scotland. First, let us check out the timings for a curl request to Amsterdam.

time_namelookup:

0.000931

time_connect:

0.031447

time_appconnect:

0.000000

time_pretransfer:

0.031521

time_redirect:

0.000000

time_starttransfer:

0.068919

time_total:

0.069020

Pretty quick! Not surprising, as Amsterdam is only 900km away from Scotland. Now let’s try Singapore, which is around 6800km from Scotland.

time_namelookup:

0.001426

time_connect:

0.263027

time_appconnect:

0.000000

time_pretransfer:

0.263073

time_redirect:

0.000000

time_starttransfer:

0.531548

time_total:

0.531722

We can see that the round-trip time (RTT) for Amsterdam is 69ms, and unfortunately for our customers, the RTT for Singapore is 533ms.

We want better response times for our customers, regardless of where they are in the world; but we do not want to give our EMEA ) customers one IP address or DNS name, and other customers in other regions a different IP address or DNS

name. This would be cumbersome and confusing. We want a single-entry point to our service that can be used globally and routed efficiently and effectively.

To solve this, we’ll use the anycast pattern and advertise our Global IPs from our AS’s in as many facilities as we need to keep our customers happy.

Broadcasting Our Global IPs

First, we need to add our Global IP to each of the machines we want to route traffic to. We can add a new interface to the /etc/network/interfaces file.

# /etc/network/interfaces 
auto lo:0
iface lo:0 inet static
    address 147.75.40.40 #Your Global IP 
    netmask 255.255.255.255

Then we can bring up our interface with ifup lo:0

Next, we need to install bird and configure it to broadcast our IP address to the local AS via the Border Gateway Protocol (BGP).

apt -y update && apt -y install python3.6 python3-pip git bird

Equinix Metal provides some helper scripts to configure bird, which can be installed with:

cd /opt
git clone https://github.com/packethost/network-helpers.git 
cd network-helpers
pip3 install jmespath 
pip3 install -e .

Now we can use these helpers to generate our bird configuration and restart the service.

./configure.py -r bird | tee /etc/bird/bird.conf
systemctl restart bird

That is it! You need to repeat this on each of your machines that you want to advertise your Global IP. In this example, we now have two machines, geographically distributed, answering to a single IP address broadcasting to multiple AS.

Let us look at the new response times from curl. First, we will hit our Global IP from Scotland; which will be routed to Amsterdam (1000km).

time_namelookup:

0.000371

time_connect:

0.027531

time_appconnect:

0.000000

time_pretransfer:

0.027566

time_redirect:

0.000000

time_starttransfer:

0.061749

time_total:

0.061836

As well as the response time, we can run a traceroute to ensure the IP was resolved in Amsterdam.

traceroute to 147.75.40.40 (147.75.40.40), 64 hops max, 52 byte packets

1  192.168.86.1 (192.168.86.1)  3.826 ms  6.728 ms  3.926 ms
2  192.168.0.1 (192.168.0.1)  5.452 ms  6.977 ms  7.106 ms
3  * * *
4  renf-core-2b-xe-902-0.network.virginmedia.net (81.97.51.89)  23.752 ms  19.608 ms  19.857 ms
5  * * *
6  * * *
7  * * *
8  * * *
9  * m686-mp2.cvx1-b.lis.dial.ntli.net (62.254.42.174)  1206.166 ms  27.833 ms
10  213.46.174.118 (213.46.174.118)  28.407 ms  37.980 ms  34.374 ms
11  ldn-bb1-link.ip.twelve99.net (62.115.122.188)  41.069 ms * *
12  adm-bb3-link.ip.twelve99.net (62.115.113.210)  54.836 ms
    adm-bb3-link.ip.twelve99.net (213.155.136.99)  40.088 ms
    adm-bb4-link.ip.twelve99.net (62.115.113.238)  34.877 ms
13  adm-b1-link.ip.twelve99.net (62.115.137.65)  40.951 ms  33.051 ms  34.763 ms
14  packethost-ic-349744-adm-b1.c.telia.net (62.115.176.233)  35.449 ms  43.872 ms  34.326 ms
15  0.et-0-0-0.bsr1.am6.packet.net (45.11.244.53)  35.634 ms  46.148 ms  38.603 ms
16  * * *
17  * * *
18  am6-c3.medium.x86-01 (147.75.40.40)  35.437 ms  32.807 ms  40.736 ms

Now we can do the same from a machine in Hong Kong, which is routed to Singapore (2500km).

time_namelookup:

0.000016

time_connect:

0.117823

time_appconnect:

0.000000

time_pretransfer:

0.117909

time_redirect:

0.000000

time_starttransfer:

0.235641

time_total:

0.235738

Finally, we can see the traceroute run from our Hong Kong machine is routed to Singapore.

traceroute to 147.75.40.40 (147.75.40.40), 30 hops max, 60 byte packets

1  147.75.42.140 (147.75.42.140)  12.815 ms  12.723 ms  12.687 ms
2  0.ae101.dsr1.hkg2.packet.net (147.75.43.36)  11.745 ms  11.707 ms  11.673 ms
3  0.ae0.dsr2.hkg2.packet.net (147.75.43.33)  7.123 ms  7.084 ms  7.093 ms
4  156.xe-0-1-0.bbr1.nrt1.packet.net (147.75.43.46)  50.078 ms  50.038 ms  49.966 ms
5  430.xe-0-0-35-2.bsr2.sin2.packet.net (45.11.244.100)  118.265 ms  118.753 ms  118.747 ms
6  10.254.4.18 (10.254.4.18)  117.729 ms 10.254.4.14 (10.254.4.14)  117.801 ms  117.751 ms
7  10.254.4.84 (10.254.4.84)  117.692 ms  117.708 ms 10.254.4.82 (10.254.4.82)  117.634 ms
8  sg1-c3.medium.x86-01 (147.75.40.40)  117.588 ms  117.738 ms  117.660 ms

Building globally low-latency services that your customers will love has never been so simple.