Know Thy API

As part of our digital transformation at Equinix, we provide web portal interfaces for our customer interactions with our products and services. As with many modern platform service providers, we now have re-built our products with modern microservice architectures running as workloads on resilient Kubernetes infrastructures. These microservices expose their portion of the application functionality as individual endpoints (think URLs), providing application programming interface (API) interactions. This architecture allows for more agile development, where different application components can be developed and improved by different teams much faster and more reliably, using continuous integration and continuous development lifecycle (CI/CD) principles and tools.

However, this architecture poses a security challenge. The overall security and robustness of application functionality heavily relies on the security of the APIs. Some of these API requests and responses travel in their entirety through the internal infrastructure. On the other hand, a good portion of the API calls toward these microservices come directly from outside API clients — deployed by Equinix customers — and so originate from outside the Equinix network. At Equinix, we aim toward a zero-trust infrastructure and avoid treating internal API and external API control plane security with different assumptions and premises. However, given that we have much better control over our software development lifecycle and our deployment environment, the externally exposed API and the path that it takes through our infrastructure requires more scrutiny.

This article has been written with these externally sourced API requests in mind, however, the same principles apply to our internal APIs as well.

The “dos and don’ts” of deploying API security

I am a fan of perimeter security when it comes to ingesting APIs from the outside world. Build a castle, then build a moat around it with a drawbridge and guards. But what does this mean when you get out of the medieval era and move into the modern world?

• Use a fortified API gateway that exposes HTTPS to receive API requests.
• Put a web application firewall or something that can catch your OWASP top X API threats.
• Do your first pass input validation right at the edge—at the firewall and API gateway—and use any other edge-capable security tool.
• Check user, service account or client (anything that is sending an API to you) also at the edge, allowing in only authenticated users, accounts or clients to make the API request.
• Don’t put all the credentials or user databases at the gateway, instead use a classic RADIUS-like AAA model with the API gateway acting as an authentication edge proxy for your authentication service at the backend. This way you can delegate your authentications to any type of identity provider in the market and, hopefully, even get some GDPR mileage out of your vendor.
• Same thing with authorization: delegate the authorization to some service that can do role-based access control (RBAC) and granular service level authorization. This means avoiding generic, “buffet style” all you can eat tokens and forcing the authorization to be specific to a service so that any lateral movements within the system can be avoided. Just because you came in through the door, should not mean you can get anywhere and eat anything: follow a zero-trust model and scope your authorization tokens, tie them to the microservices that you deem are warranted, but please don’t add any personally identifiable information (PII) inside the token, unless you intend to encrypt it all the way to the client.

Greater API intelligence means greater API visibility

There are new API intelligence tools on the market that give you visibility into what API is coming through your gateways. They not only show you every API request and response that goes through your gateway, but also which endpoints they are going to and coming from. For a security team, this is a great way to build visibility into API traffic, get organized, find base patterns and see anomalies — whether they are software or deployment anomalies or they are actual security incidents that need more attention. Whether you call this machine learning, advanced monitoring, supervised learning or configuration is up to you. Either way, some leg work is obviously required. For instance you will need to identify your endpoints with tags that make sense for your analysis; whether it is tagging by product name, cluster name, environment type (dev or production), or even the name of the product engineering lead to send emails to—it’s totally up to you.

Clear API specifications can speed anomaly detection

For architecture teams trying to set architectural patterns, API standards and build maturity for the API ecosystem, ask the product team to provide an API specification for the new product, load it in the tool and detect any runtime violations up to your sensitivity level. For example, was the API expecting specific header or specific types for a payload and is what you see in the field not matching? You can catch that or even block it. Whether you think you are dealing with reconnaissance or malicious payload injection patterns, you can detect that as well and investigate accordingly based on what you think the severity level of the anomaly is.

Some of these tools even tell you which source the API is coming from. For example, if it is external, which IP address is it coming from? Is it a special client associated to a customer? Is it a known user using that special client? Have the customer support team close by to let your customer know you are receiving anomalous traffic from their IP address or employees or clients.

Knowing base patterns and being able to detect anomalies goes a long way even for dealing with internal traffic (inside the castle) that is outside the norm. Is it somebody who is stress testing the wrong production endpoint instead of using a dev environment endpoint? Or is it an attacker that is inside and performing exfiltration acts? This is where having a good inventory of internal IP address ranges comes in handy during anomaly investigation. One example is an inventory of IP ranges that product workload clusters (say Kubernetes) are using. You can even talk to your SRE team and find out if the Kubernetes clusters are all using up-to-date OS for their work nodes and if they are pooling container images free of CVEs from the image repositories or not. And if not, then it is time for your product team to address those old JIRA tickets to deal with those CVEs. Similarly, verify your service architecture to validate which clusters are talking to each other and ensure you have enabled secure communications between them.

Before we close

One important thing to consider is examining sensitive and personal data flowing through the APIs for your privacy compliance. How far through the infrastructure do those sensitive payloads go and how are they protected? Is the masking the product team applied really effective?

Following the API is one of your best ways to figure out what sort of consent is needed from users and where you need to go to delete stuff when users want to be forgotten. Also, note that sensitive data is not only the PII, but include things that are business sensitive that your customers don’t want their competitors to find out about. A tool that gets you visibility into what flows through your API infrastructure can be supervised to learn, or customized to detect, what is considered sensitive information by your customers and therefore for your company.

Learn more about the Equinix API Playground.