Why Public Internet Routing Is Breaking Enterprise AI Performance

AI success isn’t just about hardware and models. Where organizations host workloads matters, as does the connectivity across those distributed workloads. When enterprises place workloads in an infrastructure environment they don’t fully control and connect them via the unpredictable public internet, it inevitably leads to poor results.

In the early days of AI, organizations were primarily concerned with speed and convenience, so cloud-native infrastructure and internet connectivity made sense. Now, as they shift from experimentation into production, many of them are realizing they need something more dependable.

By moving AI inference workloads inside your environment and connecting them with private interconnection, you can maintain the control needed to optimize results and minimize risk.

For better AI performance, place workloads inside the network

There’s a data center spending spree happening today, driven by increased demand for AI capacity. Unfortunately, many new data centers aren’t designed for proximity and low latency. They’re built where it’s most convenient, not where they’d be most effective. Any data center that’s located far away from the workloads can’t truly be called AI-ready.

Since data center demand is ramping up, it stands to reason that enterprises also need more network bandwidth to connect those data centers. In some cases, the backends of AI systems are so large that traditional networking can’t keep up. This is forcing many organizations to rethink their approach.

Equinix recently helped one major hyperscaler implement private connectivity between two U.S. metros, and we had to set hundreds of cross connects to meet their bandwidth needs. Although most enterprises aren’t moving anywhere near as much data as the hyperscaler in this example, they’re still moving much more than they’re used to, and they’re having to make decisions about how to increase capacity without sacrificing control.

Case study: Dow Jones Newswires

Dow Jones Newswires is a news service catering to high-frequency trading (HFT) firms. Their clients execute trades at machine speed, and they need insights available within microseconds. This requires solving both sides of the latency equation:

• Zero use of public internet anywhere in the data path: Dow Jones Newswires uses Equinix Fabric® for direct, predictable routing.
• Infrastructure where the trades are happening: The company ensures proximity by meeting customers where they are: inside an Equinix data center.

Most enterprises won’t have latency requirements as strict as those in HFT, but they can still learn from this example. It shows what’s possible when customers trust Equinix to optimize both performance and proximity. Read the full Dow Jones Newswires story.

How to build an AI experience that keeps users coming back

Optimizing AI infrastructure has become a user-experience imperative. Chatbots are the primary way human users interact with AI these days. There’s very little friction for users to switch chatbots, and if you give users a reason to leave, they will. User experience is everything.

Latency impacts user experience, but what really matters is dependably low latency. The one thing that users won’t tolerate is variability. Consider two examples:

• Application A consistently experiences 100 milliseconds delay before responding to a user.
• Application B usually experiences 50 milliseconds delay, but occasionally 500 milliseconds.

From an experience standpoint, Application A is preferable, because users know what to expect. With Application B, it’s less about the delay itself, and more about the potential unmet expectations. This underscores the importance of deterministic, low-latency routing for AI inference.

Optimizing the internal user experience

Employees are users too. They’ve become accustomed to using AI in their personal lives, and they expect to use it at work as well. Enterprises need a private AI strategy to support these employees. Otherwise, they may pursue shadow AI. This means using their chosen models with no centralized oversight or governance, putting company data at risk.

An effective private AI strategy must include performance SLAs and traceability for data in motion. The public internet provides neither.

Case study: Air Canada

Air Canada supports more than 120,000 customers daily across 180 global airports. They strive to make customer interactions as frictionless as possible. To achieve this, they adopted global hybrid multicloud networking. They used interconnection solutions like Equinix Fabric and tapped into our industry-leading portfolio of native cloud on-ramps.

With easy access to cloud services, Air Canada implemented advanced AI and machine-learning capabilities, which they use to enhance the customer experience. Read the full Air Canada story to learn more.

Deploy private inference to maintain control over sensitive data

Many enterprises have special security and compliance requirements, and the right inference infrastructure and connectivity will go a long way toward meeting those requirements.

The internet uses best-effort routing, where traffic bounces between exchange points to get from source to destination. This means that enterprises have no control over the route their data takes. Data could cross national borders, potentially breaking sovereignty regulations. Even though the violation is unintentional, the enterprise could still face stiff non-compliance penalties. And since public internet connectivity doesn’t provide traceability, they’d have no way of knowing that the infraction even occurred.

Enterprises must also be careful about exposing sensitive datasets to cloud infrastructure. Putting data into a cloud-native environment means making that cloud provider solely responsible for protecting it. This is a gamble that some enterprises can’t afford.

They also have to be vigilant against potential vendor lock-in. If the enterprise becomes overly dependent on a single cloud provider to drive their entire AI strategy, what happens when that provider changes the terms of their contract?

Enterprises need global reach and infrastructure flexibility, so complete isolation from the cloud is not the answer. They need a way to use cloud infrastructure on their own terms, without limiting themselves or putting data at risk. This could mean deploying a hybrid multicloud architecture:

• Using private inference infrastructure for sensitive datasets that must remain outside the cloud
• Moving other, less sensitive datasets into the cloud when it’s beneficial to do so

When enterprises incorporate hybrid multicloud into their AI strategies, they must use private connectivity to reach cloud services. This allows them to control exactly which datasets they move into the cloud, which route that data follows to the cloud, and what happens when that data’s no longer needed in the cloud.

If they pair private connectivity with a storage environment that they control, it helps protect against vendor lock-in. If they no longer need a particular cloud environment, they can sever the connection and walk away. They’ll still have authoritative copies of their data inside their storage environment. And if that storage is located inside an Equinix vendor-neutral colocation data center, they’ll have access to cloud on-ramps from all major cloud providers, making cloud-to-cloud switching quick and easy.

Case study: Drug discovery and development

One Equinix customer, a prominent life sciences company, deployed private inference infrastructure to protect their extremely sensitive data. In this case, data is the single most important asset the company has. Without their proprietary drug formulas, they’d have nothing of value to offer.

They knew they couldn’t trust that data to an environment they don’t control completely. They deployed at Equinix to enable parallel environments across all cloud hyperscalers. This means they can easily switch to whichever environment is best for their needs at the moment.

Equinix also helped the customer deploy dedicated connectivity to ecosystem partners with built-in visibility and control. For instance, they created a connection that was restricted to a single IP address at their hardware provider. This allowed support staff to securely access the customer environment, with all activity properly monitored and logged.

What this means for enterprise leaders

With all the different ways that proximity and latency impact AI workloads, the network should be the first place leaders look when they’re trying to diagnose poor AI results. Yet all too often, they look to models or hardware instead.

Since the network touches everything in their distributed AI architecture, any optimizations they make at the network layer will create a chain reaction that leads to better results across models and infrastructure:

• With the right connectivity deployed on a vendor-neutral infrastructure foundation, enterprises can ensure model choice. They can find the right model for the job and connect to it without delay. This helps optimize cost-efficiency, performance and reliability.
• For organizations deploying their own GPUs, low-latency connectivity enables a reliable data pipeline and higher utilization rates. Those that rely on cloud providers for their AI infrastructure can do so in a way that supports their business goals, instead of risking vendor lock-in.

Equinix offers the services needed to help enterprises with their AI goals:

• Our colocation data centers are vendor-neutral interconnection hubs that span the globe, so customers can always deploy inference infrastructure and find ecosystem partners in the right places.
• Our interconnection solutions provide control and reliability that the public internet doesn’t. For instance, Equinix® Fabric One™ is our managed, intent-based any-to-any connectivity service. It enables the infrastructure flexibility that distributed AI demands.
• Equinix® Inference Exchange is our distributed AI inference solution. It can be deployed in days inside an Equinix data center, with pre-connected access to customer data across every major model, cloud, and AI service provider.

Both Equinix Fabric One and Equinix Inference Exchange are part of Equinix® Connected Cloud, our neutral foundation for distributed digital architectures. It represents a simpler way to operate across the world’s most trusted neutral AI exchange.

To learn more, read our solution brief, “Land once, reach every cloud and AI environment.”