Harnessing the Rise in Collaborative AI Development

As organizations increasingly depend on data to support and automate decisions and actions, they are realizing that collaboration can help them get there faster. Collaborative analytics enable disparate organizations to collaborate on advanced problems, pull in data from disparate sources, including the edge, and use a variety of technologies and tools to gain deeper insights.

For many compelling reasons, collaborative edge analytics have found a home within the 210+ secure, globally interconnected International Business Exchanges™ (IBX^®) data centers of Equinix. As the world’s largest carrier-neutral interconnection platform, Platform Equinix^® provides businesses inherent low-latency proximity to hundreds of clouds and thousands of networks – all of whom use Equinix as their global service access and traffic distribution points. Today the power of interconnection (private peering) at the edge of clouds and networks has extended well beyond service providers who have been leveraging it for decades. More than ever, public and private sector entities alike are discovering the advantages of proximate, private peering with digital service providers and one another across geo-strategic edge locations at Equinix.

As cloud adoption continues to grow exponentially, we are also witnessing our customers increasingly depend on the growing number of cloud adjacent digital services at Equinix to collaborate, innovate and harvest value. As a case in point, the COVID-19 pandemic highlights the need for speed, efficiency and better insights in inter/intra agency collaboration. Artificial intelligence (AI) can help but it requires a neutral place where data providers and consumers can securely interact, transact and exchange data. Our AI Data Marketplace, provided in partnership with Dell Technologies, Nokia and a growing number of others playing a key role in this new approach to data exchange, offers a sound approach for solving this.

Governance is essential for sharing data

Few will argue the need for collaborative informatics to solve problems with timely, actionable insights. Each agency often holds an integral piece of a larger puzzle that can only be solved through inter/intra agency collaboration. And, as a previous blog, “How to Accelerate Government AI Initiatives with Data Sharing” outlined, large volumes of diverse data are needed to train AI models. The key challenge in facilitating this relationship is in establishing a mutually agreed upon data governance and stewardship strategy for the trusted sharing and exchange of an organization’s most coveted asset with confidence.

Data marketplace proof of concept with airlines

Along with our technology and data science partners, Equinix was able to help solve these challenges through a productive proof of concept (POC) with an American and European airline. Both had a need for collaborative analytics but were averse to openly sharing their respective data. The aircraft manufacturer also played an integral role in the process. Our AI Marketplace at Equinix POC addressed this challenge through a consortium-based federated learning approach. A physically and logically secure meeting place was established with clearly defined rules of engagement, as well as mutually agreed-upon governance and operational methodology. A trusted third party provided oversight and full lifecycle data custodianship.

Source: Equinix AI Data Marketplace white paper

A federated model can be useful in the following scenarios:

• Keeping raw data private: When data providers do not want to let raw data out of their security domain, federated learning can be leveraged to build the model locally and then share the anonymized model with the data consumer.
• Data volume: When large volumes of data are generated at the edge, the data provider can build a local AI model and ship it instead of the raw data to reduce the cost of backhauling large datasets.

Consider a scenario whereby a global agency such as the Centers for Disease Control (CDC) needs to collaborate with literally hundreds of mission partners, supply chain entities and others in response to a global pandemic. To do that, it must acquire, transport and deposit multiple streams and sources of data into an analytics engine for processing. As illustrated in the diagram below, localized last mile access to a corresponding Equinix location can enable rapid acquisition and transport of data across our global, software-defined fabric to a pre-determined analytics terminus. And, it’s highly likely that many of the target collaboration partners are already leveraging Platform Equinix for private interconnection with clouds, networks and interagency peering.

Source: Equinix  Fabric^TM is a software-defined interconnection that enables on demand private transport to/from any Equinix location in minutes. See the Equinix Fabric™ for the most current map.

Any of the locations illustrated above can represent tiered and/or final destination data exchange locations. For example, participating healthcare facilities across the Australian continent can readily extract qualified collaborative subsets of data and executable algorithms and leverage this platform to rapidly and securely share it with the global network of disease surveillance participants.

The Equinix AI Data Marketplace

Data marketplaces allow data providers and data consumers to share, buy or sell data and algorithms privately and securely using a programmable, community-owned safe infrastructure that organizes trust. Sharing data assets via a securely interconnected, neutral data marketplace, governed by a trusted third party such as a consortium, is a promising opportunity for government agencies meeting their missions in the 21^st century. The following illustration provides a high-level structural depiction of an Equinix AI Data Marketplace:

There are many data marketplace solutions available, but they may not fully address the data sharing challenges and concerns faced by government agencies. As noted above, an Equinix-based Data Marketplace provides cloud adjacency and/or isolation from cloud as warranted by the active data sharing participants. These secure enclaves are built in NIST 800-53/FISMA High Certified facilities to ensure the marketplace, analytics and infrastructure operate within and will provide a consistent baseline of security for operations.

To gain a deeper understanding of how a data marketplace can help accelerate AI-based innovation for your agency, read the AI Data Marketplace white paper and download the playbook.

You may also want to schedule an interactive virtual Digital Edge Strategy Briefing.