No agency is an island and gathering, processing and analyzing the volumes of data needed to successfully run government missions and operations requires greater intelligence ̶ specifically artificial intelligence (AI). This has generated significant interest in the Federal Government to invest in AI and related technologies. A report released by the White House states that investments in nondefense AI and quantum information sciences R&D will grow from about $1.4 billion in FY2020 to about $2.2 billion in FY2021, and it is expected to double by FY2022.[i]
In our recent article, “Harnessing the Rise in Collaborative AI Development,” we talked about governance being essential for sharing data and the need for “collaborative informatics to solve problems with timely, actionable insights.” We also recommended a consortium-based federated learning approach to AI and data analysis between agencies and public and private sector partners to accelerate the design, testing and deployment of these intelligent IT infrastructures.
AI Digital Data Marketplace Solution on Platform Equinix Help Government Agencies Manage Data
The demand for data from within and outside of government agencies is increasing exponentially. Government agencies want to use and share their data and algorithms effectively but are faced with security and data control issues.
Read MoreWe developed the AI Data Marketplace test drive environment on Platform Equinix® as a place to build an interactive consortium between government agencies and service providers. It also provides a proof of concept (POC) platform for developing the kind of AI-driven applications that can deliver the intelligence, insights and efficiencies today’s government agencies require.
Why data marketplaces are essential for piloting AI-applications/solutions
Data sharing is a core consideration when developing AI solutions. The amount and diversity of data accessible to AI algorithms determine their functionality and accuracy. As discussed in a previous article, “How to Accelerate Government AI Initiatives with Data Sharing,” to effectively train an AI model, data and algorithms need to be shared and exchanged across a broad ecosystem of data providers and consumers, other agencies, partners, clouds, applications, data scientists and users. Data scientists need access to “ready-now, approved” AI platforms that can be provisioned quickly to prototype and scale AI models. And meeting the demands of training increasingly large AI models and datasets requires high-performance compute power proximate to diverse data sources, fast local storage and high-speed, secure access to multiple clouds and data brokers.
Data marketplaces help address these needs by allowing data providers and data consumers to share, buy or sell data and algorithms privately, securely and compliantly using a programmable, community-owned infrastructure that organizes trust. Governance models regulate how the members of the marketplace interact as shown below:
- Consortium setup and membership registration: The consortium establishes the shared infrastructure for the data marketplace to be governed and administered by the membership organization.
- Asset registration and trade agreements: The consortium provides access credentials to its members so they can register themselves and be admitted to the data marketplace. Members decide what information they want to publish/use and create a data trade agreement. The data trade agreements are stored in an immutable, distributed ledger for auditability.
- Data scientist and production workflows: Data scientists can leverage the data marketplace to experiment and train AI models.
The AI Data Marketplace and government IT ecosystems at Equinix
There are many data marketplace solutions available, but they may not meet the data sharing challenges and concerns faced by government agencies. An agency concerned about sensitive data may want to store it in an on-premises environment or an International Business Exchange™ (IBX®) government-grade data center, while using public clouds for non-sensitive data. The AI Data Marketplace at Equinix provides cloud adjacency and/or isolation from cloud as warranted by the active data sharing participants. These secure enclaves are built in NIST 800-53/FISMA High Certified facilities to ensure the marketplace, analytics and infrastructure operate within and will provide a consistent baseline of security for operations.
Agencies can come to the Marketplace and leverage a vendor-neutral environment to test the interoperability of various partners’ solutions when processing data sets and queries against AI platforms, on-premises and in private and public clouds. On Platform Equinix, you have access to thousands of different service providers, including those specializing in networks, clouds, bare metal compute/storage, databases and analytics, security, and AI and machine learning. Many of the IT partners that are already helping federal agencies in their journey to cloud such as Knight Point, Dell and IBM are part of the interconnected government IT ecosystem at Equinix. And government network providers that provide contract vehicles such as GSA EIS, Networx and WITS are also part of the IT ecosystem at Equinix. By working closely with the best of the best in any given technology, you can “try before you buy” and reduce the risks and limitations of single vendor lock-in.
The marketplace also enables hundreds of federally approved service providers and vendors to test out their solutions and services on a consistent and integrated platform that currently supports government agencies. The AI Data Marketplace allows agencies, service providers and vendors to all have a greater understanding of what it takes to design and test various data and AI environments/scenarios and enable the best results before making an investment in an existing solution or a new development. The diagram below illustrates the different roles within an AI Data Marketplace test environment stack.
Testing to prepare for any mission
The use cases that can be tested on the AI Data Marketplace platform, with its rich ecosystems of service providers and vendors, are unlimited. For example, you can use predictive models to test out various natural disaster emergency scenarios, such as an earthquake, hurricane or wildfire, for deploying first responders and planning long-term recovery strategies. Or you can test the meantime for repair of military vehicles and aircraft and their components to better perform predictive maintenance and anticipate and reduce costs over time. This regular data analysis practice lowers the risk of losing property and lives, while increasing overall safety and success of the mission. In the example below, data from different military aircraft is being collected, processed and analyzed within the Equinix AI Data Marketplace leveraging a data mining as a service architecture. The Marketplace has all of the components and providers needed to test out different AI-based data analytics scenarios and decide which gives the most accurate intelligence for the best mission outcomes.
Getting started with a Digital Edge Strategy Briefing
An interactive virtual Digital Edge Strategy Briefing with a team of Equinix government industry experts is where agencies, service providers and vendors can start the journey toward testing out AI-enabled applications and solutions. Gaining a solid understanding of an AI solution’s outcome by performing a multi-vendor POC for testing different variables is critical for ensuring your mission’s success. Contact us to schedule an interactive virtual Digital Edge Strategy Briefing.
The AI Data Marketplace POC platform
Located just 60 miles from Washington, D.C., our Culpeper IBX data center campus has the most secure and technologically sophisticated colocation facilities on the East Coast. It hosts a robust and fast-growing ecosystem of federal government agencies, network and cloud service providers, and data processing and AI technology vendors. This where you can deploy your POC test bed on Platform Equinix and remotely monitor the results. To learn more, read the Equinix AI Data Marketplace.
[i] The White House Office of Science and Technology Policy, “ARTIFICIAL INTELLIGENCE & QUANTUM INFORMATION SCIENCE R&D SUMMARY: FISCAL YEARS 2020-2021,” August 2020.
Subscribe to the Equinix Blog