Laying the foundation: Data infrastructure is instrumental for successful AI projects

Written by Jens Eriksvik

Data infrastructure is the backbone for enabling successful artificial intelligence projects. It consists of the ecosystem of technologies and processes that govern how businesses and organizations collect, store, manage, and analyze the operational data that fuels its AI initiatives. Without a robust data infrastructure, driving successful AI initiatives becomes almost impossible – your journey will likely grind to a halt after a few implementations.

• AI models are data-driven
  Machine learning relies heavily on the quality and quantity of data used to train models. Inefficient data infrastructure, filled with inconsistencies, fragmentation, and limited accessibility, will significantly undermine the effectiveness of these models.

• Data quality determines performance
  "Garbage in, garbage out" applies. Unreliable or inaccurate data leads to flawed insights and ultimately compromises the decision-making capabilities of AI systems.

• Scalability is essential:
  Models should regularly be retrained to accommodate changes in the underlying data. Which means a constant input of new data into the infrastructure. As AI projects evolve, more data will be required from new sources. Additionally, generative models will input more data into the system. A scalable infrastructure is therefore critical to accommodate the AI journey.

Getting the data right is hard

There are multiple reasons why businesses are struggling with their data infrastructure. The volume, variety and velocity of data is increasing, the lifespan of IT systems is becoming shorter, and the fragmentation across different sources is increasing. Moreover, the regulatory environment is becoming increasingly complex, and M&A and reorganisation activities are constantly undermining the data structures. This puts a strain on the organizational setup, tools, IT systems and governance that a business puts in place. 

In short, while demand for quality data increases, so does the data turbulence, fragmentation and regulation. Getting it right requires careful consideration for the business context and strategy.

Building a data infrastructure 

A business data infrastructure should be designed based on the use-cases it intends to support. A business that only focuses on generative AI for office worker use, has vastly different requirements than machine learning for automated parameter setting for industrial applications. There are of course many different areas and domains of AI that should be covered, but starting with three prominent areas illustrates the focus of the data infrastructure design: Predictive machine learning models, Generative AI and Computer vision. 

Exhibit 1: Data requirements, infrastructure considerations and data management strategies for different AI domains

The above exhibit 1 summarizes the key considerations for data, infrastructure, and specific data techniques across three AI domains: Predictive machine learning, Generative AI, and Computer vision. 

While all domains necessitate high-quality and diverse training data that accurately reflects the real-world problem the AI model is aiming to solve, the emphasis differs. Predictive ML prioritizes feature engineering for optimal model performance, while Generative AI thrives on large-scale datasets and potentially utilizes synthetic data generation. Computer vision often requires annotated data, such as bounding boxes, for tasks like object recognition. All domains benefit from scalable infrastructure with significant computational resources, often involving GPUs. Robust data storage and processing capabilities are paramount as well. 

However, specific data techniques diverge. Experimentation platforms are crucial for efficient model development in Predictive ML, while Generative AI necessitates model monitoring and evaluation to ensure unbiased outputs. Computer vision leverages data augmentation techniques and transfer learning from pre-trained models to enhance training efficiency. 

Data strategies will however differ from use-case to use-case, requiring organizations to be diligent in how they handle data based on what they want to use it for. Even within the three presented domains, strategies can be widely different based on the desired outcome of the AI solutions. However, there are some considerations to make around each domain that are worth noting, especially around labeling. Labeling of computer vision data for supervised training will require special pipelines. Setting these up in an efficient manner will be critical to ensure quantity and quality of the data. Labeling data for Gen AI and predictive models might be easier and could be made automatic. However, that requires good governance, ensuring proper metadata and good relational connections between entities and data points. 

Building a robust data infrastructure for AI

Unfortunately there is no easy fix to the data infrastructure problem. It requires businesses to work through the data lifecycle from data creation, data management and maintenance across the stack, from infrastructure to business. To lay the foundation for enterprise-grade adoption of AI and to become an algorithmic business.

1. Governance

• Clear data governance policies, standards, and procedures to ensure data quality, security, and compliance with regulations.

• Define roles and responsibilities for data management, including data stewards, data custodians, and data governance committees. Work cross-functionally across business, legal, tech and other support functions. 

• Implement data governance tools and technologies to enforce policies, monitor data usage, and track data lineage across the organization. Keep track of used data to train your AI systems.  

• Implement infrastructure and tools that support ethical AI practices, such as explainable AI (XAI) techniques that provide insights into AI decision-making processes.

2. Security

• Implement cybersecurity measures to protect sensitive data from unauthorized access, breaches, and cyber threats.

• Encrypt data at rest and in transit, implement access controls and authentication mechanisms, and regularly audit and monitor security controls.

• Conduct regular security assessments, vulnerability scans, and penetration testing to identify and mitigate security risks proactively.

• Implementing anonymization of data wherever possible lowers risk of sensitive personal data leakage.

3. Organization and skills

• Establish dedicated AI teams comprising data engineers, data scientists, AI researchers, and domain experts to drive AI initiatives effectively.

• Foster cross-functional collaboration between AI teams, IT departments, and business units to align AI infrastructure with organizational goals and priorities.

• Invest in training and upskilling programs to develop talent in AI, data engineering, and data science, ensuring that teams have the necessary skills and capabilities to succeed.

4. AI Infrastructure

• Design and build scalable AI infrastructure, including hardware accelerators (e.g. GPUs), distributed storage systems, and specialized AI frameworks (e.g. TensorFlow, PyTorch).

• Optimize infrastructure for AI workloads, including model training, inference, and deployment, to achieve optimal performance, efficiency, and cost-effectiveness.

• Leverage cloud-based AI services and platforms to accelerate development, simplify deployment, and scale AI applications rapidly.

• Design resilient and elastic architectures that can scale dynamically to handle peak workloads and accommodate future growth.

• Leverage containerization and orchestration technologies, such as Docker and Kubernetes, to streamline deployment and management of AI applications at scale.

5. Data integration and data platform

• Integration will force mapping of data models on technical and business level. Implement robust data integration processes to integrate separate data sources and systems, ensuring seamless data flow and interoperability. 

• Establish data governance mechanisms for data integration, including data mapping, data lineage tracking, and metadata management to ensure data consistency and quality.

• Implement a comprehensive data platform that provides a unified infrastructure for managing, storing, and analyzing data from various sources. This platform should support scalability, security, and accessibility while facilitating efficient data integration processes.

Getting started is more important than doing everything right

Building a robust data infrastructure is crucial for successful AI adoption, as it forms the foundation for effective AI initiatives. Without a solid data infrastructure, organizations risk encountering numerous challenges that can hinder the progress of their AI initiatives. Key considerations include data quality, scalability, and security, which directly impact the performance and reliability of AI systems.

Start building a robust foundational data infrastructure for prioritized use-cases, identify what needs to be in place across the data value chain for each AI use-case. Use the lenses of governance, security, organization, skills, infrastructure and data integration. This will enable a gradual clean-up of data models, harden security, strengthen transparency and improve success rate of AI projects. 

But, bottom line, businesses need to go through the painful process of cleaning out bad data. And this in many cases requires business-knowledge and manual work.