The Importance of Context in AI x Infra: Your Infra is a Graph
In this talk
Roxane Fischer explores why most AI tools fail at infrastructure tasks: they lack the full picture. Without awareness of existing cloud resources, dependencies, or configuration drift, tools like Copilot and Cursor miss key context needed for safe, accurate code generation.
She introduces a new paradigm: viewing infrastructure as a graph. By treating resources as nodes and their relationships as edges, developers can map real-world systems more accurately. This shift enables AI to understand the true shape of infrastructure—not just static files, but the live environment too.
Fischer shows how any shift’s agents outperform traditional tools by integrating live cloud data. With access to both code and runtime environments, they identify issues others miss, like insecure IAM users. Her message is clear: contextual AI isn’t optional—it’s the future of DevOps.
Understanding the Role of Context in AI-Powered Infrastructure
Roxane Fischer, CEO and co-founder of any shift, presented a compelling talk at AI Native DevCon titled "The Importance of Context in AI x Infra: Your Infra is a Graph." The session focused on the pivotal role of context within AI applications aimed at infrastructure management and code generation. Fischer's insights tackled both theoretical challenges and pragmatic solutions within the DevOps domain, setting a foundation for enhancing AI's impact on infrastructure through contextual awareness.
The Challenge of Context in AI Tools
Fischer began by highlighting the mission of her company, any shift, which develops AI DevOps agents that automate mundane processes like converting GitHub issues into pull requests. She pointed out the limitations of existing AI tools, which often lack the contextual insight necessary to generate effective infrastructure code. "Spoiler alert: your infrastructure is a graph. And so you should see it as is," she remarked, emphasizing the necessity of viewing infrastructure as interconnected nodes and relationships.
In a practical example, Fischer described the typical developer scenario of creating a VPC peering connection in Terraform. She pointed out the pitfalls of hardcoded values and dependency mismanagement, which often occur due to a lack of comprehensive context. "Very often…you will have like hardcoded values and also leftover resources in your cloud because it's never perfect," Fischer noted. Although tools like GitHub Copilot offer improved coding practices, they still miss crucial context, such as existing VPCs.
Limitations of Current AI Tools
Fischer explored why tools like Copilot and Cursor fail to meet the mark. While they can generate code snippets, they don't possess a holistic understanding of the environment. "They don't have the entire context of your infrastructure and so they are missing those dependencies," she explained. Even tools that map across code files, like Cursor, couldn't connect code with live resources that only exist in the cloud.
During a live demonstration, Fischer contrasted the capabilities of any shift’s AI agent with Cursor when tasked with identifying insecure IAM users. Cursor could only assess risks based on infrastructure-as-code repositories, whereas any shift’s agent could also refer to the live cloud context. Fischer illustrated, "Because cursor didn't have…cloud context it couldn't find this [temporary Goof] user which was obviously a security issue," underscoring the importance of unified context in generating accurate remediation strategies.
Infrastructure as a Graph: A Paradigm Shift
Transitioning towards solutions, Fischer reiterated her thesis: “Your infrastructure is a graph.” By visualizing infrastructure as a network of nodes and edges, developers can better understand resource interconnections. However, accurately mapping this graph is complex due to multi-cloud environments and ephemeral resources. Understanding these connections requires defining meaningful edge labels and creating deterministic mappings between code and live resources.
Fischer proposed two principal approaches to construct this context-rich graph representation. The first involves using graph databases to maintain a comprehensive knowledge graph that integrates code, cloud resources, and dependencies. “It's great because [you have] your entire graph in context with super high quality data,” she stated, while acknowledging the maintenance challenges. The second approach involves managed cloud provider servers, which offer real-time infrastructure data, albeit with limitations on scalability and API constraints.
The Future of Contextual AI in DevOps
Fischer concluded with a powerful assertion: “Context is king.” She argued that AI's ability to generate precise code for a given stack fundamentally relies on understanding infrastructure as a graph. Fischer expressed optimism about the future impact of contextual AI in DevOps, crediting community efforts and partnerships for driving advancements in this area.
