Many companies are looking toward agentic AI to solve increasingly complex problems. While specific definitions vary, the fundamental promise of agentic AI is that a user can define high-level goals and have the system decompose that goal into tasks and then execute them autonomously. Such systems are typically characterized by some kind of persistent memory and the ability to make API calls (“tool use”).
Unfortunately, most such systems are plagued by limitations that make them unappealing to use in practice:
- Usability: Getting agentic systems to do what you want can be frustrating and require knowledge of the underlying implementation.
- Efficiency: Foundation models (e.g., LLMs) are already resource hogs. Setting a swarm of them loose on an open-ended problem is a recipe for eye-popping costs.
- Fragility: Getting agents to work on your own system is hard enough. If you want to deploy them on other systems with different data, good luck.
- Data Availability: The information required to complete a task is often scattered across multiple systems and fraught with issues (like cleanliness) that taint results in unpredictable ways.
In this post, we’ll explore how Graft addresses these problems to build unusually powerful agentic networks. First, we’ll spill the tea on our secret sauce. Then, we’ll enumerate some of the resulting properties.
The Secret Sauce
What makes Graft’s agentic networks capable of so much more than swarms of agents? The secret lies in how our networks are defined and what the agents in them are capable of.
The Network
Graft consists of a Declarative Agentic Network Grammar (DANG) to define end-to-end agentic networks with access to extensive data and compute resources, plus a system that instantiates them. In the default case, that system is Graft. Those data and compute resources include modifying database tables at scale and heterogeneous compute for performing machine learning and data science (ML/DS) tasks.
This separation of the what from the how pays huge dividends, which we explore below. Collectively, we refer to these networks as the intelligence layer.
The Nodes
Unlike RAG and enterprise search tools, which only connect agents with search, Graft agents are also imbued with capabilities like structured analytics (via multi-table NL2SQL) and predictive analytics (agents build, apply, and update models). For example, some of Graft’s agents perform data cleaning. On our own data, which we know the best, those agents contributed a 10% increase in relative accuracy over networks without them.
If RAG agents are AI+Search, Graft agents are AI + Search + SQL + ML/DS.
It can be instructive to think about agents as falling into two categories: workflow agents and query agents (see Figure 1). What I’ve described above are workflow agents, and they can be visualized directly in Graft via an Agentic View (see Figure 2). Query agents, on the other hand, employ a variety of strategies to route user queries to the right parts of the intelligence layer.
What a Difference a DANG Makes
With more powerful agents and a declarative network definition—separate from the implementation and the data—some truly exciting and differentiating properties of Graft’s agentic networks emerge.
Holistic
A DANG defines how data is ingested, prepared, and enriched with various AI tools spanning predictive, structured, search, and generative use cases. That end-to-end specification gives Graft full visibility into data provenance and artifact dependencies, both of which are typically opaque in networks where all collaboration is ad hoc. As we’ll discuss below, this visibility translates into better performance and steerability.
Graft, as the default implementer of these networks, handles all the infrastructure, orchestration, and monitoring. Outside of what is laid out in the spec, the implementer is free to make implementation choices based on the available resources or other optimization criteria.
Portable
The grammar abstracts the implementation and data specifics from the specification. Networks, including all intermediate artifacts, are instantiated at deployment time. This means a DANG spec need not include any sensitive information, even in contexts where the network is designed to operate on such information.
For instance, say that you have a suite of tools for developer productivity. These tools include your company’s source code, tickets, customer information, employee information, and so on. You can build a network on top of this data, which might include predictive models or other derived artifacts, and then hand that network to someone else without compromising data privacy.
The recipient of this network may have different terminology, processes, or operational characteristics, but the network will learn all of that when the network is implemented on the recipient’s data. For instance, the network in Figure 3 estimates ticket completion times, and would build an entirely separate predictive model when instantiated at another organization.
Thanks to DANG, Graft supports rapid domain adaptation and portability.
Usable
We built Graft to make the AI of the 1% accessible to the 99%. Users can define and use these agentic networks via UI, chat, search, and APIs for a range of user sophistication.
When you have a problem and a swarm of agents, you actually have two problems: the original one and the question of which agent you should ask. LLM routers, which claim to help with this problem, presume (often incorrectly) that there exists a single agent capable of responding to the query.
“Oh good, a swarm!” — Nobody
Instead, Graft presents a unified interface to query these networks. Rather than a swarm of agents, Graft uses an intent mesh to direct user interactions to appropriate parts of the network. That means, regardless of the complexity of the underlying network, Graft presents a simple interface to users.
That unified chat interface satisfies queries using different strategies:
- Search (RAG++): Retrieval augmented generation is popular for a reason: it’s a good way to ground a response in the data when that data vastly exceeds the context window of a model. Graft’s implementation goes quite a bit further than generic RAG (semantic+lexical search plus a reranker), but the architecture is similar.
- Structured (SQL): Some questions require exact and auditable answers. If you ask Graft such a question, it may write and execute multi-table SQL to get the answer. See Figure 4. Users can inspect the generated queries to confirm their correctness.
- Follow-up: Previous responses, and the artifacts they generated, might be the explicit subject of a follow-up query.
- Pass-through: For general knowledge questions, Graft may simply pass the query along to a foundation model.
- Meta: Graft can answer questions about the network itself, such as what data is being pulled from which data sources or where a specific prediction came from.
- Clarification: Natural language is ambiguous. If Graft sees multiple interpretations of a query, it might ask follow-up questions of its own.
Some queries may involve multiple strategies and leverage multiple parts of a network. For example, if you ask for a table of ARR lost, organized by customer objections, that might touch parts of the network that are reading unstructured sales call transcripts to extract customer objections and use a query strategy that generates code to do the math.
Performant
Graft achieves better performance, both in terms of quality and resource consumption, compared with simplistic chains of LLMs or swarms of agents. It accomplishes this through artifact reuse, implementation freedom, model agnosticity, and better abstraction of implementation details.
Many agent networks are spectacularly wasteful. They generate intermediate results or predictions that are then promptly thrown away, requiring that they be regenerated for subsequent queries. Graft, by contrast, persists key artifacts and reuses them both across queries and across different parts of the network.
Because Graft builds the implementation, it’s also empowered to adjust that implementation with the aim of improving result quality or performance. For example, Graft could duplicate a network node that applies a certain model and then modify the copy to use a different choice of model. A user might be presented with an eye test—”do you prefer the results on the left or the right?”—where the difference lies not only in the models used but also in other implementation parameters like retrieval settings or predictive model hyperparameters.
Not all models are created equal. Some are only proficient within a restricted domain or modality, shedding superfluous expertise in irrelevant domains and getting more efficient as a bonus. Some trade overall quality for speed and cost, and you can save huge amounts of resources in contexts where that trade-off is acceptable. Graft was built from Day One to be agnostic to the choice of foundation models. Have a proprietary vendor you prefer? Drop in your API key. Want to use a variety of open source models? Have at it. We host a curated selection of open source models to choose from. You can even point Graft to your proprietary model APIs and it’ll use that. And you can use a combination of all of these in a single network and change your selections at any time via a dropdown menu.
Finally, because DANG separates the specification from the implementation, there’s a satisfying answer to the question of how performant Graft is: as performant as the resources you provide to it. In the default case of Graft implementing the spec, we’ve selected best-in-class components that auto-scale to fit instantaneous demand.
Extensible
Another useful property of DANG is extensibility. Graft has over 150+ live data connectors, can employ any combination of foundation models, handles multiple modalities (tables, text, images, video, audio, code, logs, etc.), and allows users to construct complex workflows by linking agents together. Figure 5 shows a query that uses tables, text, code, and logs. Those are all increasingly common properties of agentic systems.
Graft’s enrichment architecture goes further, allowing users to encode expert knowledge embodied as machine learning artifacts (e.g., classification models or labeled examples). These artifacts and the outputs they produce can be consumed by other parts of the network, allowing them to employ a new skill. These skills often encode domain expertise or tribal knowledge, offering a much more explicit and controllable way to customize these networks to fit the needs of an organization than expensive crapshoots like fine tuning.
Steerable
One size does not fit all, but many agentic systems limit feedback to good/bad or only route that feedback to humans. Instead, DANG enables Graft to accept different kinds of feedback and apply them automatically.
The topology of a DANG network defines the provenance of information. Graft can use this provenance to route feedback to the source of the error. In Figure 6, we modify Figure 3 to highlight the path data takes to an agent that does predictive modeling, which Graft can trace back when deciding how to apply feedback.
For example, a response could include a simple factual error: “Customer X will renew their account at $Y/yr,” when you may know for a fact they are not renewing. While one could certainly append the correction to a list of feedback and pass the list as context to every query, that’s a rather ham-fisted approach. Similarly, including that correction as part of a fine-tuning dataset is an expensive option that, it must be emphasized, does not guarantee the tuned model has even learned the fact in question. Instead, DANG tells Graft where the fact originated, which in this example is a prediction from a machine learning model. In that case, not only can Graft explicitly correct the factual error, but it can retrain the model that produced it along with anything downstream that consumed it. The result is a surgical point correction that propagates through the network automatically, thus improving future queries that rely on that fact in any way.
Further extending that example, imagine that the factual error was part of a broader issue with messy data. In the Graft UI, a user can enter an active learning workflow that prioritizes the examples that will have the most impact on the model. After making a few corrections, the model might be able to generalize the corrections and fix the rest of the data. As I mentioned at the top, models like these have improved the relative accuracy of our internal networks by 10%.
Surgical feedback routing supports multiple kinds of feedback, including implementation parameter exploration and point feedback with active learning that generalizes and propagates through the network.
Conclusion
Graft’s Declarative Agentic Network Grammar (DANG) separates the structure of the solution from the instantiation of it on specific infrastructure and data. The nodes in those networks have a more robust suite of capabilities, including the ability to perform structured analytics and predictive modeling. As a result, Graft’s agentic networks are more holistic, portable, usable, performant, extensible, and steerable. While we don’t consider any of these issues ‘solved,’ we do believe we’ve made significant progress.
Stepping back, what we’ve done at Graft is apply traditional software engineering practices—e.g., layers of indirection, reusable abstractions, divide and conquer—to give networks of agents properties that the individual agents do not possess. Instead of unleashing a swarm of nondeterministic and unwieldy LLMs on an underspecified natural language query and hoping for the best, DANG places them within a structure we understand and can control. The result is fewer confabulations, lower bills, and answers you can audit. Bigger models and more agents won’t magically deliver the properties you need. Graft does.
.png){kind=icon}
