How to Build Differentiated Agentic AI Products: A Practical Guide

Not every problem needs an agentic AI solution. But when it does, how do you ensure you’re solving the right problem with the appropriate level of complexity? In this post, I will share lessons and practical tips that I’ve gathered from my experience with Georgian’s AI Lab working alongside and advising Georgian portfolio companies on agentic AI implementations. 

The promise of agentic AI appears to be immense, but getting from early pilots to production-ready systems is complex. According to a March 2025 survey from Georgian and NewtonX, 91% of surveyed R&D leaders reported that their company is piloting or using agentic AI. This level of adoption may be unsustainable, however. For example, Gartner recently predicted that “over 40% of agentic AI projects will be canceled by 2027, due to escalating costs, unclear business value, or inadequate risk controls”. 

This post is for builders who want to turn their early agentic AI pilots into production-ready products. Before we go any deeper, I’d like to clarify some language I will use in this post.

What is agentic AI? 

Agentic AI refers to  software designed to (a) interpret a task and any available context (data), (b) independently plan and reason about how to complete that task (and any identified sub-tasks) and then (c) complete the task with minimal human oversight. It is helpful to think about agentic AI as a design pattern—systems that reason, decide, and act with different levels of autonomy. What matters is how agentic a system is.

What is the difference between agentic workflows and autonomous agents? 

Both agentic workflows and autonomous agents are types of agentic systems but differ in how much control is given to the software by builders.

• Agentic workflows follow a set of predefined steps. Agentic workflows are more structured, easier to test, and usually more reliable than autonomous agents.
• Autonomous agents are more freeform than workflows—autonomous agents  can make decisions, react to new information, and take independent actions. However, autonomous agents are more challenging to control and debug.

Georgian’s Agentic Platforms and Applications Whitepaper illustrates the levels of agency that agentic systems can have.

An important consideration when building agentic systems is determining the desired level of autonomy for the system. The more autonomy you give an agentic system, the more flexible and powerful it can be. However, higher levels of autonomy make the system harder to predict, control, and debug. Consider starting small by beginning with workflows that have lower autonomy to build confidence and reliability into the system. You can incrementally add more autonomy once the system is able to demonstrate its effectiveness.

I’m considering implementing agentic AI—where should I start?

 

Start with the problem, not the technology

While it may be tempting to begin experimenting with Large Language Models (LLMs) or advanced tooling, consider starting by identifying the problem you are trying to solve. Once the problem is identified, consider tying it to expected business value such as revenue, cost savings, or other measurable key performance indicators (KPIs). 

Use cases tied to business value are more likely to yield strong results than those that don’t provide a business value. Georgian and NewtonX’s research shows that while 91% of surveyed R&D leaders are trying out agentic AI, only 26% expect cost savings and 16% expect new revenue from their agentic implementations. Companies at the highest level of AI adoption in Georgian’s research were more likely to tie their AI efforts directly to ROI.

Ask yourself: Is agentic AI the right approach for the problem I’m trying to solve?

Not every problem needs an agentic solution; to determine if yours does, consider the following questions:

• Is there a clear task to complete, with well-defined steps?
• Is there a clear signal when the task has been successfully completed?
• Will the solution need to make decisions or adapt to changing inputs along the way?
• Is the environment or input data dynamic and constantly changing?

If the answer to these questions is “yes”, then an agentic approach may be appropriate. If not, a simpler automation approach may suffice.

Using agentic AI to build a differentiated solution

Let’s assume that you have defined a problem and tied your use case to business value. Before you start building, consider the gap that exists between the problem you are trying to solve and the technology available to develop your agentic solution. Foundation models (like GPT-4 or Claude) are powerful, but they are general-purpose and will often solve only about 60-70% of your problem.

The remaining 30–40% of the problem  may be  challenging to solve, but can provide significant value. This “last mile” is where you can enhance foundation models with:

• Real-world constraints
• Domain expertise
• User expectations
• Edge cases you can’t ignore

Users are likely to prefer smart, adaptive and personalized products. Simply stringing together an LLM and a tool is unlikely to engage users for long. A well-designed system that effectively addresses this “last mile” may separate a basic demo from a durable, competitive product. 

In addition to meeting user expectations, your solution should consider the rapid pace of model improvement. New models today may be innovative in their capabilities, but those capabilities are likely to become standard in future models.

Differentiated agentic systems generally have some of the following characteristics:

• They are able to decompose complex tasks into smaller, more manageable subtasks
• They are able to reason
• They leverage personalization to provide a user with a solution tailored to the user’s needs
• They offer seamless integration between all parts of the agentic system (including individual agents and relevant data sources)
• They are designed for evolving user needs, not static capabilities

Putting it all together with an example

Consider the example of a health care provider looking to use agentic AI to provide personalized treatment plans for patients. Here is how they might consider implementing agentic AI: 

How can I build agents?

Understand the building blocks of agentic AI

Agentic systems can be composed of the following components:

• LLMs—the brains—handle understanding and generating output
• Retrieval mechanisms pull in relevant context from knowledge bases and external data sources, ensuring up-to-date and accurate information
• Tools allow agents to perform actions such as triggering alerts or scheduling follow-ups
• Memory helps the agent learn from past interactions and improve over time

These components are modular, meaning that you don’t need to incorporate all of them into your agentic system. You can choose the building blocks that best fit your problem and integrate them incrementally to develop your solution.

Build using agentic frameworks

Instead of assembling the building blocks yourself, you can use agentic frameworks—toolkits that include features such as planning, memory, tool use, orchestration, and more. Consider starting with open-source frameworks as they have a lower barrier to entry, and can allow you to explore frameworks more quickly.

Two examples of frameworks are:

• ag2 – optimized for conversation-based agents
• Agno – provides strong planning and tool use abstractions

Each framework has its strengths, so consider the framework that best fits your use case. If you’re not sure how to select the right framework for you, tools like ChatGPT can help you explore options based on your specific use case.

How can I ensure that my agent actually works?

Building the agent is just the beginning. Systemic evaluation is what makes an agent production-ready. Evaluations are difficult, but necessary. 

Start with unit testing

Each part of the system should be tested in isolation so that you can pinpoint issues or failures without getting confused by other parts of the system. When conducting unit tests, evaluate agents using the following questions:

• Is the retrieval stage pulling the correct information from the database or API?
• Is the appropriate history being retained in memory?
• Is the reasoning step making effective decisions?
• Are all parts of the system functioning as expected?

Look beyond your agent’s correctness. Other questions to ask include:

• How efficient is the system?
• How many tool calls does it require?
• Where in the process is the agent getting stuck?

Build reliability in the system through integration testing

Once unit testing is complete, test how all components work together through integration testing. The following questions may be helpful to ask when integration testing:

• If the agent retrieves incomplete data, does it still make effective decisions?
• If memory drops a key detail, does the agent lose track of its task?

Think of agentic systems as chain reactions, where the output from one step feeds into the next. A small error early in the process may lead to bigger problems in the final result, making integration testing essential.

Takeaways

• Start with a problem or use case and tie it to ROI.
• Agentic AI is not for everything. Agentic AI may be a useful solution for problems that are complex and dynamic.
• Autonomy is a spectrum. Build confidence with low-autonomy workflows before scaling to agents.
• Frameworks can accelerate success. Use composable, open-source frameworks that match your needs.
• Differentiate through the last 30%. Focus on domain-specific logic, user expectations and integration.
• Test like a software engineer. Rigorous evaluation—including unit and integration testing—is the foundation for reliable deployment.

About Asna Shafiq

Asna Shafiq is a Machine Learning Engineer in Georgian’s AI Lab, where she works with portfolio companies to accelerate their AI roadmap by leveraging cutting-edge AI. Prior to joining Georgian, she spent 8 years shaping solutions at Loopio and analyzing weather patterns at Environment Canada. Asna has a degree in Computer Engineering from University of Toronto. Off hours, you can find her practicing yoga and eating her way through the world.