What Even is an Agent?

There are various definitions, but what I found useful is “An agent is an LLM-based system that has varying degrees of agency to solve a particular human instruction”. At the lowest level of agency, is a pre-planned set of steps that might be encountered in the scope of an agent’s execution. For example, if an HTTP 404 is encountered on an agent that scrapes a webpage, then the human programmer could have coded some steps to recover and crawl another part of the website. At the highest, the steps are entirely left to the agent, which means that the AI programmer would write its own recovery code to crawl another part of the website. Or probably leave it altogether. Of course, this entails risk on the part of any business that gives a high level of agency to an AI. More on that later.

At its most basic form, an agent can think, act, and observe the results of its actions. To think is to map out the required steps for an action. To act is to generate text, call pre-defined scripts, or even generate its own script. Lastly, to observe is to analyze the results of the previous action. judging whether the initial instructions are fulfilled, and calling another pass if more steps are needed. Agents also have “tools”, which are used to perform a highly specific task, and can be chained together with other tools and/or workflows.

Smolagents is a library from the Huggingface team that uses CodeAgent, a high-level abstraction that uses “Code as Action”. This means that the agent is empowered to not only call the registered tools but also create code that utilizes them, giving it much more freedom to use and reflect on the provided tools.

See this helpful image to illustrate its effectiveness. It is an example of an agent with high agency, since it can act on its own by creating its own code. For more, consult this paper, Executable Code Actions Elicit Better LLM Agents.

For the rest of this blog, I will present how my own agent creates rumors in the tabletop role-playing game “Blades in the Dark”. These rumors could be the basis of adventures in the game. It’s a toy example, but should provide the overview of how this could work in your domain. Real-world examples could be a web crawler that adjusts its crawling behavior, an agent that creates an earnings report, or an programmer copilot that creates an interactive lecture coded in JS.

You can interact with this agent in Huggingface Spaces. Simply type out what you need and add some additional context. The app uses Qwen 32B Coder Instruct from an Huggingface managed endpoint. The agent has several tools at its disposal, general web search, a retrieval tool that reads from a single document file describing the setting of the game, and python functions that serve as “oracles”. Each of these oracles simulate dice rolls and possibilities in the scene.

Note that it uses some free endpoints so at peak times, you might have trouble getting in.

We start with requesting a rumor from the agent. View below how the agent thinks, creates and executes the code, and “roll” with the results.

The Rumor in a Tavern

This situation is a straightforward one. The agent has some tools, python functions that simulate die rolls that correspond to specific Doskvol rumors.

Me: give my smuggler crew a rumor. They are in a tavern looking for leads

Step 1

[THOUGHT]:
To generate a rumor for the smuggler crew, I will start by rolling a 3d6 to generate a random street rumor.
[CODE]:

import random
# Simulate rolling a 3d6
die_roll = ''.join(str(random.randint(1, 6)) for _ in range(3))
print(f"Die roll result: {die_roll}")

> Die roll result: 163

Step 2

Thought: Now that we have the die roll result (163), we will use the get_street_rumor tool to generate a street rumor based on this result.

Code:

rumor = get_street_rumor(die_roll="163")
print(f"Generated rumor: {rumor}")

> Generated rumor: Street rumor: "Brightstone heir + Forbidden love + Rival crew"

Step 3

Thought: Now that we have the generated rumor, “Brightstone heir + Forbidden love + Rival crew”, we will evaluate the likelihood of this rumor being favorable for the smuggler crew’s mission using the get_oracle_result tool.

Code:

likelihood_of_favorable_outcome = "medium"
oracle_result = get_oracle_result(likelihood_of_favorable_outcome=likelihood_of_favorable_outcome)
print(f"Oracle result: {oracle_result}")

> Yes, but it's a partial success. You did it, but there are some minor negative consequences

Step 4

Thought: The oracle result indicates that the rumor is partially favorable for the smuggler crew’s mission. We can provide the rumor to the smuggler crew as it might still be useful despite some minor negative consequences.

Final answer:
Rumor: There’s a Brightstone heir involved in a forbidden love affair with someone from a rival crew. This could lead to valuable information or opportunities.

The Forbidden Love Affair

In this one, I’ve prepared a retriever tool that informs the main agent some facts about the city. It is a simple BM25 retrieval tool. No chunking, indexing, and offline pipelines involved.

Me: Can you decribe Brightstone? And make some artistic liberties with this forbidden love affair.

Step 1

[THOUGHT]:
To provide a detailed description of Brightstone and a forbidden love affair with artistic liberties, I will first request the retrieval_agent to provide information about Brightstone from the Blades in the Dark setting.
[CODE]:

task = "Retrieve detailed information about Brightstone from the Blades in the Dark setting, including its key characteristics, atmosphere, and notable locations."
retrieval_agent(task=task)

> 
Here is the final answer from your managed agent 'retrieval_agent':

1. Task outcome (short version):
Brightstone is a wealthy and influential city in Doskvol, known for its broad streets, clean canals, and luxurious architecture. It is home to many of the city's elite and features key locations such as the Bowmore Bridge, Barrowcleft Market, and Charterhall.

2. Task outcome (extremely detailed version):
Overview:
Brightstone is one of the wealthiest and most influential districts of Doskvol... [truncated]

Atmosphere:
The atmosphere of Brightstone is one of opulence and luxury... [truncated]

Key Characteristics:
Bowmore Bridge: A massive structure of bright white stone and shimmering metal-work, considered the largest bridge in the Imperium... [truncated]

Step 2

Thought: Now that we have detailed information about Brightstone, we can proceed to create a forbidden love affair narrative that fits the setting. Next, I will request the retrieval_agent to provide examples of forbidden love affairs in fiction and literature to understand artistic liberties that can be taken.

Code:

task = "Provide examples of forbidden love affairs in fiction and literature, focusing on elements that can be creatively adapted to fit a setting like Brightstone."
retrieval_agent(task=task)

>
Here is the final answer from your managed agent 'retrieval_agent':

1. Task outcome (short version):
Examples of forbidden love affairs in fiction and literature have been gathered, and elements that can be adapted to fit a setting like Brightstone have been identified.

2. Task outcome (extremely detailed version):
Examples of forbidden love affairs in fiction and literature have been gathered from various sources, including historical love affairs, fantasy stories, dystopian narratives, and contemporary romances. The examples include love between a prince and a slave girl, a queen and a Roman general, a Shadowhunter and a Downworlder, a human and a supernatural being... [truncated]

Step 3

Thought: With the examples of forbidden love affairs and the elements that can be adapted, I will now craft a narrative that fits the setting of Brightstone. I will incorporate elements such as clashing worlds, supernatural elements, moral dilemmas, and societal hierarchies to create an engaging and thought-provoking story.

Code:

# Crafting the narrative
brightstone_description = """
Brightstone, a district of Doskvol, is a city of opulence and luxury, known for its broad streets, clean canals, and luxurious architecture. Its streets are illuminated by bright electric lights, and the buildings are constructed from fine, pale marble blocks, rich timbers, and intricate ironworks. The city is home to many of the city's elite and features key locations such as the Bowmore Bridge, Barrowcleft Market, and Charterhall.
"""

forbidden_love_affair = """
In the heart of Brightstone, amidst the opulent palaces and luxurious mansions, there lived two individuals... [truncated]
"""

# Combining the description and the forbidden love affair
final_narrative = brightstone_description + forbidden_love_affair
final_answer(final_narrative)

Final answer:
Brightstone, a district of Doskvol, is a city of opulence and luxury, known for its broad streets, clean canals, and luxurious architecture. Its streets are illuminated by bright electric lights, and the buildings are constructed from fine, pale marble blocks, rich timbers, and intricate ironworks. The city is home to many of the city’s elite and features key locations such as the Bowmore Bridge, Barrowcleft Market, and Charterhall.

In the heart of Brightstone, amidst the opulent palaces and luxurious mansions, there lived two individuals whose love was forbidden by the very society they inhabited. On one side was Elara, a brilliant and ambitious young engineer from the lower echelons of society. Known for her innovative ideas and relentless drive, Elara worked tirelessly in the factories of Lockport, dreaming of a better future for herself and her people.

On the other side was Lucius, a high-ranking member of the elite Bluecoat patrols who patrolled the streets of Brightstone. His life was one of privilege and power, but he felt a sense of emptiness that he could not explain. It was during one of his patrols that he first laid eyes on Elara, who was working late in one of the factories.

…

The Crew Needs Clothes

My smuggler crew is helping Elara and Lucius, an engaged couple. Elara is an ambitious young engineer while Lucius is a high-ranking member of the blucoats. Lucius wants the party’s help to break Elara out of jail because of made-up charges of sabotage. The crew needs to secure some prison warden clothes, and decide to go to an underground clothier that specializes in disguises.

Step 1

[THOUGHT]:
To proceed with the task, I will first roll a 3d6 to simulate the attempt to sway the clothier for discounted clothes. This will help us determine the likelihood of a favorable outcome.
[CODE]:

import random

# Simulate rolling a 3d6
die_roll = ''.join(str(random.randint(1, 6)) for _ in range(3))
print(f"Die roll result: {die_roll}")
> 352

Step 2

Thought: Now that we have the die roll result (352), we will use the get_oracle_result tool to evaluate the likelihood of a favorable outcome when attempting to sway the clothier for discounted clothes.

Code:

likelihood_of_favorable_outcome = "medium"
oracle_result = get_oracle_result(likelihood_of_favorable_outcome=likelihood_of_favorable_outcome)
print(f"Oracle result: {oracle_result}")
> No, it failed. There are negative consequences.

Step 3

Thought: The oracle result indicates that the attempt to sway the clothier for discounted clothes will fail, and there will be negative consequences. Given this information, we need to consider alternative methods to obtain the necessary attire.

Since our previous attempt to sway the clothier failed, we will roll for a random encounter using the get_random_encounter tool to find potential challenges or opportunities that may arise during the process of obtaining the clothes.

Code:

import random

# Simulate rolling a 3d6
die_roll = ''.join(str(random.randint(1, 6)) for _ in range(3))
print(f"Die roll result for random encounter: {die_roll}")
> 511

Step 4

Thought: Now that we have the die roll result for the random encounter (511), we will use the get_random_encounter tool to generate the encounter and determine if it can provide an opportunity to obtain the necessary attire.

Code:

encounter = get_random_encounter(die_roll="511")
print(f"Random encounter: {encounter}")
> Your word prompt is "cutthroat"

Step 5

Thought: The random encounter generated by the get_random_encounter tool is “Cutthroat.” This suggests that a cutthroat, a professional assassin or a ruthless individual, might be involved in some way. We can use this encounter to our advantage by approaching the cutthroat for help in obtaining the necessary attire.

Given that the cutthroat is likely to be well-connected and possibly have access to various resources, we can attempt to negotiate with them for the clothes. Here’s how the interaction might proceed:

1. Approach the Cutthroat: Find a discreet location to meet the cutthroat.
2. Negotiate: Explain the situation and offer a deal. Since the cutthroat is likely to be interested in money, we can offer a substantial sum to secure the clothes.
3. Obtain the Clothes: If the negotiation is successful, the cutthroat will provide the necessary attire.

Let’s proceed with this plan.

Code:

# code truncated

> Your crew approaches a known cutthroat in a shadowy alley of Nightmarket. The cutthroat, a lean and dangerous figure with a scarred face, looks up from his ledger when you enter.

Cutthroat: What do you want? I don't deal with nobles or their parties. Why should I help you?

Smuggler: We need some warden disguises. We heard you might have what we need. We can pay well.

Cutthroat: (smirks) Pay well, huh? I might be interested. What do you have?

Smuggler: (showing a bag of gold) Here. We can pay in advance.

Cutthroat: (eyeing the gold) Not bad. I know a place where you can get the clothes. Follow me.

The cutthroat leads you to a hidden room deep in Nightmarket, where you find a selection of fine clothes. You pay the cutthroat and take the clothes, ready to break in.

Step 6

[THOUGHT]:
To proceed with the task, I will first request the retrieval_agent to gather detailed information about the prison
…

In the last case, the agent ran out of max turns. That is an important limit so tokens do not run out of hand. We can amend that with either more turns or a failover system prompt that simply summarizes everything and uses that.

Taking it into production

How do we proceed from here? As with all things machine learning, there needs to be evaluation, observability, safety, and other implementation concerns that need to be established.

Evaluation sets are ground truth data where the inputs are mapped to expected outputs. Quantifiable metrics should be collected ranging from automated ones like BLEU, ROUGE, or semantic similarity. LLM-judges or human judges can also be utilized, where a closer annotation is given to each input-prediction pair based on the ground truth evidence. Automated evaluation is well suited for rapid iterations, while the latter evaluation would take more time and resources, but would yield deeper and more valuable insights.

Observability is critical for these systems in production since agents with high agency will consume a lot of resources (think API and rate limits), and may do unexpected things in production.

To that end, Safety Guardrails are crucial to prevent the unexpected (and grave!) from happening in the first place. Guardrails can be in the form of topic blacklists to code guards inside the tools. Prompt engineering, or a static list of keywords could be utilized also. You do NOT want to have “drop table” called in an SQL agent, for example.

Lastly, rate limiting and provisioning is important to set bounds on how much the agents are utilizing. Planning and generating code consumes a lot of tokens. Web search and other managed endpoints would also need close management.

That’s it. Hopefully you got something from this. LLMs are so much more than one-shot prompting. With agents, developers can be as creative as they want, but with great agency comes great responsibility.