AI Engineer Agentic Track: The Complete Agent & MCP

Day 4 - How Tool Calling Works: The Truth Behind AI Agent Autonomy

Beginner 15
Rupinder Singh
7 views
Explains what tools really are: function calling is your code acting on tokens the LLM generates, shown with an airline price prompt, a move-north prompt, and the true shape of the agent loop.

Week 1 · Day 4 · Lecture 18. This is the point where the course explains what tools actually are, the essential idea behind agentic AI. The short version: an LLM never runs anything. It generates tokens, and the code you write reads them and decides what to do.

What "tools" are

LLMs are models that take an input sequence of tokens and generate the most likely tokens to follow it. (If you are unsure how that works, the instructor's YouTube series on LLMs is linked in the course resources.) Tools give an LLM new abilities beyond generating text, such as querying a database, running some SQL, looking up a stock price, or editing a Google Sheet as we did in n8n.

Tools give LLMs abilities

Tool calling is also called function calling. You write a function (a bit of Python), tell OpenAI "as part of answering this question, you may also run this function on my computer", and it sounds spooky: is OpenAI going to reach into my computer, and how do matrix calculations that generate text suddenly decide to run code? The reality is much more mundane. A lot of agentic AI comes down to clever prompting, JSON, and plain if statements.

Tool calling: in theory versus in practice

In theory, it feels magical: the user talks to software, the software calls an LLM in the cloud, and the LLM itself reaches out to tools like searching the internet, querying a database, editing a Google Sheet or looking up a stock price.

Tool calling in theory

In practice, the tools hang off your software, not the LLM:

  1. The user asks a question and your software calls the LLM with it.
  2. The LLM only generates tokens. Those tokens can be a reply to the user, or a request such as "don't reply yet, I want to run a SQL query, please use the SQL tool".
  3. Your software makes the SQL query.
  4. Your software calls the LLM again: the user said this, we used a tool, here are the results, now what?
  5. The LLM continues generating tokens.
Tool calling in practice

The thing that calls the tool is your code, not the LLM.

Demo 1: a tool call that is only a prompt

To show that tool use simply comes down to prompting, the instructor pastes a vanilla request into ChatGPT, with no real tool behind it. The prompt says the model is a customer support assistant for an airline, that it answers users' questions, and that it has a special ability to look up ticket prices: respond like "GET_TICKET_PRICE: London" to get the price of a ticket to London. The user question is "How much does it cost to go to Paris?" The model replies GET_TICKET_PRICE: Paris.

ChatGPT replying GET_TICKET_PRICE: Paris

That was a tool defined and chosen through nothing but a prompt and a response. A student once asked: if the user wants the price to Paris, why not just call the function directly, why involve an LLM at all? Because the user never typed "GET_TICKET_PRICE: Paris". They said "how much does it cost to go to Paris?" The LLM understands that cost and price mean the same thing, that Paris differs from London, and how to craft the exact call. Turning free-form natural language into specific discrete actions was close to an unsolved problem a few years ago; now LLMs do it in a heartbeat.

Demo 2: autonomy is prompting plus interpreting output

Second prompt: you are exploring a room, you can move in four directions by responding MOVE NORTH, MOVE EAST, MOVE SOUTH or MOVE WEST to learn more about what's there; respond with your choice. ChatGPT answers MOVE NORTH. Asked again, it picks east.

ChatGPT replying MOVE NORTH

This is another tool-call example, but the bigger point is that it is a real example of what gets packaged up as autonomy. The model chose to move north, and nothing told it to. Attach some code that reads that reply to a little robot and you would say ChatGPT autonomously moved one square. Sounds fancy; it is just how you prompt and how you interpret the output. Tokens in, tokens out.

The agent loop, the same trick

It feels as if an agent loop is software calling an LLM in a loop, with the LLM somehow using tools:

The agent loop feels like this: tools attached to the LLM

Actually, it is this: your software repeatedly calls the LLM, and based on the response may call some functions, then keeps calling the LLM. Your software owns the loop and calls the tools. The LLM just takes input tokens and gives output tokens.

The agent loop is actually this: tools attached to the software

What you should now understand

  • Tool calling (function calling) lets an LLM "use" a function, but it is your software that executes it.
  • The LLM signals a tool call through the tokens it generates, guided by your prompt, usually in a parseable format such as JSON.
  • The LLM's real value is understanding the meaning and intent of free-form text and mapping it to the right discrete action.
  • "Autonomy" is prompting plus code that interprets the output, with if statements around it.
  • An agent loop is code that repeatedly calls an LLM and the tools it asks for; the loop and the tools belong to your software.

Resource links

Next

On to the Day 4 lab (lab 3), where you will build tools and an agent loop yourself, starting with the Digital Twin project: reading a PDF and mastering system prompts.

Frequently Asked Questions (7)

What is tool calling (function calling)?

Giving an LLM new abilities, such as querying a database or looking up a stock price. You offer a function, and the LLM signals through the text it generates when it wants that function used.

Does the LLM really run code on my computer?

No. The LLM only generates tokens. Your own software reads those tokens, runs the real function, and then calls the LLM again with the result.

If it is just a prompt, why do we need an LLM at all?

Because users do not type exact calls. The LLM understands that how much does it cost to go to Paris means the same as asking for a ticket price, and turns that free-form language into the right discrete action.

What do the airline and room-exploring prompts show?

The airline prompt shows a tool being defined and chosen purely through a prompt and a response: GET_TICKET_PRICE: Paris. The room prompt shows apparent autonomy: the model picks MOVE NORTH, then something different next time, and code could act on that.

What does autonomy really mean here?

It is a fancy name for how you prompt the model and how you interpret its output, with some if statements around it. Tokens in, tokens out.

How does the agent loop really work?

Your software calls the LLM in a loop. Based on each response it may call functions, then keeps calling the LLM. The software owns the loop and the tools; the LLM only takes input tokens and gives output tokens.

Where do I learn how LLMs work if I am not sure?

The instructor has a YouTube video series on how LLMs work, linked in the course resources. In short, an LLM generates the most likely tokens to follow an input sequence of tokens.