AI Engineer Agentic Track: The Complete Agent & MCP

Day 3 - Call 8 LLMs with the OpenAI-Compatible API: GPT, Claude, Gemini

Beginner 20 minutes
Rupinder Singh
7 views
How the OpenAI-compatible endpoint lets one client shape call OpenAI, Anthropic, Google, DeepSeek, Groq, OpenRouter and even local Ollama models — and what six different LLMs said about virtues that become vices when over-optimized.

Week 1 · Day 3 · Lecture 14. Lab file: 1_foundations/2_lab2.ipynb. With a question generated in the previous lecture — "What is one human virtue that can become a vice when optimized too aggressively, and why? Answer in no more than two sentences" — it's time to put it to eight different LLMs and compare the answers.

The trick: one API shape, many providers

Each provider has its own proprietary API, but nearly all of them — OpenAI, Anthropic (the last holdout, but they eventually added it), Google, DeepSeek, and even Ollama running locally on your own machine — also expose an OpenAI-compatible endpoint. That means you can talk to any of them using the exact same request format OpenAI uses, just by changing two things: the base_url and the api_key.

In practice this means you create a client using the OpenAI Python client library for every provider, not just OpenAI itself — you just override where it points:

anthropic = OpenAI(base_url=ANTHROPIC_BASE_URL, api_key=anthropic_api_key)

Confusing at first glance (a variable called anthropic that's really an instance of the OpenAI class) but powerful: the exact same .chat.completions.create(...) call now talks to Claude instead of GPT. Switch the base URL to localhost and the same code calls a local Ollama model instead. One shape, many destinations. (Guide 9 in the guides folder walks through this in detail.)

Setting up the run

  • Base URLs for each provider are set once as constants, so the client-creation cells stay short and readable.
  • Two lists track the run: competitors (which model answered) and answers (what it said) — kept in the same order so they line up.
  • A small helper function wraps the repeated logic: append the competitor name, append the answer, print/display it — so each provider call is just a couple of lines.
  • Small, fast models are used deliberately here rather than each provider's strongest flagship — the point of this exercise is exploring different agentic flows and providers, not benchmarking peak intelligence (that's covered elsewhere).
  • Where supported, a reasoning_effort parameter controls how much a model "thinks" before answering — set to none here, though low/medium/high/extra high are also options.

Six calls, six very different answers

All eight calls follow the identical pattern — client.chat.completions.create(model=..., messages=...), then read response.choices[0].message.content. Here's how the walkthrough plays out (your exact models and answers will differ — that's expected, since Day 3's question was itself LLM-generated and providers update their models constantly):

Provider / modelAnswer (paraphrased)
OpenAI — GPT nanoDiscipline, over-optimized, becomes rigidity or obsessive control — prioritizing short-term efficiency over flexibility and compassion.
Anthropic — Claude Sonnet (via OpenAI-compatible endpoint)Conscientiousness taken to an extreme collapses into perfectionism and rigidity, paralyzing decisions and damaging relationships.
Google — Gemini Flash LiteFrugality over-optimized becomes stinginess — accumulation over utility, isolating the individual from the social value moderation protects.
DeepSeek — DeepSeek V4Discipline, over-optimized, becomes rigidity, suppressing creativity and adaptability (a succinct variant of the OpenAI answer).
Groq (inference provider) running an open-source model (GPT-OSS-120B)Humility, over-optimized, becomes self-effacement — suppressing your own voice and necessary self-advocacy.
OpenRouter, routing to Kimi K2.6 (Moonshot AI)Honesty pursued to an extreme becomes cruelty masquerading as integrity — truth without tact erodes the trust it claims to serve.

A few things worth noticing in this run: Groq is an inference provider, not a model — the actual model answering was GPT-OSS-120B (OpenAI's open-source model, released as an answer to DeepSeek's open-source push), just served extremely fast on Groq's hardware. And OpenRouter was used here to reach Kimi K2.6 from Chinese startup Moonshot AI — at the time of recording, considered the strongest openly available model, ahead of the more hyped DeepSeek (check Artificial Analysis for the current leaderboard). Kimi also visibly took much longer to answer (37 seconds of reasoning) — a reminder that "smartest" and "fastest" are frequently in tension.

Notice too how much the different models converged on similar shapes of answer (discipline/conscientiousness → rigidity shows up three separate times) while still phrasing it distinctly — a nice illustration of both the shared "reasonable answer" space and each model's own voice.

What you should now understand

  • Most LLM providers expose an OpenAI-compatible endpoint, so one client shape (the OpenAI Python class) can talk to OpenAI, Anthropic, Google, DeepSeek, Ollama, and more — you only change base_url and api_key.
  • Tracking parallel competitors/answers lists, plus a small helper function, keeps a multi-provider comparison loop readable.
  • Groq and OpenRouter are not model providers — they're an inference provider and a router, respectively, serving other companies' open-source models.
  • reasoning_effort (none/low/medium/high/extra high) controls how much a supporting model "thinks" before responding.
  • Different models frequently converge on the same underlying idea while expressing it in noticeably different voices — worth watching for as you compare answers yourself.

Resource links

Next

With eight answers collected, the next step is judging them against each other — an early taste of the "LLM as a judge" pattern covered later in the week.

Frequently Asked Questions (7)

Why is the client variable called "anthropic" if it's really an instance of the OpenAI class?

It's just a naming convention for clarity in your own code — the variable is named after the provider it talks to, even though under the hood it's an OpenAI() client object with its base_url and api_key overridden to point at Anthropic instead of OpenAI.

Can I call a local Ollama model with the exact same code I use for OpenAI?

Yes — Ollama exposes its own OpenAI-compatible endpoint. Point the same OpenAI() client's base_url at your local Ollama server (typically localhost) instead of a cloud URL, and the rest of your chat.completions.create code is unchanged.

Do all providers support the reasoning_effort parameter?

No — it depends on whether the specific model supports adjustable reasoning. Where it is supported, options typically range from none through low, medium, high, up to extra high; check that provider and model's documentation if a call errors on this parameter.

Is Kimi K2.6 the same as DeepSeek?

No — they're different companies. Kimi K2.6 is from Moonshot AI, a separate Chinese startup, while DeepSeek is from DeepSeek AI. At the time of this lecture, Kimi K2.6 was considered the strongest openly available model, ahead of DeepSeek's models, though rankings change quickly — check Artificial Analysis for the current picture.

Why did one model (Kimi K2.6) take so much longer to respond than the others?

It spent significantly more time "thinking" before answering (37 seconds in the lecture's example) — a reminder that stronger reasoning often trades off against response speed. This is part of why the lab deliberately uses small, fast models for most of the comparison rather than each provider's top reasoning model.

What happens if a provider's API call fails partway through my loop?

The lecture doesn't show explicit error handling, but in your own version it's worth wrapping each call in a try/except so one failing provider (bad key, rate limit, wrong model name) doesn't stop the rest of your comparison from running.

Do I need to use the exact same 8 providers/models shown in the lecture?

No — pick whichever providers you have keys for and want to try. The point is understanding the OpenAI-compatible pattern and building a comparison loop, not matching the instructor's exact roster, which will already be somewhat out of date by the time you watch this.