Choosing an LLM for your task

Kinds of models and the main families

Models differ first by what they produce. Chat and text models write prose and code, embedding models turn text into lists of numbers for search, image models draw pictures from descriptions, and speech models convert between audio and text. Many current chat models are multimodal and also accept images or audio as input.

Among text models you will meet a handful of families: GPT from OpenAI, Claude from Anthropic, Gemini from Google, Llama from Meta, plus Mistral, Qwen and DeepSeek. Each family comes in several sizes, from large flagship models to small fast ones, and some offer reasoning variants that think through a problem before answering.

Under the hood most generative text models are decoder-only transformers, built to continue a sequence. Encoder-style models such as BERT read text rather than write it and are still common for classification and embeddings. You rarely choose by architecture, but the distinction explains why an embedding model cannot chat.

You will also hear the term foundation model. It describes a large model trained on broad data that serves as a base for many tasks, either used as it is or adapted with further training.

Open weights or a closed model

A closed model keeps its weights with the provider; you reach it only through an API. You get a managed service with safety systems, scaling and support, but your data travels to the provider, you accept its terms, and it can change or retire the model on its own schedule.

An open-weight model can be downloaded and run on your own hardware or by a host you choose. That gives you control over privacy, offline use and versions. It also hands you the work: hardware, updates, monitoring and safety filtering become your job.

Open weights are not the same as open source, and they are not automatically free for every use. Licences range from permissive ones to licences with restrictions on use or scale, so read the licence before you build a product on a model.

There is also a middle path. Many hosting companies and routers serve open-weight models through the same kind of API as closed ones, so you can start with a hosted copy and move to your own server later without rewriting the app.

Size, cost and latency

Bigger models usually handle hard reasoning, long instructions and subtle writing better. They are also slower and cost more per token. Smaller models are fast and cheap and often good enough for classification, extraction, short rewrites and simple questions.

Cost is the sum of input and output tokens, each multiplied by the provider's rate, and output tokens are often the more expensive kind. Long system prompts and chat histories add input tokens to every request, so a cheap model with a bloated prompt can cost more than you expect.

Latency has two parts that users feel differently: the time until the first token appears and the time until the answer is complete. Streaming hides some of the wait. Check rate limits and the context window size as well, because a model that fits your quality bar but refuses your traffic is no use.

A hosted service is more than the model inside it. It bundles the model with content filters, logging, quotas and uptime guarantees, and two services running the same open-weight model can still differ in speed, limits and supported features.

• Hard, rare, high-stakes tasks: start with a large model.
• Frequent, simple, well-defined tasks: try a small model first.
• Private data or offline use: consider an open-weight model you host.

Improve before you switch

Many failures blamed on the model are really prompt failures. Before you change model, give clear instructions, the context the model cannot know and an example of the output you want. That is the cheapest fix and the one to try first.

When the model needs facts it was never trained on, such as your notes or yesterday's news, retrieve the relevant text and add it to the prompt. This pattern is called retrieval-augmented generation, and it has its own lesson later in the course.

Fine-tuning trains an existing model further on your own examples. It suits a stable format, tone or narrow task with plenty of good examples, not knowledge that changes often. Training a model from scratch is rarely sensible outside large research teams.

A useful order of escalation is prompt, then retrieval, then fine-tuning, then a different model. Each step costs more effort than the one before, so move on only when your test set shows the current step has stopped helping.

Compare on your own test set

Public leaderboards measure general skills on public tasks. Your task is narrower and your users phrase things their own way, so ten real examples of your own predict your results better than any ranking.

Build the test set before you look at any output. For each input, write one line describing what a good answer must contain. Run every candidate with the same prompts and settings, record time and tokens, and score each answer as correct, partly correct or wrong.

Score without knowing which model wrote which answer, for example by shuffling the outputs. Then pick the cheapest, fastest model that clears your bar, and keep the test set: you will rerun it whenever a provider updates a model or you change a prompt.

Keep the scoring simple enough that you will actually do it. Three marks and a short note on each failure are plenty for a personal project; you can add automated checks once you know which failures matter most.

Write down the model name and the date of the run next to the scores. Hosted models are updated over time, and without a record you cannot tell whether a later drop in quality came from the model or from your own changes.