What generative AI is and how LLMs work

From hand-written rules to learned patterns

Generative AI means models that produce new content, such as text, code, images or audio, in answer to an input. Large language models, or LLMs, are the text branch, and they are the engine behind chat assistants and coding tools.

Early chatbots were scripts. A programmer wrote rules that matched keywords and returned canned replies, which broke as soon as a user phrased something unexpectedly. Machine learning replaced the rules with patterns learned from examples, and neural networks made those patterns far richer.

The current generation rests on the transformer, a network design built around attention. Attention lets the model weigh every earlier part of the input when deciding what comes next, which is why it can keep track of a long paragraph instead of only the last few words.

An LLM is first trained on a very large body of text to predict missing or next words. That base model is then trained further on instruction-and-answer examples and on human preference ratings, which turns a text continuer into an assistant that follows requests.

None of this involves understanding in the human sense. The model has learned statistical regularities of language so well that its output often looks like reasoning, and it is useful for exactly that reason, but it has no goals, beliefs or senses of its own.

Tokens: how the model sees text

Models do not read letters or whole words. A tokenizer cuts the input into tokens, which are frequent chunks of characters, and maps each one to a number. A common short word is often a single token, while a rare or long word is split into several pieces.

Spaces, punctuation and capital letters all affect how text is split, and every model family uses its own tokenizer. Many languages other than English need more tokens for the same meaning, so the same message can be cheaper in one language than in another.

Tokens matter to you for three practical reasons. Providers bill per token, limits are measured in tokens, and the time a reply takes grows with the number of tokens the model must read and write.

You do not need to count tokens by hand. The API reports them in the usage field of every response, and most providers publish a tokenizer tool where you can paste text and see how it is split.

• Cost: input tokens plus output tokens, each at the provider's rate.
• Limits: the context window and the maximum reply length are token counts.
• Quirks: tasks such as counting letters in a word are hard because the model never sees single letters.

Next-token prediction

Given the tokens so far, the model computes a probability for every token in its vocabulary. One token is chosen, appended to the input, and the whole step repeats. The answer you read is built this way, one token at a time, until the model emits a stop token or reaches a length limit.

The choice is not always the single most likely token. A setting called temperature reshapes the probabilities: low values favour the likeliest tokens, higher values let less likely ones through. That is why the same prompt can produce different wording on each run.

Because each token depends on everything before it, an early wrong turn tends to carry through the rest of the reply. Some newer models spend extra tokens working through a problem before giving their final answer, which helps on multi-step tasks at the price of time and cost.

Chat requests add one more layer. Your messages are wrapped with role labels, such as system, user and assistant, and the model has been trained to continue that structured transcript with the next assistant turn. The roles are still just tokens in the input, which is why the model can be confused when untrusted text pretends to be an instruction.

The context window

The context window is the maximum number of tokens a model can consider in one request, counting both what you send and what it writes back. Everything the model knows about your conversation has to fit inside that window.

The model itself keeps no memory between requests. A chat app appears to remember because it sends the earlier messages again with every new one. When the history grows too long, the app must drop or summarise old parts, or the provider rejects the request.

A bigger window is not free. Long inputs cost more and slow the reply, and a detail buried in a very long input can get less attention than one placed near the start or the end. Sending only what the task needs is usually the better design.

Why models make things up

An LLM is trained to produce text that looks like a plausible continuation, not to check whether it is true. When it lacks the facts, it still produces fluent text, and that text can be wrong in a convincing way. This is called hallucination or fabrication.

Several things make it more likely. The model's knowledge stops at its training cutoff, rare facts are weakly represented, it has no built-in way to look anything up, and a question that assumes something false invites an answer that plays along.

You cannot switch hallucination off, but you can make it less frequent and easier to catch. Give the model the source text it should use, ask it to say when it is not sure, prefer low temperature for factual tasks, and check important claims yourself. Later lessons on retrieval and evaluation turn these habits into code.

Hallucination is not the only quality problem. Models can also be outdated, biased by patterns in their training text, or simply bad at exact arithmetic, so treat every answer as a draft from a fast but unreliable assistant and keep your own judgement in charge.