Essential AI Terminology

LLM (Large Language Model) — An AI model trained on text data. Examples: GPT-4, Claude, Gemini.

Parameters — The internal numbers the model adjusts during training. More parameters generally means more capability. GPT-4 has over 1 trillion parameters.

Weights — The specific values of parameters after training. When people say "open-weight model," they mean the trained model is publicly available.

Neural Network — The architecture behind deep learning. Layers of interconnected nodes that process information, loosely inspired by the brain.

Transformer — The specific neural network architecture that powers all modern LLMs. Introduced in the 2017 paper "Attention Is All You Need."

Prompt — The input you give to an AI model. Can be a question, instruction, or any text.

Completion — The model's response to your prompt.

Token — The basic unit LLMs process. Roughly 0.75 words.

Context Window — The maximum number of tokens the model can process at once (input + output).

System Prompt — Hidden instructions that set the model's behavior before the user's message. Used by developers to customize AI behavior.

Temperature — A setting controlling response randomness (0 = deterministic, 1+ = creative).

Top-p (Nucleus Sampling) — Another randomness control that limits token choices to the most probable options.

Training Data — The text corpus used to train the model. GPT-4 was trained on a large portion of the internet.

Fine-tuning — Additional training on a specific dataset to specialize a model for particular tasks.

RLHF (Reinforcement Learning from Human Feedback) — A training technique where humans rate model outputs to teach the model to be more helpful and less harmful.

Knowledge Cutoff — The date after which the model has no training data. It won't know about events after this date.

Hallucination — When an AI generates plausible-sounding but incorrect information. A major limitation of all current LLMs.

Grounding — Techniques to reduce hallucination by connecting the model to real data sources (search, databases, documents).

RAG (Retrieval-Augmented Generation) — A technique where the model searches a knowledge base before responding, reducing hallucination.

API (Application Programming Interface) — A way for developers to access AI models programmatically, enabling AI-powered applications.

Agent — An AI system that can take actions autonomously — browsing the web, running code, calling APIs — not just generating text.

MCP (Model Context Protocol) — A standard for connecting AI models to external tools and data sources.

Multimodal — A model that can process multiple types of input (text, images, audio, video).

Embedding — A numerical representation of text that captures its meaning. Used for search, similarity, and classification.