Training AI Is More Than Feeding It the Internet
Featuring
In this live stream, Kewin was joined by Piotr Miłkowski, Senior AI Systems Engineer at Callstack, to unpack what it means to train an AI model. The conversation moved beyond surface-level AI usage and focused on the layers that make a model useful: pre-training, instruction tuning, alignment, guardrails, and inference infrastructure.
A freshly trained model can contain useful patterns from its dataset, but it is not automatically ready for conversation. It needs instruction-based training to understand interaction formats, alignment to guide acceptable behavior, and guardrails around input and output to reduce unsafe or unwanted responses.
Better Data Beats More Data
One of the strongest points from the discussion was that dataset quality matters more than raw size. Piotr explained that a useful dataset needs proper distribution, domain coverage, and enough variation for the model to learn relevant patterns. A smaller, curated dataset can outperform a much larger noisy one when it better represents the task the model is expected to handle.
The stream compared different data sources, including Wikipedia, Reddit, and TikTok-style comments, to show how training data affects a model’s output. A model trained on encyclopedic content will respond differently from one trained on short social comments, not only in accuracy but also in tone, structure, and response length.
Tokenization Shapes Model Behavior
Kewin and Piotr also explored tokenization, one of the less visible but essential parts of how language models process text. Models do not read text letter by letter. They process tokens, which may represent words, word fragments, symbols, formatting markers, or special control tokens.
The discussion covered chat templates, tool-call tokens, thinking tokens, and how inference servers transform API-style messages into a single model-readable input. This helped explain why small implementation details can affect how a model interprets a prompt, structures an answer, or handles tool use.
Training Means Adjusting Connections
The stream included a practical explanation of what model training changes under the hood. Piotr described training as the process of adjusting weights between neurons based on examples, errors, and repeated learning passes. Using TensorFlow Playground as a visual reference, the discussion showed how features, layers, learning rates, and neuron connections influence what a model can detect.
This section made the training process easier to reason about: a model is not storing a perfect copy of every example. It is learning statistical relationships that help it respond to new inputs. When the training data lacks coverage, the model is more likely to fail, hallucinate, or respond outside its domain.
Models Infer, They Do Not Understand Like Humans
The stream closed with a nuanced distinction between memorization, understanding, and inference. Current AI models can memorize some structures, detect patterns, and generate responses based on learned correlations. They can also be trained to “think” in a specific format by showing them reasoning traces during training.
But this is not the same as human understanding. The model does not know it is right or wrong in a human sense. It generates output by navigating patterns learned from data. That distinction is important for anyone building AI-powered products, especially when reliability, domain fit, and safety matter.
Watch the full recording to learn how AI models are trained, why dataset quality matters, how tokenization affects model behavior, and what it really means when a model performs inference.
