Preface

Generative AI has become a cornerstone of modern search and conversational tools, yet even the most advanced systems make surprisingly basic errors. This article examines a recent wave of spelling and character-count mistakes in Google’s AI-driven Search overviews to explain why these systems falter on tasks humans find trivial. By exploring how large language models (LLMs) encode language and produce output, we aim to clarify the technical reasons behind persistent spelling oddities, summarize the practical implications, and suggest why these flaws matter even as AI capabilities expand. The goal is not to mock the technology but to offer a balanced, accessible explanation so readers can better understand what to expect from AI-powered tools.

Lazy bag

Google’s AI sometimes produces wrong letter counts and misspellings, a symptom of how LLMs represent text. These models break inputs into tokens, not letters, and they learn statistical patterns over those tokens. As a result, they can be excellent at generating fluent prose or solving complex tasks but still be error-prone at exact character-level questions. The issue is known, nontrivial to fix, and highlights that AI outputs should be verified rather than accepted uncritically.

Main Body

In recent updates to its search product, Google expanded the role of generative AI, adding concise AI overviews designed to summarize and clarify queries. These overviews aim to streamline how users get information, but they have occasionally produced strange results — from incorrect letter counts in simple words to outright misspellings of well-known names. For example, an AI-generated response might claim a word contains an unexpected number of a given letter or display a familiar name with letters rearranged. Such output has prompted public amusement and concern alike.

To understand why these errors occur, it helps to look beneath the surface at how contemporary LLMs process language. Most are built on transformer architectures that convert text into a sequence of tokens. Tokens are the basic units the model operates on; they might be entire words, common subwords, syllables, or even single characters, depending on the tokenization scheme. When a prompt is provided, the model translates each token into a high-dimensional numerical encoding. It then predicts subsequent tokens by modeling statistical relationships among those encodings.

This token-centric approach is powerful for many reasons. It allows models to generalize across contexts, produce fluid, context-aware text, and solve tasks like summarization, translation, or code generation. But it also introduces a blind spot: models do not inherently understand text as a sequence of discrete characters in the way humans do. They do not possess an explicit internal representation that maps directly to each letter of a word. Instead, they rely on learned patterns across tokens.

Counting the number of occurrences of a specific letter or spelling a word exactly requires precise character-level reasoning. Because many tokenizers group common letter sequences into single tokens, the model might not separate every individual character. Even when tokens are small, the learned encodings prioritize contextual meaning and next-token prediction over exact character fidelity. As a result, tasks that demand precise orthography or letter counts can expose weaknesses. Researchers have long joked that asking an LLM how many "r"s are in "strawberry" is a reliable way to find such mistakes — and for good reason.

Experts explain that this behavior is not a simple bug but an outcome of architectural trade-offs. Tokenizers are designed to balance efficiency and expressiveness: a vocabulary of many small tokens increases granularity but raises computational cost and data sparsity; a vocabulary of larger tokens reduces complexity but sacrifices character-level precision. Even if one could design a perfect tokenizer that aligns with human intuitions of what constitutes a "word," models would likely still form internal chunks for statistical convenience. That fuzziness makes it unlikely that a perfect, universal solution for spelling and character-count accuracy will emerge from tokenization alone.

Google and other organizations are aware of these limitations and continuously iterate on model design and safety layers. In some cases, companies patch specific problematic behaviors — for instance, correcting a response that mistakenly returned a canned assistant reply in place of a dictionary definition. But many of the spelling-related issues remain resilient because they stem from fundamental aspects of model architecture and training objectives, which emphasize predicting the most probable next token rather than adhering to deterministic character rules.

Importantly, this limitation does not negate the enormous utility of LLMs. These systems can write coherent essays, generate functional code snippets, and help researchers explore complex problems. Their value frequently lies in pattern recognition, synthesis, and creative generation rather than in rote mechanical precision. Nonetheless, the conspicuous errors are useful reminders: AI systems are fallible. They can produce plausible-sounding but incorrect outputs, and without careful scrutiny, users may be misled.

From a practical perspective, users and product designers should treat AI outputs as helpful but not infallible. Verification strategies — cross-checking facts, using character-level checks for spelling-sensitive tasks, or feeding tasks that require exactness into specialized tools — reduce the risk of accepting incorrect information. For developers, potential mitigations include hybrid systems that combine token-based LLMs with deterministic character-level modules for tasks like spelling, counting, or format validation. Another approach is fine-tuning or prompting techniques that nudge models toward more careful, stepwise reasoning, though these methods are not guaranteed to eliminate all errors.

In short, the spelling anomalies observed in Google’s AI overviews illustrate a broader truth about modern AI: high-level competence in many domains does not guarantee flawless performance at low-level, discrete tasks. Recognizing the distinction between statistical language ability and exact character manipulation helps set realistic expectations and guides better use of AI in everyday contexts. As research progresses, some of these gaps may narrow, but for now, the safest course is to appreciate AI’s strengths while remaining vigilant about its weaknesses.

Key Insights Table