Outcome focus: Explained how algorithm complexity shows up in everyday product work, then walked through an e-commerce recommendation feature from naive loops to indexed lookup.
software engineeringalgorithmssystems designtypescriptperformance
Algorithm complexity is easy to learn badly.
You memorize that binary search is O(log n), hash table lookup is usually O(1), nested loops are often O(n^2), and sorting is commonly O(n log n).
Then you go back to building software and wonder where those facts are supposed to live.
They rarely show up as clean interview problems. They show up as product behavior.
A page loads slowly after a customer imports a larger catalog. A dashboard works in staging and times out in production. A search endpoint is fine with 200 records and miserable with 2 million. A batch job keeps getting moved later in the night because it no longer fits inside the original window. A recommendation feature gives the right answer, but only while the data is small enough for everyone to ignore the cost.
Complexity is not just a math topic.
It is a way to notice when a design is borrowing time from the future.
What Big O is actually buying you#
Big O is a rough language for growth.
It does not tell you the exact runtime of a program. It does not know your CPU, your database, your network, your cache hit rate, or the shape of your users' data. It ignores constant factors on purpose.
That sounds like a weakness until you see the point.
Big O helps you ask what happens when the input stops being cute.
If a function does one pass over 1,000 products, it will probably survive 10,000 products. If it compares every product to every other product, the story changes quickly. 1,000 * 1,000 is one million comparisons. 10,000 * 10,000 is one hundred million. The code may look similar. The growth is not.
That is the useful part.
Complexity gives you an early warning system. It lets you look at a piece of code and say, "This might be fine for an admin tool, but it should not sit inside a high-traffic request path." Or, "This is acceptable for a nightly job, but not for keystroke search." Or, "The database should own this lookup because it already has the index."
It turns performance from a surprise into a design conversation.
Complexity starts with the question#
The first mistake is asking "what is the fastest algorithm" before asking what the system needs.
Fast for what?
Fast for a single user clicking a button? Fast for a streaming job processing 50 million rows? Fast for a cold-starting serverless function? Fast for a dashboard that must feel instant, even if the answer is slightly stale? Fast for a fraud system where missing the rare case is more expensive than extra computation?
Different constraints change the answer.
Insertion sort is O(n^2), which sounds bad. For a tiny list that is already mostly sorted, it can be perfectly reasonable. A built-in sort is usually the right choice because runtime authors have already handled years of edge cases, optimizations, and weird inputs. A database index can make more sense than sorting in application code. A cache can beat both if the same question is asked all day.
Complexity does not replace judgment.
It gives judgment a vocabulary.
The everyday places it shows up#
Most product engineers do not spend their day implementing sorting algorithms from scratch.
They choose data structures.
They decide whether to scan an array or build a map. They decide whether to precompute a result or calculate it on demand. They decide whether a search belongs in application memory, a database query, a search engine, or a vector index. They decide whether to load a file into memory or stream it. They decide whether a batch job should run as one giant step or as chunks that can be retried.
That is algorithm complexity in the wild.
It is not glamorous.
It is the difference between a feature that ages well and a feature that becomes a tax.
Lookup: the simplest example that matters#
Imagine you have a list of users and you need to find one by id.
The direct version is a loop:
type User = {
id: string;
name: string;
};
function findUserById(users: User[], id: string): User | undefined {
return users.find((user) => user.id === id);
}This is O(n). In the worst case, the user is at the end or not there at all, so the function checks every item.
That is fine if users has 50 items and the function runs once.
It is less fine if it runs inside another loop:
function hydrateOrders(orders: Order[], users: User[]): HydratedOrder[] {
return orders.map((order) => ({
...order,
user: findUserById(users, order.userId),
}));
}Now the cost is roughly orders * users. If both are large, this quietly becomes O(n * m).
The fix is not complicated:
function indexUsersById(users: User[]): Map<string, User> {
return new Map(users.map((user) => [user.id, user]));
}
function hydrateOrders(orders: Order[], users: User[]): HydratedOrder[] {
const usersById = indexUsersById(users);
return orders.map((order) => ({
...order,
user: usersById.get(order.userId),
}));
}The index costs O(m) to build. Each lookup is usually O(1). The whole operation becomes closer to O(n + m).
Same feature.
Different shape.
That is the kind of optimization that actually matters in product code. Not cleverness. Structure.
Sorting: use the boring tool until the boundary changes#
Sorting is another place where developers can overthink the wrong part.
If you need to sort a small in-memory list, use the built-in sort and move on. In JavaScript and TypeScript, that is usually Array.prototype.sort(). In Python, use sorted() or .sort(). In most cases, the runtime implementation is better than the custom sort you are about to write.
The design question is not "should I implement merge sort."
The better questions are:
- How many records are we sorting?
- How often do we sort them?
- Is the data already sorted upstream?
- Is this happening on every request?
- Should the database return this in the right order?
- Can the UI page through the result instead of sorting everything?
- Is "top K" enough, or do we need a full ordering?
That last question matters.
If the user only needs the top 10 products by score, sorting one million products just to throw away 999,990 of them may be wasteful. A heap or database limit can avoid unnecessary work. Complexity points you toward the boundary: maybe the right solution is not a better sort, but not sorting the whole thing.
Memory complexity is where systems get honest#
Time complexity gets most of the attention.
Space complexity is where production systems often fail.
A function can be fast and still dangerous if it loads too much data into memory. This happens with large files, exports, imports, logs, images, analytics jobs, and model features.
The risky pattern is familiar:
const text = await file.text();
const rows = text.split("\n");For a small file, this is fine.
For a large file, the service may allocate the original file, the split array, many strings, and whatever transformed data comes next. The feature does not degrade gracefully. It falls over.
A streaming design changes the space complexity. Instead of holding the whole file, the system processes chunks or lines as they arrive. The total work may still be O(n), but memory can stay close to O(1) relative to file size.
That distinction matters for data products.
If you build reporting, integrations, ingestion, customer imports, model scoring, or event pipelines, memory behavior is part of the product. Users do not care that the algorithm is technically linear if the process dies halfway through their file.
Caching is not magic O(1)#
Caching is often described as an O(1) improvement.
Sometimes that is true in spirit. If a service repeatedly asks the same expensive question, a cache can turn repeated computation into a fast lookup.
But a cache is not a free performance spell.
It introduces invalidation rules, memory pressure, staleness, permissions, and observability questions. It can also hide a slow underlying design until the cache misses. In distributed systems, a cache can make the average case look excellent while making the tail case worse.
The useful complexity question is:
What are we avoiding, and how often do we avoid it?
If a product page calculates recommendations from scratch on every request, caching the final result may help. If the catalog changes often, user behavior changes minute by minute, and recommendations are permissioned, the cache key and invalidation model may be harder than the computation.
Complexity gets you to the conversation.
Architecture decides whether the trade is worth it.
A recommendation feature as a worked example#
Now take a small e-commerce feature.
We have products. Each product has an id, name, and category. We have a user's browsing history. We want to recommend the top K products from categories the user has already shown interest in.
Here is the data:
type Product = {
id: number;
name: string;
category: string;
};
const products: Product[] = [
{ id: 1, name: "Laptop", category: "Electronics" },
{ id: 2, name: "Phone", category: "Electronics" },
{ id: 3, name: "Shoes", category: "Fashion" },
{ id: 4, name: "T-Shirt", category: "Fashion" },
{ id: 5, name: "Headphones", category: "Electronics" },
];
const browsingHistory: Product[] = [
{ id: 2, name: "Phone", category: "Electronics" },
];
const k = 2;For this user, we want products from Electronics, excluding the product they already viewed. A reasonable output is:
[
{ id: 1, name: "Laptop", category: "Electronics" },
{ id: 5, name: "Headphones", category: "Electronics" },
];The naive implementation is direct:
function getRecommendationsNaive(
products: Product[],
browsingHistory: Product[],
k: number,
): Product[] {
const recommendations: Product[] = [];
for (const product of products) {
const wasViewed = browsingHistory.some((viewed) => viewed.id === product.id);
const matchesHistory = browsingHistory.some(
(viewed) => viewed.category === product.category,
);
if (!wasViewed && matchesHistory) {
recommendations.push(product);
}
if (recommendations.length === k) {
break;
}
}
return recommendations;
}This is readable.
It is also doing repeated scans of browsingHistory for every product. If products has n items and browsingHistory has h items, the cost is roughly O(n * h).
With one viewed item, nobody cares.
With a long browsing history and a large product catalog, it starts to matter. The implementation keeps asking the same questions:
- Has the user viewed this product?
- Is this category represented in the user's history?
Those are lookup questions.
So use lookup structures.
function getRecommendations(
products: Product[],
browsingHistory: Product[],
k: number,
): Product[] {
const viewedProductIds = new Set(
browsingHistory.map((product) => product.id),
);
const viewedCategories = new Set(
browsingHistory.map((product) => product.category),
);
const recommendations: Product[] = [];
for (const product of products) {
if (viewedProductIds.has(product.id)) {
continue;
}
if (!viewedCategories.has(product.category)) {
continue;
}
recommendations.push(product);
if (recommendations.length === k) {
break;
}
}
return recommendations;
}Now we build two sets in O(h), then scan products once in O(n) in the worst case. The total is O(n + h). The memory cost is also O(h) because we store viewed ids and categories.
That is a good trade for many request-time features.
The code is not fancy. It is just shaped correctly.
When the first optimization is not enough#
The Set version still scans the product catalog.
That may be fine. It may not.
If the catalog has a few thousand products, scanning it once is probably acceptable. If the catalog has millions of products and this endpoint is hot, scanning the whole catalog for every request is a bad bargain.
The next move is to pre-index products by category:
function indexProductsByCategory(products: Product[]): Map<string, Product[]> {
const index = new Map<string, Product[]>();
for (const product of products) {
const existing = index.get(product.category) ?? [];
existing.push(product);
index.set(product.category, existing);
}
return index;
}
function getRecommendationsFromIndex(
productsByCategory: Map<string, Product[]>,
browsingHistory: Product[],
k: number,
): Product[] {
const viewedProductIds = new Set(
browsingHistory.map((product) => product.id),
);
const viewedCategories = new Set(
browsingHistory.map((product) => product.category),
);
const recommendations: Product[] = [];
for (const category of viewedCategories) {
const candidates = productsByCategory.get(category) ?? [];
for (const product of candidates) {
if (viewedProductIds.has(product.id)) {
continue;
}
recommendations.push(product);
if (recommendations.length === k) {
return recommendations;
}
}
}
return recommendations;
}Now the request does not scan every product. It scans only products in categories the user has viewed, stopping once it has K recommendations.
The index costs O(n) to build and O(n) memory to store. But if the catalog is reused across many requests, that cost can be paid once and amortized. In a real system, this index may live in a database, cache, search engine, or recommendation service rather than in application memory.
This is where complexity becomes architecture.
The question changes from "how do I make this loop faster" to "where should this lookup live."
The product version is more complicated#
The category example is intentionally simple.
A real recommendation system has more rules.
You may need to remove out-of-stock products. You may need to respect geography, pricing, age restrictions, contractual visibility, brand exclusions, marketplace sellers, personalization rules, diversity constraints, and sponsored placements. You may need to avoid recommending the same product every time. You may need to rank by margin, conversion, similarity, freshness, or customer intent.
Each rule changes the shape of the algorithm.
If you bolt every rule onto one request-time loop, the feature becomes slow and brittle. If you precompute too much, recommendations become stale. If you push everything into the database, the query may become difficult to reason about. If you use a search service, you inherit indexing lag and scoring configuration. If you use a model, you still need retrieval, ranking, filtering, and fallback behavior.
This is why complexity matters beyond code challenges.
It helps you see which work should happen at write time, read time, build time, batch time, or request time.
That is product architecture.
The common failure modes#
The first failure mode is nested loops hiding inside helper functions.
The code reads nicely. Each function does a small thing. But the composition multiplies work. A loop calls a helper, the helper scans an array, that helper calls another helper, and the endpoint becomes slow without any one line looking guilty.
The second failure mode is sorting too much.
Teams often sort a whole collection to display a tiny page or top K result. That is sometimes fine. It is sometimes a quiet waste. If the product only needs a limited ranked set, use a query limit, heap, index, or precomputed ranking where appropriate.
The third failure mode is pretending memory is infinite.
Reading a full file, full table, or full API response into memory is the easiest version to write. It is also the version that tends to fail at the worst time, with the largest customer, during the most important import.
The fourth failure mode is moving work to the wrong system.
Application code scans because nobody added a database index. A database query does ranking work that belongs in a search engine. A cache stores user-specific data without considering permissions. A frontend filters thousands of rows because the backend endpoint is too broad.
The fifth failure mode is optimizing before measuring.
Complexity is a warning signal, not a replacement for profiling. Sometimes the theoretical hot spot is not the actual hot spot. Network calls, serialization, layout, database locks, and third-party APIs can dominate runtime. Use complexity to form hypotheses. Use measurement to confirm them.
What I look for in code review#
When reviewing product code, I am usually not asking whether every function has the best possible complexity.
I am asking whether the complexity matches the boundary.
For request paths, I look for repeated scans, unbounded loops, full collection sorts, missing pagination, and database calls inside loops.
For batch jobs, I look for memory spikes, lack of chunking, fragile retry behavior, and steps that cannot be resumed.
For APIs, I look for endpoints that return too much data and force every client to filter locally.
For UI code, I look for expensive recalculation on every render, especially when the input only changes occasionally.
For data systems, I look for joins, groupings, and aggregations that should be indexed, partitioned, materialized, or moved upstream.
None of this requires exotic algorithms.
Most of the value comes from noticing shape.
How to practice#
Take a feature you already understand and ask four questions.
First, what is the input size today?
Second, what could the input size become if the product works?
Third, where are we doing repeated work?
Fourth, what data structure or system boundary would let us ask the question directly?
That last question is the one that changes your engineering taste.
An array answers "what are all the things."
A map answers "what thing belongs to this key."
A set answers "have I seen this thing."
An index answers "which records match this condition without scanning everything."
A stream answers "can I process this without holding it all."
A cache answers "have I already paid for this result."
A queue answers "can this work happen outside the request."
Complexity teaches you which question your current structure is bad at answering.
The point#
Algorithm complexity is not about showing off.
It is about protecting the user from the shape of your first draft.
The first draft is allowed to be direct. It should be readable. It should prove the behavior. But once the behavior matters, you have to ask how it grows.
Will this work when the catalog is 10 times larger?
Will it survive a customer import?
Will it run on every request?
Will it duplicate work that a database index, cache, stream, or queue should own?
Will the next developer understand the trade?
That is where Big O becomes useful. Not as trivia. Not as a whiteboard ritual. As a practical habit for building features that do not collapse the moment people actually use them.