Tries & Digital Search Trees

AVL trees, Red-Black trees, Splay trees: all give O(log n) insert, delete, search, successor, predecessor, min/max — as long as comparing two keys is O(1). That's true for integers. It is not true for strings.

Comparing two strings of lengths r and s costs O(min(r, s)) — you have to walk both strings to find where they first differ (or confirm they're equal).

Put strings in a balanced BST and every one of those O(log n) comparisons secretly costs O(M), where M is the longest string in the tree. Total: O(M log n), not O(log n). The tree structure is fine — the comparisons are the hidden cost.

Question: can we design a structure where the cost of an operation depends only on the length of the word we're searching for — never on how many words are stored, and never with a repeated re-comparison of characters we've already matched? That structure is the trie.

Fix an alphabet Σ. A trie is a tree where every node stores:

• A bit marking whether the string spelled out from the root to this node is itself a stored word.
• An array of |Σ| pointers — one possible child per character of the alphabet.

Every node corresponds to exactly one string: the sequence of characters on the path from the root down to it.

Looking up a string w means following at most |w| pointers — each an O(1) array lookup. Total cost: O(|w|). Notice what's missing from that bound: n, the number of strings stored. A trie with 10 words and a trie with 10 million words answer the same query in the same time, provided the query word itself is short. That is the entire promise of a trie — and it directly answers last slide's question.

insert(word):
  v = root
  for each character c in word:
      if v has no child for c:
          create that child       // new node
      v = v.child[c]              // descend
  mark v as "end of word"

Before you watch: each box below is one trie node, and its letter is the edge that reaches it from its parent — not the node's entire storage. Every node really holds a full array of 26 pointer slots (one per possible next letter); the diagram only draws whichever slot is actually filled and leaves the other ~25 empty ones out, purely for readability. Those hidden empty slots are exactly what L13·07's space cost — Θ(N·|Σ|) — is warning about: they're still allocated in real memory, one box just doesn't draw the rule to show you the exception.

Amber nodes mark the end of a stored word (a "●" confirms it); teal nodes are pure branching points, not words themselves. Watch which letters get reused from an earlier word and which are genuinely new — that reuse is the entire space-saving idea of a trie. "bear" and "bell" are walked one character at a time so you can see exactly where the pseudocode's create-vs-reuse decision happens; every word after that inserts in a single step, since by then the pattern is established.

Descend along the prefix's own characters to find the node for "bu" — exactly like a search, but you don't need that node to be a word-end itself. Then explore every node in the subtree below it, collecting every word-end found. This is precisely what an autocomplete box, or a phone contact search, does on every keystroke.

A BST ordered by string comparison would need a range query ("everything between 'bu' and 'bv'") — correct, but still paying string-comparison costs at O(log n) nodes. A trie finds the prefix node in O(|prefix|) and then just walks a subtree — no comparisons against unrelated words at all.

delete(word):
  v = the node found by searching for word
  unmark v as "end of word"
  while v has no children and v is not a word-end:
      parent = v's parent
      remove v from parent
      v = parent

Delete "bull". Its path shares letters with "buy" — the pruning must stop exactly at the node still needed by "buy", and go no further. Removing one node too many would silently delete a different word.

A trie with N nodes needs Θ(N · |Σ|) space — every node carries a full array of |Σ| pointers, even when only one or two are ever used. For English text (|Σ| = 26) that's wasteful; for a genome (|Σ| = 4) it's less severe, but the idea generalizes badly to large alphabets.

Whenever a node has exactly one child and is not itself a word-end, it's carrying no branching information — merge it into the edge above it. Repeat until every remaining node either branches or ends a word. This is a compressed trie (also called a PATRICIA-style trie), and it can be stored in O(s) space, where s is the number of words, using index ranges instead of full substrings at each node.

Every key is 4 bits, so every key needs a 4-edge path, and only the leaf is the key. Everything above it exists purely to be walked through:

root
├─0─ a ─0─ b ─0─ c ─0─ [0000]
│         └──1─ d ─0─ [0010]
│    └──1─ e ─1─ f ─0─ [0110]
└─1─ g ─0─ h ─0─ i ─1─ [1001]
              └──1─ j ─1─ [1011]

1 root + 2 + 3 + 5 + 5 leaves = 16 nodes. Eleven of them (a…j and the root) store nothing. For n keys of b bits, that's up to O(n·b) nodes — the cost now scales with key length, not alphabet size.

Store the actual key at every node — like a BST — but decide left-or-right using one bit of the key at each depth instead of comparing whole keys. Depth 1 uses bit 1, depth 2 uses bit 2, and so on.

Every node is now a real, useful key; nothing is a pure router. Five keys, exactly five nodes — you'll build this very tree in the next animation, and it comes out at height 3. That is a Digital Search Tree (DST).

A plain BST over these keys can degenerate into a chain depending on insertion order; AVL and Red-Black fix that, but only by carrying balance factors or colour bits and doing rotations on the way back up. A DST branches on bits of the key rather than on comparisons against what's already stored — so it stays short by construction. Watch the next animation finish with zero rotations and no balance metadata anywhere.

Nothing is free. In a trie only the final node needs a real check; every step before it is a bare array lookup. In a DST every node holds a genuine key, so you pay a full key comparison at every level (L13·11 traces exactly this). That trade only pays off when a comparison is cheap — which is precisely the case for fixed-length keys that fit in a machine word: a 32-bit address, an ID, a hash code.

• An empty DST is empty. A non-empty DST's root holds any one key-value pair from the set — there's no ordering requirement on which key goes where.
• All remaining pairs whose key's first bit is 0 go into the left subtree; all whose first bit is 1 go into the right subtree.
• The left and right subtrees are themselves Digital Search Trees, built on the remaining bits (bit 2 decides the split one level down, bit 3 the level after that, and so on).

The first key inserted becomes the root. Every later key descends from the root using its own bits, read most-significant-bit first, choosing left (bit = 0) or right (bit = 1) at each level — until it reaches an empty pointer, where it is inserted as a brand-new node. Its final depth is however many bits it took to find that empty slot.

In a trie, only the last node reached needs its isWord bit checked — every earlier step is just an array lookup. In a DST, every node holds a real key, so at every level you must compare the target against the key stored there before deciding whether to continue.

Search 1: look up 1011, which is stored — found after 2 comparisons and a match on the third node. Search 2: look up 1010, which is not stored — watch it fail only after reaching a node with no further child to follow.

In a DST, whatever sits at depth d is reached because bit d of its key was read at that level — depth 1 reads bit 1, depth 2 reads bit 2, and so on, for every node, always. A BST splits deletion into three cases by child count (0, 1, or 2) and has a safe move for each. A DST doesn't get that luxury: promoting any subtree to a shallower depth can break this rule for everything beneath it — and that danger shows up whether the node you're deleting has one child or two.

Two small examples, one after another: first, deleting a node with exactly one child — the obvious "just promote it" move — and where it silently loses a key. Then a two-child deletion, using BST's actual rule (copy the in-order successor's key up, remove the successor from below) — showing that it only survives by luck, not by any real guarantee.

In a BST, deleting an internal node promotes its in-order predecessor or successor — any descendant will do. In a DST that's wrong: every node's depth encodes which bit it's supposed to branch on. Promoting an internal descendant to a shallower depth would silently misalign its own children's bit rule.

Delete key 0110, the root. Instead of promoting any descendant, find a leaf anywhere in its subtree (a leaf has no descendants of its own to misalign), copy that leaf's key into the deleted spot, and remove the leaf from its original position — a trivial removal, since leaves have no children.

IPv4 has 4 billion addresses. No router can hold one rule per address, so it holds rules for blocks instead — and a block is written as a prefix: 10* means "every address whose first two bits are 1, 0".

Sorting post, same idea. Two rules on the wall: PIN starting 30… → Rajasthan bag; PIN starting 3020… → Jaipur bag. A letter for 302017 matches both. It goes in the Jaipur bag, because 3020… is the more specific rule. That is longest-prefix matching, and it is the whole problem.

• What goes INTO the trie: the routing table's prefixes — the rules on the wall. Each prefix is a path down from the root, and the node where that path ends is marked with its cable.
• What you SEARCH with: one arriving packet's complete destination address. Not a prefix — a full address.
• The answer: the deepest marked node you passed on the way down. Deepest = longest = most specific.

Out it goes on cable B. The rule is simply: keep walking, remember the last mark you passed, stop when you fall off the tree.

Scale it up: roughly a million prefix rules covering 4 billion addresses, every packet resolved in at most 32 steps. That compression is why the structure exists.

Prefix search (L13·05): you are given a prefix and want every key underneath it — "st" → stock, stop. Walk down, then collect everything below.

Longest-prefix match (here): you are given a full key and want the deepest stored prefix sitting above it on the path. Walk down, remember the last thing you passed. Same tree, opposite directions.

A hash table needs an exact key. To use one here you would have to look up all 32 possible prefix lengths of the address separately and keep the longest hit. The trie answers it in a single walk, because the tree's shape already encodes "more specific = further down".

A DST is the wrong tool for this one: it stores each whole key at the first free node it finds, in a position that has nothing to do with what that path spells. Longest-prefix matching needs exactly the opposite — a prefix must sit at the node its own path spells out. Real routers use bit-tries and their compressed PATRICIA form.