Huffman Trees & Applications

Lecture 4, the 13-record trace: the lone run 15 was read and rewritten in every single pass while bigger runs merged around it. You asked (we made you ask): could a smarter schedule avoid that waste?

Lecture 5, replacement selection: runs now come out unequal by design — ours were lengths 4, 8, 1. Unequal runs are no longer an accident; they are the normal output of a good Phase 1. So "in what order do we merge unequal runs?" is now a question we face on every single sort.

When two runs of lengths a and b are merged, every record of both is read and written once — cost proportional to a + b. And the output run of length a + b may be merged again later, moving those same records again.

So a record's total movement = how many merges its run participates in. Records in a run merged early and often move many times; records in a run merged once move once. Intuition forming: small runs can afford many merges; big runs should merge as few times as possible. Let's make that precise — and provable.

• A merge tree records a merge schedule: square external nodes = the initial runs (with their lengths as weights); circular internal nodes = merges, weight = sum of children.
• A record moves once for every internal node above its run — i.e. its run's depth.
• Total merge cost = Σ (run length × its depth) = Weighted External Path Length:

The second equality is the practical one: just add up the merge sums. Same number, two readings.

put all n weights in a min-heap, each a 1-node tree
repeat n − 1 times:
    a ← pop smallest tree
    b ← pop next smallest
    c ← new node, children a and b,
        weight = a + b          // one merge
    push c
// the single remaining tree is optimal

The forest starts as n single runs. Each step, the two lightest trees fuse into a new tree whose weight is their sum — and that sum is exactly the cost of that merge, posted to the ledger. When one tree remains, the ledger total is the WEPL. Weights: 4, 6, 8, 9, 15, 28 (a favourite exam configuration).

Huffman order: merge the two smallest first — 1 + 4 = 5 (cost 5), then 5 + 8 = 13 (cost 13). Total movement: 18 records.
WEPL check: 1·2 + 4·2 + 8·1 = 2 + 8 + 8 = 18 ✓

The other order: 4 + 8 = 12 (cost 12), then 12 + 1 = 13 (cost 13). Total: 25 records.
WEPL: 4·2 + 8·2 + 1·1 = 25. The big run of 8 got dragged through two merges — 39% more I/O, for choosing the wrong order on just three runs.

• Lecture 4's lone "15": carrying it down pass after pass was a bad merge tree — it sat at maximum depth. Huffman would merge it into the smallest partner immediately: minimum total movement, question answered.
• Was Lecture 4 wrong to pair evenly? No — when all runs are equal, Huffman itself produces the balanced, complete tree (all weights tie, pairing is level by level). Balanced pairing is the special case; Huffman is the general law. (Homework asks you to argue this.)

For k-way merging the same idea generalizes: a k-ary Huffman tree (pad with dummy zero-length runs so the arity works out) — mentioned for completeness, not examined in depth.

A 100-character file uses 6 letters with these counts: a:45 · b:13 · c:12 · d:16 · e:9 · f:5. A fixed-length code needs ⌈log₂ 6⌉ = 3 bits per character → 300 bits.

Idea: give frequent letters short codes and rare letters long ones — like Morse code gave 'E' a single dot. Danger: with variable lengths, where does one code end and the next begin? Solution: a prefix code — no codeword is a prefix of another. Then decoding is unambiguous: read bits, walk the tree, hit a leaf, emit, restart.

And here is the punchline: expected code length = Σ freq × depth = WEPL again. Minimizing bits is the same problem as minimizing merge cost — frequencies play the role of run lengths. Same greedy, same tree.

• Build the Huffman tree on frequencies (animation →).
• Label every left edge 0, every right edge 1.
• A character's code = the 0/1 path from root to its leaf. Leaves only ⟹ prefix property is automatic.
• Frequent characters end up shallow (short codes); rare ones deep (long codes) — Huffman guarantees the optimum.

Decode demo for later: with the tree built, 1100·0·100·1101 reads f–a–c–e: "face" in 12 bits instead of 4 × 3 = 12… try "dead" and "cab" in the homework — that's where the savings show.

Suppose someone built a different tree on just {4, 6, 28}: merge 28 with 4 first (32), then merge with 6. Result: 6 sits at depth 1, while heavy 28 sits deeper, at depth 2.

Swap 6 and 28's positions (heavy rises to depth 1, light sinks to depth 2) — nothing else about the tree changes:

Cost dropped from 70 to 48 just by moving the heavier weight shallower. Any tree with a heavy weight sitting deeper than a light one can always be improved this way — so it was never optimal to begin with.

Swapping a heavy weight (at deep level d_deep) with a light weight (at shallow level d_shallow) changes the cost by exactly:

Check it against the example: −(28−6)×(2−1) = −22 — exactly the 70→48 drop.

Both factors are always positive whenever heavy-sits-deeper happens (bigger weight, bigger depth-gap) — so ΔWEPL is always negative. Conclusion: in any optimal tree, no heavier weight can sit deeper than a lighter one — lighter weights always sink to the bottom.

Go back to the real 6-weight tree from L6·03. Its deepest level (depth 4) holds exactly one sibling pair: 4 and 6 — the two lightest weights of all six — sitting under their shared parent node "10". This is not a coincidence: every internal node has exactly two children, so the deepest occupied level always comes in sibling pairs, and by Obs 1, the lightest available weights are exactly the ones Obs 1 will always push into those bottom slots. So some optimal tree always begins with the two lightest as siblings — precisely Huffman's first move.

Merge 4+6→10 (cost 10) and the 6-weight problem becomes a 5-weight problem: {10, 8, 9, 15, 28} — literally the forest after L6·03's animation step 1. The same two arguments now apply to this smaller forest: its own two lightest (8, 9) must again be optimal siblings at its own deepest level.

Repeat: 6 weights → 5 → 4 → 3 → 2 → 1, each step justified the same way. Complexity: heap init O(n); n−1 pops/pushes at O(log n) each → O(n log n) total — seconds by hand with the forest-and-ledger method.