Amortized Analysis Techniques

A pile of plates in the mess. You can only touch the top: place a plate on top (PUSH) or take the top plate off (POP). Last In, First Out — LIFO. Both touch one plate, so each costs 1 unit of work.

The new operation today: MULTIPOP(S, k) — "pop up to k elements in one call."

MULTIPOP(S, k)
  while not EMPTY(S) and k ≠ 0
      POP(S)        // 1 unit per element removed
      k ← k − 1

Read the loop guard carefully — it stops when either k elements are popped or the stack runs empty, whichever comes first. So its cost is:

Example: MULTIPOP(S, 5) on a stack holding only 2 elements pops just 2 and stops. Cost = min(2, 5) = 2 — not 5.

Step 6 unrolled, iteration by iteration: k=3, pop 40 (k becomes 2) → pop 20 (k becomes 1) → pop 10 (k becomes 0, loop exits). Three pops, cost 3.

Total: 8 units for 6 operations — average 1.33 per operation, even though one operation alone cost 3. Hold that thought.

A 4-bit counter is an array A[3] A[2] A[1] A[0] of 0s and 1s. A[0] is the ones place, A[1] the twos, A[2] the fours, A[3] the eights:

Counting works like a car odometer. In decimal, 199 + 1: the 9 rolls to 0 and carries, the next 9 rolls to 0 and carries, then 1 becomes 2 → 200. Binary is identical, except digits roll over at 1 instead of 9: 0111 + 1 = 1000.

INCREMENT(A)   // cost = number of bits flipped
  i ← 0
  while i < k and A[i] = 1
      A[i] ← 0            // this 1 rolls over: carry
      i ← i + 1
  if i < k then A[i] ← 1  // carry lands here

In words: walk up from A[0], turning every 1 you meet into 0 (the carry rippling), and when you first meet a 0, turn it into 1 and stop. Cost = number of bits you touched.

Result: 0111 → 1000, which is 7 → 8. ✓  Four bits were flipped, so this increment cost 4 — the worst possible on 4 bits.

Now increment again, 1000 → 1001 (8 → 9): the loop looks at A[0], finds 0, exits immediately, sets A[0] ← 1. Cost 1. The expensive increment zeroed the low bits — and made its successors cheap. Hold this thought: it becomes "Reason 1" on the next slide.

An expensive operation leaves the structure in a cheap state

The cost of operation i is not independent of operations 1…i−1: each operation reshapes the structure, and an expensive operation usually reshapes it in a way that makes the following operations cheap. Multiplying n × worst-cost silently assumes each operation faces a freshly worst-case structure — which the previous expensive operation just destroyed.

n operations can't all hit their worst case in one sequence

An operation's worst case requires the structure to be in a specific extreme state — and reaching that state consumes operations from the same budget of n. The expensive event and its own setup compete for the same sequence, so the expensive event is necessarily rare.

The hostel mess card

You pay ₹3,000 once a month, then eat "free" for 30 days. Is a meal expensive? The single-payment view says ₹3,000. The honest view spreads (amortizes) it: ₹100 per day, every day, guaranteed — no probability, no luck involved. Amortized analysis does exactly this to operation costs.

Union-Find in Kruskal's MST

Kruskal's does 2|E| find operations and |V|−1 unions. One find with path compression can be expensive — but it flattens the tree, so the next finds on that path are nearly free. Charging every find its worst case ignores this "one pays, many benefit" structure entirely.

• A worst-case guarantee — over any adversarial sequence.
• Deterministic — holds for every run, not on average luck.
• An upper bound: total amortized ≥ total actual, always.

• Not average-case analysis — no probability distribution on inputs.
• Not a claim about one operation — a single call may still be slow.
• Not a replacement for Big-O — amortized costs are still expressed in Big-O; what changes is the quantity being bounded.

Sum the invented prices: Σ ĉ_i = 2 · (number of PUSHes), since only PUSH is ever charged anything. Sum the real work: Σ c_i = (number of PUSHes) + (number of POPs, whether standalone or inside a MULTIPOP). Every popped element was pushed exactly once, so (number of POPs) ≤ (number of PUSHes). Therefore Σ c_i ≤ 2·(number of PUSHes) = Σ ĉ_i. Law 1 holds — total invented cost never falls short of total real cost, for any sequence.

At any prefix ending at operation t, Σ_i≤t ĉ_i − Σ_i≤t c_i is exactly "total rupees taped on so far minus total rupees spent so far" — i.e. the rupees still taped onto elements currently on the stack. That equals the current stack size, a count of physical elements, which can never be negative. Law 2 holds at every single moment, not just at the very end.

Try the accounting trick on a 2-3 tree: insert an element and tape an O(lg n) coin onto it, to be spent when that same element is later deleted — identical in spirit to the multipop stack's PUSH-prepays-a-POP move. Here it fails.

By the time the element is deleted, the tree may have grown to n′ > n. Deleting now costs O(lg n′) — but the coin taped at insertion only covers the smaller, stale O(lg n). The coin comes up short.

Don't charge a delete to the insert of the same element. Charge it to whichever insert most recently grew the tree to its current size n′ — a different operation, possibly far away in the sequence.

That insert can be charged at most once: for the tree to reach size n′ again after shrinking, another insert has to regrow it. DELETE becomes amortized O(1); INSERT absorbs its own O(lg n) work plus this one contingent future charge.

Meter at the wall socket, not at the device

An inverter at home: most of the day it charges quietly from the mains; during a power cut it discharges to run the fans. If you meter consumption at the fans, usage is spiky — nothing for hours, then a burst. If you meter at the wall socket, the draw is small and steady, because the burst was prepaid into the battery over many quiet hours.

The potential method meters your data structure at the wall socket. Φ ("phi") = the battery's charge level. Cheap operations do their small work plus a little charging (Φ rises). The expensive operation's burst is powered mostly by discharge (Φ falls). The steady wall-socket reading is the amortized cost:

We were already doing this — just with coins

Look back at the accounting method's bank balance. At every moment, the total credit could be read directly off the structure's current shape — no memory of history needed:

The potential method's only new idea: skip the coins. Don't track who holds what — just define the balance directly as a function of the current state, and call it Φ. One number replaces all the bookkeeping.

Φ(D₀) = 0 (empty), Φ ≥ 0 always. ✓

(k′ = min(|S|, k).) Every operation is amortized ≤ 2 ⟹ O(1). Note it reproduces the accounting method's prices — with zero creativity about where coins live.

Φ(D₀) = 0 (all zeros), Φ ≥ 0 always. ✓

Say increment i resets t_i bits (the carry run) and sets at most one bit:

The t_i — the entire expensive cascade — cancels algebraically. That cancellation is the whole method. INCREMENT is amortized O(1).

A dynamic array (C++ vector, Java ArrayList, Python list) keeps two numbers: n = elements stored, cap = slots allocated. Append works like this:

APPEND(x)
  if n = cap                    // table full?
      allocate new array of size 2·cap
      copy all n elements across  // cost n
      cap ← 2·cap
  write x into slot n; n ← n+1    // cost 1

So an append costs 1 normally, but n + 1 when it triggers a doubling. Worst case of one call: Θ(n). Yet every language calls append "O(1)" — amortized, and now we can prove it three ways.

A cost-2ʲ spike happens only once every 2ʲ appends — rarity and size grow at exactly the same rate, so the running total never crosses the 3n ceiling. That "3" is about to appear in all three proofs.

Total work of n appends = n unit writes + all the copying. Copies happen at sizes 1, 2, 4, 8, … so total copy work = 1 + 2 + 4 + … + 2^{⌊log(n−1)⌋} < 2n (a geometric sum — each term is more than everything before it combined, and the last term is < n… so the sum is < 2n).

T(n) < n + 2n = 3n ⟹ 3 per append, O(1).

Stand at the moment just after a doubling: capacity 2m, m elements, all savings spent. Before the next doubling, exactly m more appends arrive. Charge each ₹3: ₹1 writes it, ₹2 banked. At the next doubling, 2m elements must be copied, and the bank holds m × ₹2 = ₹2m — exactly enough. Each new element's ₹2 copies itself plus one old element whose savings died in the previous doubling.

Φ = 2n − cap (≥ 0: right after a doubling the table is exactly half full, and it only fills from there).

Normal append: c = 1; n rises by 1, cap unchanged → ΔΦ = +2; ĉ = 3.

Doubling append: just before, table is full: n = cap, so Φ = 2n − n = n. The operation copies n and writes 1: c = n + 1. After: n+1 elements, cap = 2n → Φ = 2(n+1) − 2n = 2. So ΔΦ = 2 − n, and ĉ = (n+1) + (2−n) = 3. The n's cancel — the same signature cancellation as the counter's carry run.