URL Shortener system design: how it works

How URL Shortener works, step by step

Turns long links into short ones and redirects fast, with a cache in front of the database.

Clients

Browser

Client · Hypothetical. Someone creating a short link, or clicking one.

Gateways

Load balancer

Gateway · Hypothetical. Spreads requests across the service instances.

Services

Shorten API

Service · Hypothetical. Creates a short code for a long URL.

ID generator

Service · Hypothetical. Hands out unique numbers so two links never share a code.

Redirect service

Service · Hypothetical. Looks up a short code and redirects the browser.

Queues

Click events

Queue · Hypothetical. Records each click for analytics without slowing the redirect.

Caches

Link cache

Cache · Hypothetical. Keeps popular short codes in memory.

Databases

Link database

Database · Hypothetical. The durable record of every short code and its long URL.

Flow: Create a short link

From pasting a long URL to getting a short link back.

1. Browser → Load balancer (Shorten this URL). Someone pastes a long URL and presses Shorten.
2. Load balancer → Shorten API (Shorten this URL). The load balancer sends the request to one of the shorten API instances.
3. Shorten API → ID generator (Next id?). The shorten API asks for the next unique number.
4. ID generator → Shorten API (1048576). It gets a number no one else will get, and encodes it in base62 as a short code like aZ3k9.
5. Shorten API → Link database (Save aZ3k9). It saves the short code together with the long URL.
6. Link database → Shorten API (Saved). The database confirms the link is stored.
7. Shorten API → Load balancer (Short link). The shorten API answers with the new short link.
8. Load balancer → Browser (Short link). The browser shows the short link, ready to copy.

Flow: Follow a short link (cache hit)

The usual path: from clicking a short link to landing on the long URL, answered from the cache.

1. Browser → Load balancer (GET /aZ3k9). Someone clicks a short link.
2. Load balancer → Redirect service (GET /aZ3k9). The load balancer sends the click to one of the redirect service instances.
3. Redirect service → Link cache (Look up aZ3k9). The redirect service checks the cache first.
4. Link cache → Redirect service (Long URL). Most of the time the cache already has it, so no database call is needed.
5. Redirect service → Load balancer (301 redirect). The redirect service answers with a redirect to the long URL. It also drops a click event on the queue for analytics, without waiting for it.
6. Load balancer → Browser (301 redirect). The browser follows the redirect and opens the long page.

Flow: Follow a short link (cache miss)

The slower path: the cache doesn't have the link, so the database answers and the cache is filled for next time.

1. Browser → Load balancer (GET /aZ3k9). Someone clicks a short link that nobody has clicked lately.
2. Load balancer → Redirect service (GET /aZ3k9). The load balancer sends the click to one of the redirect service instances.
3. Redirect service → Link cache (Look up aZ3k9). The redirect service checks the cache first, as it does for every click.
4. Link cache → Redirect service (Not in cache). The cache doesn't have this code. The link may be new, or it may have gone unused for long enough to be removed to make room.
5. Redirect service → Link database (Find aZ3k9). Only now does the redirect service go to the database, which is slower than the cache but holds every link.
6. Link database → Redirect service (Long URL). The database returns the long URL. If it had no such code either, the link wouldn't exist and the answer would be a 404.
7. Redirect service → Link cache (Save aZ3k9). The redirect service copies the answer into the cache with an expiry time, so the next click on this link is a cache hit.
8. Redirect service → Load balancer (301 redirect). The redirect service answers with a redirect to the long URL. It also drops a click event on the queue for analytics, without waiting for it.
9. Load balancer → Browser (301 redirect). The browser follows the redirect and opens the long page. This click took longer than a cache hit, but later clicks won't.