GamifiedGamified challengesPracticeRaceLeaderboardPricing

Learn data structures by watching them move, then doing it yourself. Press M anytime to toggle sound.

A product of Amypo Technologies, Coimbatore.

Learn

  • All topics
  • Learning path
  • How it works
  • About dotprep

Practice

  • Practice arena
  • Daily challenge
  • Gamified challenges
  • Race mode
  • Leaderboard

Interview prep

  • Java interview
  • Coding questions
  • Company questions
  • System design
  • Jobs & careers

You

  • Dashboard
  • Settings
  • Pricing
  • Create account

© 2026 dotprep

Content last updated 20 November 2026

GamifiedGamified challengesPracticeRaceLeaderboardPricing
System designScalability basics

Scalability basics

Vertical vs horizontal scaling, stateless services, latency vs throughput, and the path from one server to many.

beginner 11 min read 4 interview questions

On this page

1/6
  1. 01Vertical and horizontal scaling
  2. 02Stateless services
  3. 03From one server to many
  4. 04Latency, throughput and percentiles
  5. 05Interview questions
  6. 06Quick check
Back to top

Vertical and horizontal scaling

Vertical scaling (scale up) means a bigger machine: more CPU, memory, faster disks. It's simple, needs no code changes, and is often the right first step. But there's a ceiling, big machines get expensive fast, and one machine is still a single point of failure.

Horizontal scaling (scale out) means more machines behind a load balancer. It has no hard ceiling and survives individual failures, but the application must be designed for it: any server must be able to handle any request.

VerticalHorizontal
LimitLargest machine you can buyPractically none
FailureSingle point of failureSurvives losing a node
ComplexityLowLoad balancing, shared state, coordination
Cost curveGrows faster than linearlyRoughly linear with commodity machines

Stateless services

To add servers freely, web and application servers should be stateless: they keep nothing between requests that another server would need. Session data, uploads and caches move to shared stores (a database, Redis, object storage), so the load balancer can send each request anywhere.

  • Sticky sessions (pinning a user to one server) work, but break when that server dies and make load uneven.
  • Store sessions in a shared cache, or use signed tokens (such as JWTs) that carry the session with each request.
  • Put user files in object storage (like S3), not on a server's disk.

From one server to many

  1. One server runs the app and the database.
  2. Move the database to its own machine, so each can be sized separately.
  3. Put a load balancer in front of several stateless app servers.
  4. Add a cache for hot reads and a CDN for static files.
  5. Add database read replicas; send reads to replicas, writes to the primary.
  6. Move slow work (emails, thumbnails) to a message queue and background workers.
  7. When one database can't take the writes or the data, shard it.

Each step fixes the current bottleneck. In an interview, describe which bottleneck you're addressing rather than adding every box at once.

Latency, throughput and percentiles

Latency is how long one request takes; throughput is how many requests the system handles per second. They're related but different: batching raises throughput and can raise latency.

Report latency as percentiles, not averages. A p99 of 800 ms means 1 in 100 requests takes at least 800 ms. A page that makes 50 backend calls will hit a p99 call on most loads, so tail latency matters more than the mean.

See it animated

Big-O complexity

Interview questions

Try answering out loud first, then open the model answer.

Q1When would you scale vertically rather than horizontally?beginner

Early on, or for components that are hard to distribute (a relational database primary). It's cheaper in engineering time and keeps the system simple. Switch to horizontal when you approach the hardware ceiling, need fault tolerance, or the cost curve turns against you.

Q2Why should application servers be stateless?beginner

So any server can handle any request: the load balancer can spread load evenly, you can add or remove servers at any time, and a crash loses no user state. State lives in shared stores built for it.

Q3What is p99 latency and why does it matter?intermediate

The latency that 99% of requests beat. Averages hide slow outliers, and when one user action fans out to many backend calls the chance of hitting at least one slow call is high, so the tail becomes the experience. Systems are usually tuned against p95/p99 targets.

Q4What is a single point of failure and how do you remove it?beginner

A component whose failure takes down the whole system: one database, one load balancer. Remove it with redundancy: multiple instances, replicas with automatic failover, a pair of load balancers with a floating IP, and deployment across availability zones.

Quick check

Question 1

1.Horizontal scaling means…

Question 2

2.Which makes it easiest to add app servers?

Question 3

3.Which number best describes tail latency?

0 of 3 answered

How to approach a design interviewLoad balancing
Log inStart free