Skip to content

Programming

Scaling Your Backend API Without a Rewrite

Find the real bottleneck before you re-architect: N+1 queries, indexes, pooling, caching, queues and replicas, in order, with when each stops working.

Anika Rukhsar
Anika Rukhsar
Founder
Published
Jun 27, 2026
Reading time
9 min read
Scaling Your Backend API Without a Rewrite

Most backend APIs that "can't scale" are being held back by one or two specific bottlenecks, not by their architecture. Measure where the time goes, then fix things in a set order: query shape, indexes, connection pooling, caching, background jobs, read replicas. Splitting into services comes last. Each step costs more than the one before it, so you only move on when the cheaper fix has been measured and has run out.

Key takeaways

  • Measure p95 latency per endpoint before changing anything. Without a baseline you can't tell a fix from a coincidence.
  • The cheapest fixes usually matter most: fetch only the fields you need, kill N+1 queries, and add the index the query plan asks for.
  • Caching multiplies whatever you cache, including wrong answers. Check status codes and freshness rules before you widen it.
  • Size connection pools, memory limits and worker counts from usage measured under real load, not from defaults or guesses.
  • Split out a service when one workload has a different scaling profile that you can't isolate any other way. Fashion isn't a reason.

Measure first: the only baseline that matters

Before you touch code, get three numbers for each of your busiest endpoints: request rate, p95 latency, and error rate. Averages hide the problem. An endpoint can average 120 ms while one request in twenty takes over a second, and that slow twentieth is what your users notice and what times out mobile clients.

You don't need an expensive APM product to start. Structured request logs with a duration field are enough to work out percentiles. Then break each slow request into its parts: time in the database, time waiting for a connection, time in outbound HTTP calls, time serialising the response. Whichever part dominates tells you where to look in the table below.

Load-test the hot paths with a tool such as k6, Gatling or Locust, using realistic data volumes. A query that is instant against 500 seeded rows behaves differently against two million.

The order of operations

Work down this table. Don't skip a row because the next one sounds more interesting.

StepSymptom you'll seeLikely causeCheapest fixWhen it stops being enough
1. Over-fetching / N+1Query count per request grows with page size; large response payloadsORM lazy-loading relations in a loop; SELECT * for cards that show four fieldsEager-load or batch relations; select only the fields the client renders; paginateQueries are already minimal but each one is still slow
2. Missing indexesOne query dominates DB time; sequential scans in EXPLAINFilters and sorts on unindexed columnsComposite index matching the WHERE + ORDER BY; check the plan again afterwardsWrite latency rises from too many indexes, or the working set no longer fits in memory
3. Connection poolingRequests wait before the first query runs; "too many connections" errors at peaksEach app instance opens its own connections; serverless or many workers multiply themA pooler (e.g. PgBouncer) in front of the database; pool size set from measured concurrencyThe database is CPU- or IO-bound, not connection-bound
4. Caching layersSame reads repeated thousands of times; DB load tracks page viewsEvery request recomputes data that changes rarelyHTTP cache headers, CDN, ISR for rendered pages, then an application cache (Redis) for shared hot objectsData is personalised or must be fresh to the second; invalidation becomes the main source of bugs
5. Background jobs / queuesSlow endpoints that send email, call third parties, resize images or build reportsNon-essential work done inside the requestReturn quickly, push the work onto a queue with idempotent workers and retriesQueue depth grows faster than workers can drain it, or jobs need strict ordering across many producers
6. Read replicasPrimary DB CPU high, mostly from reads that caching can't absorbRead-heavy list, search and reporting queriesRoute tolerant reads to a replica; keep read-your-own-writes paths on the primaryWrite volume itself saturates the primary
7. Split servicesOne workload's load profile hurts everything else, and isolating it inside the app has failedGenuinely different scaling, release or security needsExtract that one capability behind a stable API (strangler pattern)—

Step 1: stop fetching what you don't render

N+1 queries are the classic version: load 50 orders, then run one query per order to get its customer, giving 51 round trips where 2 would do. Most ORMs have an eager-loading or batching option. Turn on query logging in development and count queries per request. If the count changes with page size, you've found one.

The less obvious version is over-fetching whole rows. On our own site, the blog listing page was fetching every post's full body just to render cards that show a title, excerpt, date and image. The index response came to around 2 MB. Selecting only the card fields cut the payload sharply and helped keep Lighthouse performance scores at 93 to 100 across the site. No new infrastructure, just a narrower query.

Step 2: index for the queries you actually run

Run EXPLAIN ANALYZE (PostgreSQL) or EXPLAIN (MySQL) on the slowest queries from your logs. Look for sequential scans on large tables, and for sorts that spill to disk. A composite index whose column order matches your filter followed by your sort often turns a scan into a lookup.

Each index slows writes and uses memory, so keep only the ones that help a query that matters.

Step 3: pool connections, and address them properly

Databases handle a modest number of concurrent connections well and many thousands badly. Horizontal scaling makes this worse: ten app instances, each with a pool of 20, means 200 connections before you've served any real traffic. A connection pooler in front of the database lets many clients share a small number of real connections.

The pooler then becomes a dependency, and it fails in its own ways. In one of our deployments, a container restart gave the database a new internal IP address. The pooler was still configured with the old one, so the CMS behind it started returning 500s while the database itself was perfectly healthy. The fix was to address services by name on a shared Docker network and never hard-code an IP.

Step 4: cache in layers, and cache the right answer

Work from the outside in:

  1. HTTP caching: Cache-Control, ETag and stale-while-revalidate on responses that are the same for everyone.
  2. CDN: serve those cacheable responses from the edge so most requests never reach your origin.
  3. ISR or static generation for rendered pages that change on a schedule or on publish.
  4. Application cache (Redis, Memcached) for hot shared objects such as configuration, catalogue entries and permission lookups, using cache-aside with explicit TTLs.

A cache stores whatever you give it, mistakes included. On our own Next.js site, incremental static regeneration with dynamic params meant notFound() returned HTTP 200 for made-up URLs. Search engines saw thousands of junk pages that looked valid, and they were cached like real ones. We fixed it with middleware that checks the requested slug against the sitemap and the CMS before rendering, then returns a real 404, or a 301 to the closest real post for typos. Before you widen any cache, request a URL that shouldn't exist and check the status code that comes back.

Write your invalidation rules before you switch caching on. Prices, stock levels and permissions usually need short TTLs or event-driven purges.

Step 5 and 6: move work out of the request, then spread reads

Anything the user doesn't need in order to see a response belongs in a queue: emails, webhooks to partners, image processing, exports. Make workers idempotent, using a job key or a unique constraint, so a retry can't charge, send or publish twice.

Once caching has absorbed what it can and the primary is still busy with reads, add a read replica. Send reads that can tolerate a little lag to the replica, such as list pages, search and dashboards, and keep "I just saved this, show it to me" flows on the primary.

Worked example: measuring p95 before and after (illustrative)

This is an illustrative example, not a client result. Assumptions: a GET /orders endpoint returning 50 orders per page, each with its customer and line items; 1,000 sampled requests at a steady load from a k6 script; same hardware and dataset for both runs.

How to get p95: sort the 1,000 durations from fastest to slowest and take the 950th value. 95% of requests were at least that fast.

Before:

  • Queries per request: 1 (orders) + 50 (customers) + 50 (line items) = 101
  • Median: 310 ms
  • p95: 880 ms
  • Of that p95 time, roughly 70% is database round trips, according to the trace spans

Change: eager-load customers and line items in two batched queries, and select only the columns the response uses.

After:

  • Queries per request: 3
  • Median: 95 ms
  • p95: 190 ms

Reading the result: the change hit both numbers, but p95 fell further because the slowest requests were the ones making the most round trips under contention. If p95 had barely moved while the median improved, the tail would be caused by something else, such as waiting for pool connections or a slow third-party call. You'd go to row 3 or row 5 of the table rather than adding a cache.

Use the same script, load and data every time, or you're comparing two different experiments.

Size limits from real load, then check they fire

Scaling also means setting limits: pool sizes, worker counts, memory caps, autoscaling thresholds. Defaults and guesses fail in both directions.

On one of our own services, the process manager's memory-restart limit was configured, but the monitoring behind it was broken. The app ran at roughly three times the cap and nobody noticed. Once monitoring was fixed, the cap turned out to be far too low and would have restart-looped the site. What we learned: set limits from memory measured under real traffic, including crawler bursts. Keep that configuration in the repo, because a deploy once overwrote a hotfix made directly on the server. And trigger the limit deliberately once to prove it works.

When splitting services is actually justified

You've reached row 7 when all of these are true:

  • One capability (search, media processing, notifications, billing events) has a load or release profile clearly different from the rest.
  • You've already tried isolating it inside the monolith with its own queue, worker pool or replica, and it still drags on everything else.
  • You have the deployment, monitoring and on-call maturity to run another service.

Even then, extract that single capability behind a stable API and leave the rest of the system as it is. A modular monolith with clean boundaries scales a long way. For a second opinion on where your API is actually spending its time, see our backend and API development work. If the bottleneck is in infrastructure or pipelines, DevOps and cloud solutions are the right place to start, and for systems already in production we offer support and maintenance.

Frequently Asked Questions

How do I find the bottleneck in a slow API?

Start with per-endpoint p95 latency, then break slow requests into their parts: database time, connection wait, external calls and serialisation. Whichever part dominates points to the fix. Turning on query logging and counting queries per request finds N+1 problems within minutes.

Should I add Redis caching or fix my database queries first?

Fix the queries first. A cache in front of an inefficient query still pays the full cost on every miss and every invalidation, and it hides the problem until traffic grows again. Add caching once the underlying reads are reasonably efficient and you've defined the freshness rules.

When do I need a read replica?

When the primary database is busy mostly with reads that caching can't absorb, such as search, list pages or dashboards with many filter combinations. Make sure those reads can tolerate a small replication lag. Keep anything a user expects to see immediately after saving on the primary.

Do I need microservices to scale my backend?

Usually not. Stateless app instances, efficient queries, pooling, caching and queues carry most APIs a long way. Extract a service only when one workload has a genuinely different scaling or release profile and you couldn't isolate it inside the existing app.

What is p95 latency and why does it matter more than the average?

p95 is the response time that 95% of requests come in under. It shows the slow tail that averages hide, and that tail is where timeouts, retries and frustrated users come from. Track it per endpoint, not across the whole API.


Work with Esparks IT Solutions

Planning a project around this? We work with businesses in the United States, Canada and the UAE, as well as the UK and Australia. Explore our Programming services and portfolio, estimate your project cost, or book a free call.

Written by

Anika Rukhsar

Anika Rukhsar

Founder

View team

Founder at Esparks IT Solutions. Writes from the team's own delivery work across software, cloud, DevOps and security.

Share this article

Work with us

Have a system to build, fix or scale?

Tell us what you are working on and where it hurts. We will tell you how we would approach it.