2 posts tagged with "rate-limit"

You built the frontend on react-query. Its cache and retry keep the browser happy. Then you stood up a Node BFF that calls three downstream services, and the retries and throttling for those calls live on the server. react-query never runs there. This post shows the exact line where react-query's job ends and where alova/server picks up, with the code for each side.

The line​

react-query lives in the browser. Its useQuery retry, its cache, its dedup, all happen per browser tab, bound to React's render cycle. None of that executes inside your BFF when your BFF calls a downstream. So the question isn't "react-query or alova". It's "who handles the server-side calls".

Retry: browser vs BFF​

react-query retries a failed query in the browser:

useQuery({
  queryKey: ['resource', id],
  queryFn: () => fetch(`/api/resource/${id}`).then(r => r.json()),
  retry: 3,
  retryDelay: attempt => Math.min(1000 * 2 ** attempt, 8000)
});

That retry protects the user's view. It does nothing for the BFF's outbound call to the downstream, because that call doesn't go through react-query.

On the BFF, alova/server retries the outbound request with backoff:

const { createAlova } = require('alova');
const { axiosRequestAdapter } = require('@alova/adapter-axios');
const { retry } = require('alova/server');

const alovaInst = createAlova({
  baseURL: 'https://downstream.internal',
  requestAdapter: axiosRequestAdapter()
});

const getResource = (id) =>
  retry(alovaInst.Get(`/resource/${id}`), {
    retry: 5,
    backoff: { delay: 1000, multiplier: 2 }
  });

// inside your route handler:
const data = await getResource(req.params.id);

Same idea, different layer. react-query covers the browser; retry covers the server.

Rate limiting: only on the server​

react-query has no concept of throttling your BFF's calls to a downstream. There's nothing in its API for "max 4 requests per 4 seconds to service X". That is a server-side concern, and alova/server's createRateLimiter is built for it:

const { createRateLimiter } = require('alova/server');

const rateLimit = createRateLimiter({ duration: 4000, points: 4 });

app.get('/api/resource/:id', async (req, res) => {
  try {
    const data = await rateLimit(
      retry(alovaInst.Get(`/resource/${req.params.id}`), {
        retry: 5,
        backoff: { delay: 1000, multiplier: 2 }
      }),
      { key: `downstream:${req.params.id}` }
    );
    res.json(data);
  } catch (err) {
    res.status(429).json({ error: 'downstream busy, retry later' });
  }
});

rateLimit consumes a point before the call leaves, so a struggling downstream isn't buried by your own traffic. The default store is the method's l2Cache; across instances you swap in a shared one (@alova/psc or redis) so the limit is real cluster-wide.

Cache: know which side it lives on​

This is the part people mix up. react-query's cache is per browser. It keeps a user from refetching the same resource in the same tab. It does nothing for your BFF hammering a slow downstream, because that call isn't in the browser.

alova's caching is a client-side strategy feature too. cacheFor controls how long a method response is kept, with modes memory and restore, and non-GET requests default to null (no cache). On the server, alova/server gives you retry and createRateLimiter; it does not hand you a server-side response cache for downstream calls. If you want to skip a slow downstream for repeated GETs, you add your own cache (or lean on the downstream's own), and layer retry/rateLimit on top.

So the split is clean: react-query owns the browser cache, alova/server owns the BFF's outbound retry and throttle. Neither steps on the other.

The two tools solve different layers. Use react-query where the request ends in the browser, and alova/server for the BFF. For the full选型 view, see the react-query vs alova comparison.

• Server retry strategy
• Server rate limit

You run a Node BFF. It sits between your frontend and three downstream services. One afternoon the biggest downstream starts returning 429s, and your BFF answers every one by firing the same request again, immediately. Within a minute the downstream is buried under retries for requests it already rejected, and your frontend sees a wall of 500s. Sound familiar?

Here is the kind of handler that gets you there:

app.get('/api/resource/:id', async (req, res) => {
  try {
    const data = await fetch(`https://downstream.internal/resource/${req.params.id}`);
    res.json(await data.json());
  } catch (err) {
    // naive retry: fire again, right now
    const retry = await fetch(`https://downstream.internal/resource/${req.params.id}`);
    res.json(await retry.json());
  }
});

This looks harmless until it isn't. Two problems hide in it:

1. The retry ignores why it failed. A 429 means "slow down". Retrying instantly is the opposite of slowing down. You amplify the very condition that broke things.
2. No client-side throttle. Your BFF happily forwards every incoming request to a downstream that is already struggling. Nothing caps how hard you push it.

If you run more than one BFF instance, a hand-rolled counter (a module-level variable, an in-memory Map) doesn't help either. Each process keeps its own tally, so your "4 requests per second" limit becomes 4 per instance, per second. On a three-node cluster that is 12.

What a BFF actually needs here​

Step back from any library. A downstream-call layer that survives rate limiting needs:

• Backoff, not an immediate repeat. Wait, then wait longer. Exponential backoff with jitter keeps every instance from retrying in lockstep.
• Retry capped and conditional. Stop after a few tries, and only for failures worth retrying (timeouts, 5xx, 429), not for 4xx that will never succeed.
• A client-side rate limit. Before you even call the downstream, check a shared budget so you don't pile on.
• A counter that survives the cluster. The limit has to live somewhere all instances agree on, not in one process's memory.

That is a fair bit of plumbing if you write it by hand. alova's server side ships two hooks that cover the first three directly, and the fourth through a pluggable store.

retry + createRateLimiter, one middleware​

alova is a request strategy layer. On the server, alova/server gives you hooks that wrap a request method. retry adds backoff, createRateLimiter adds a client-side throttle. Both wrap the same method instance you already use to call the downstream.

Set up an alova instance. On the server I'll use the axios adapter, so the request still goes out through axios. alova only decides how the request runs.

const { createAlova } = require('alova');
const { axiosRequestAdapter } = require('@alova/adapter-axios');
const { retry, createRateLimiter } = require('alova/server');

const alovaInst = createAlova({
  baseURL: 'https://downstream.internal',
  requestAdapter: axiosRequestAdapter()
});

// 4 points per 4-second window, tracked through the method's cache store
const rateLimit = createRateLimiter({ duration: 4000, points: 4 });

Now wrap the downstream call. retry goes inside, rateLimit outside:

app.get('/api/resource/:id', async (req, res) => {
  try {
    const data = await rateLimit(
      retry(alovaInst.Get(`/resource/${req.params.id}`), {
        retry: 5,
        backoff: { delay: 1000, multiplier: 2 }
      }),
      { key: `downstream:${req.params.id}` }
    );
    res.json(data);
  } catch (err) {
    // rate limit hit, or retries exhausted
    res.status(429).json({ error: 'downstream busy, retry later' });
  }
});

What changed versus the naive version:

• retry: 5 caps the attempts. backoff.delay: 1000 with multiplier: 2 spaces them at 1s, 2s, 4s, 8s, so a struggling downstream gets breathing room instead of a second hit.
• rateLimit consumes one point per BFF request before it ever reaches the downstream. Over the window, only 4 get through. The rest throw and you return 429 early, protecting the downstream from your own traffic.
• The key scopes the budget per downstream resource, so one hot key can't starve the others.

The retry defaults matter here too. With no options it retries 3 times, 1 second apart. We only override because this downstream is flaky enough to deserve more.

The cluster part​

The default rate-limit store is the method's l2Cache. In a single process that is memory, which is fine. Across instances you point store at something shared, @alova/psc or a redis adapter, and all nodes draw from one budget. That is the piece a hand-written in-memory counter can never give you without building it yourself.

What you get​

Not a benchmark, just the behavior the config produces:

The last row is request sharing: alova dedupes identical concurrent calls, so two users asking for the same resource don't trigger two downstream hits.

If your BFF calls a downstream that rate-limits and you run more than one instance, retry with createRateLimiter is worth an afternoon.

• Server retry strategy
• Server rate limit