The r/devops version of this question is the cleanest we have seen. "3 worker processes in parallel, 8 browser tabs per worker (24 concurrent pages), Each tab on its own residential proxy. When I run with a single worker, it works fine. But when I run 3 workers in parallel, I start hitting Cloudflare CAPTCHA / verify you're human on most workers." An r/webdev scraper maintainer sees challenges trigger "even with: clean residential ips, realistic user agents, proper tls fingerprinting, randomized delays."
This post is for the person whose IP is fine and who still gets challenged. We have covered what triggers a challenge and why IP reputation is the lever a proxy pulls, and how to test whether a block is the IP at all. This one is about what is left once the IP is clean: how each IP gets used.
Why one worker passes and three get challenged
Three workers fail where one passes because Cloudflare scores each IP and each session on its own behaviour, and three workers change that behaviour even when every tab has a residential IP. If the "residential proxy" per tab comes from a rotating pool with no session pinning, tab one might load the page from IP A, fetch its scripts from IP B, and submit the challenge from IP C. With one worker the pool has slack and the same IP tends to come back. With three, churn goes up and the clearance cookie stops matching the IP that earned it. "Only one or two get through." Exactly.
Four ideas, one honest answer each.
Stagger worker starts: yes, for a narrow reason. Twenty-four pages loading in the same second is a burst, and bursts are visible at the edge without any JavaScript. Staggering removes the burst and nothing else.
Limit tabs per worker: the number that matters is tabs per IP. Eight tabs on eight IPs is eight people. Eight tabs on one IP is one person with eight hands, and a shared pool can hand two tabs the same IP without telling you.
Add delays: only per IP. Random sleeps across the cluster still let one IP fire three requests in a row when three tabs share it.
Change how proxies rotate across workers: this is the one. Pin an IP to a tab for the tab's whole life and never let two workers share a session. The rest of this post is that answer.
What Cloudflare still sees when the IP is clean
Once the IP passes the reputation check, the remaining signals are behavioural and keyed to the IP: how many connections it holds open, how fast requests arrive, whether the clearance cookie comes back from the same IP and user agent, and whether the session lives long enough to look like a person.
An r/automation poster spent weeks on this. "Timing patterns and datacenter IPs are two separate kill switches. I needed to fix both or it's a no go." With "clean TLS fingerprint, realistic Chrome headers, rotating user agents" he was "still eating 403s after ~15 requests like clockwork." Fifteen, every time. That is a per-session counter tripping, not a reputation lookup. He concluded that anti-bot stacks "look much more at behavioral patterns across a session than at any individual request."
The r/webdev maintainer agrees: "the only thing that still works reliably is maintaining long-lived browser sessions with persistent fingerprints." One browser, one IP, one user agent, alive as long as a person would stay.
The cookie is where most setups break. When a visitor passes a challenge, Cloudflare sets a cf_clearance cookie for the browser that passed. Cloudflare calls the window Challenge Passage; the site owner sets it, the documented default is 30 minutes, and the docs leave the binding rules unstated. In practice, and in every thread above, a cookie presented from a different IP or user agent gets you challenged again. Nothing was detected. You threw the pass away.
One browser session means one sticky IP
The working model is one browser context, one sticky IP, one user agent, for the whole life of that context, and as many of those as you have tabs. On our residential pool a sticky session is a session ID appended to the proxy username, and the same ID keeps the same IP for up to 30 minutes. The mechanics are in our rotating versus sticky guide. Applied to a cluster:
Give every tab its own session ID, built from worker and tab number so they cannot collide. Rotate a session only when its browser context closes.
Keep the connection alive; reconnecting for every request looks like a script.
Cap requests per IP at what a person could do. A person opens a page, reads, clicks, waits. A tab pulling forty product pages a minute on one IP is not that.
When the sticky window runs out and the IP changes, treat it as a new visitor: new context, new cookie jar. Do not carry the old cf_clearance across.
Rotating residential or a fixed set of ISP IPs
Pick by the shape of the work: many sites with a few pages each wants a rotating residential pool, a few sites with many pages each wants a small fixed set of ISP IPs you keep for months. An r/webscraping poster asking whether "250 proxies and 5TB of BW" were "shared proxies or dedicated? Residential?" was asking this.
A dedicated ISP IP is one address on a consumer ISP range that nobody else uses, held for the whole rental. Cloudflare sees a visitor who comes back. For ten sites you crawl daily, a fixed set with one browser profile per IP is the calmest setup we know. For thousands of domains it is the wrong tool; no site sees enough of you to build a profile, and per-GB residential with a sticky session per context is cheaper to run. Tiers for both are on the pricing page.
When the IP was never the problem
A challenge on the first request, before any behaviour exists, is almost always the client identifying itself, and no IP changes that.
The r/node poster with Express and Axios is the textbook case: "No Puppeteer, Selenium, or headless browsers, just direct API requests." He "also tried using residential proxies, but still getting blocked, 403 Forbidden." A plain HTTP client is recognised by its TLS handshake and HTTP/2 settings before the IP is consulted. Node's TLS stack does not look like Chrome's, whatever the User-Agent header says. Swapping IPs behind it changes nothing. His second error, "The plain HTTP request was sent to HTTPS port", has nothing to do with Cloudflare: a web server got an unencrypted request on an encrypted port, usually a wrong proxy URL or target scheme.
The r/automation poster on a DigitalOcean VPS ("Locally, everything works perfectly ... the moment I run the same Playwright script on the DigitalOcean server, Cloudflare antibot blocks me instantly") is hitting the hosting ASN. Instant, not after fifteen requests. That is the reputation layer, tested in the scraper post. The r/AI_Agents thread on the "Fingerprint Trap" is right, too: a browser driven over the DevTools protocol leaves marks no proxy covers.
The personal project that cannot afford a pool
For a small personal scraper, a rotating home IP is not the cheap fix it looks like, because a first-page block from a household address is not an IP block. An r/node poster building a price comparison for Turkish cosmetics shops wrote that "sites like Rossmann instantly blacklist my IP on the first or second page" and that "buying a $100/mo residential proxy pool is currently out of my budget." His ideas: a hotspot with scripted airplane-mode toggling, and a scripted router reboot.
Both give a new address. Neither gives a new client. He runs Puppeteer and "direct API fetches", and a site that bans on page one is reacting to the request, not to a history the IP does not have yet. A new IP gets exactly as far. A phone's IP is also a carrier NAT address shared with many subscribers, and plenty of home routers get the same address back after a reboot.
What fits a personal project is smaller than a pool: a handful of ISP IPs, one shop and one browser profile each, at a shopper's pace, or a small residential plan with sticky sessions sized to a few thousand pages a month. The Cloudflare landing page answers the shorter questions.
What the shop owner on the other side sees
Cloudflare weights behaviour over IP because residential addresses are cheap to obtain and hard to blocklist, and a shop owner on r/dropshipping is living the consequence. Their site shows "around 300 active users almost all the time", "almost all of them appear to come from Singapore". They ask whether these are "sophisticated bots using residential proxies" or "GA4 Measurement Protocol spam".
First check whether those 300 sessions exist in Cloudflare's own analytics or the server logs. If not, the hits go straight to Google Analytics through the Measurement Protocol and never touch the site; no WAF rule can stop them, and the fix is a GA filter and a rotated measurement secret. If they do appear at the edge, rate-limit per IP and ASN, and put a managed challenge on the country instead of a plain block. If they pass the challenge and keep coming, that is a residential IP behaving like a person, seen from the inside.
FAQ
Why does Cloudflare show a CAPTCHA when I run multiple workers with residential proxies?
Because the extra workers changed the behaviour on each IP, not because the IPs went bad. When several workers pull from one rotating pool, the IP that passed the challenge is often not the IP that sends the next request, so the clearance cookie is invalid. Pin one sticky IP per browser context and stagger the starts.
Is the cf_clearance cookie tied to my IP address?
In practice, yes. Cloudflare's documentation gives cf_clearance a default lifetime of 30 minutes and does not spell out what it is bound to, but a cookie presented from a new IP or a different user agent is treated as no cookie and you are challenged again. Keep the IP and browser fixed for the cookie's whole life.
Does rotating residential proxies help against Cloudflare?
Only for one-page visits. For any flow that spans several requests on one site, the IP has to stay fixed for the whole browser session or each change costs a challenge.
How many requests per minute can one residential IP make on a Cloudflare site?
There is no published limit, and anyone who gives you a number made it up. A safe rule is what one person could do in a browser: one page, a pause, the next.
Residential or ISP proxies against Cloudflare and Google verification?
Both pass the ASN reputation check; the difference is continuity. A dedicated ISP IP is the same address for months, so cookies and logins stay tied to one visitor, which is what Google account checks and Cloudflare sessions reward. One site over time, ISP. Many sites briefly, residential.
Why does my Playwright script work locally but get blocked on a DigitalOcean VPS?
Because the VPS exits from a hosting ASN and your laptop from a consumer ISP. Cloudflare challenges the hosting range before your script has done anything. Route the VPS through a residential or ISP proxy and the first request passes; from the second one, the rules above apply.