The Amazon scraping threads all start the same way. Something worked, then it stopped, and the stop did not look like a block. An r/learnprogramming poster scraping the Brazilian store: "The scraper starts out fine and the pages load fully for a while. However, Amazon eventually figures out I am scraping it and, instead of blocking me, the source code it sends back as a response no longer contains the two sections" (product description and product information). On r/scrapy someone is "still getting hit with those 'Verify you are human' captchas every five minutes."
This post is for the person building a price tracker, a review monitor or a deal alert, who wants to know what Amazon does when it notices, what the IP changes about that, and when the honest answer is an API instead of IPs. The general questions (is it the IP, what a proxy does about captchas) are in scrapers that stop working and captchas and JS challenges. This one is only about Amazon.
Amazon does not block you, it moves you down a ladder
Amazon's response to a scraper is a sequence of degraded pages, and the first rung looks like success. Amazon publishes none of this, so what follows is the order people report in these threads, not a spec.
Rung one is the full page. Everything is there and your parser is happy.
Rung two is the page with holes. Status 200, title and price present, but sections Amazon fetches separately (the description, the product information table, sometimes reviews) are missing. That is the Brazil poster's symptom. Those sections are lazy-loaded by secondary requests, served to trusted sessions and quietly left out for the rest. Nothing in the response says so. Your scraper keeps writing rows with empty fields, the r/webdev poster's "bad data" with "no clear errors".
Rung three is the robot check, the "Verify you are human" page. Status 200 again, which is the cruel part.
Rung four is the dog page and 503. Minutes, sometimes the rest of the day for that address.
Which rung you start on is mostly the IP's range. How fast you descend is mostly session and rate. That split is the whole post.
Detecting each rung in code
A 200 is not a success on Amazon, so check the rungs explicitly:
def classify(resp, html):
if resp.status_code == 503:
return "blocked" # rung four
if "validateCaptcha" in html or "[email protected]" in html:
return "captcha" # rung three
if 'id="productTitle"' in html and 'id="productDescription"' not in html:
return "partial" # rung two
return "full"
The ids differ by store and change over time, so fetch one page by hand, note the section ids of the full version, and diff against that. "Partial" means slow that session down, not store the row.
What the IP range decides
The IP's range decides which rung you start on and how much patience Amazon has before the next one; it does not decide whether the ladder exists. An r/scrapy poster asked for proxies because "I've been banned many times using datacenter proxies, and residential ones seem pretty expensive." No shopper lives on a hosting company's range, so the captcha arrives early and the 503 follows. A consumer-ISP range starts higher up because it looks like a household. That is the entire advantage. Patience, not immunity.
An r/Python poster's "rotating datacenter proxies doesn't cut it anymore. Even residential proxies sometimes fail" is the proof. A residential address sending a hundred pages a minute with no cookies descends the ladder like any other, just from a higher start. Range is one input. Pace, session and headers are the others, and those are free.
How many requests one IP can make on Amazon
There is no published per-IP limit for Amazon, and anyone who quotes one made it up. The r/node poster did the maths anyway: "If I had a pool with 10-15 IPs and I have something like 85,000 requests to amazon daily which is roughly one request per second and divided per 10 IPs let's say, that's 8500 request per day per IP. Is that high enough to get the IP banned? knowing that they are probably not residential IPs."
Yes. 8,500 product pages a day from one address is nothing like a shopper. A heavy Amazon user opens a few dozen product pages in a day, in bursts with gaps, not one every ten seconds around the clock. Amazon needs no hard limit to see that. On non-residential ranges the captcha arrives long before the count matters.
The fix is boring. More addresses, fewer pages per address, a pace per address a person could plausibly keep. Sticky residential sessions each carry their own cookies; fixed ISP IPs each get a slice of the catalogue and a schedule. Either way, minimise pages per address per hour, not pool size.
The number the poster did not do is bandwidth. At 85,000 pages a day on per-GB residential, page weight times requests is the bill, and Amazon product pages are heavy if the client pulls images, scripts and fonts. Block everything that is not the HTML document, weigh one page, multiply, and put the result into the proxy calculator before you buy. That sum often decides between residential per GB and ISP IPs with no meter.
Why residential feels slow and what to do about it
Residential proxies are slower than a direct connection because the packets cross an extra hop and someone's home uplink, and cheap rotating pools make it worse by giving you a new address, and a new handshake, on every request. An r/scrapy poster measured it: "Requests without the proxy took an average of 1.5s and now with the proxy they take around 6s - 10s" on the cheapest rotating pool they could find.
What to do:
- Use sticky sessions. Keep one exit for a few minutes, reuse the connection, keep its cookies. Faster, and more like a shopper. The rotating versus sticky guide covers how long to hold one.
- Trade latency for concurrency. A slow request does not matter if twenty are in flight.
- Block non-HTML resources. Images and scripts are most of the bytes and none of the data.
- For a monitor that must fire within a minute of a price drop, use ISP proxies: fixed addresses on consumer-ISP ranges, in datacenters, short round trip, connection open all day.
ISP, residential, or an API, at three volumes
For a few hundred products checked a few times a day, a small fixed set of ISP IPs with a plain HTTP client is enough; at tens of thousands of pages a day the proxy bill is the design constraint and a managed API may be the cheaper spend. We sell IPs. That is still the honest ranking.
About 300 products, a few checks a day
The r/webdev deal alert "tracks price, review count and stock for about 300 products." That is a few thousand requests a day at most. Ten fixed ISP IPs, thirty products each, one page every few minutes, is a shopper's pace per address. A managed scraping API is also sane here. Either is fine. The Playwright fleet the poster actually had is the most fragile of the three at this size.
And you do not need the browser. Amazon renders title, price, rating and review count on the server for most ASINs, so a plain HTTP client sending a current browser's full header set, with a matching TLS fingerprint, gets those fields. The r/Python poster who "added headless Chrome rendering but it slowed everything down" paid for rendering the price never needed; the r/learnprogramming tracker went requests, Selenium, Playwright, and got no more reliable. The block was never about JavaScript. A browser only earns its cost on rung two.
A few thousand pages a day
Still ISP territory, with more addresses and a scheduler that spreads the catalogue and backs off any session that hits rung two. Residential sticky sessions work too, at the cost of a meter and the latency above. The residential versus ISP post has the general comparison.
85,000 pages a day
Now the arithmetic runs the project. Per-GB residential at that volume is a real monthly line, and a large ISP block needs scheduling and health logic to keep every address at a plausible pace. A managed API that charges per successful page is often the calmer spend here; the r/scrapy poster who "tried a managed API approach" said it had "been saving me a ton of sleep". If you build it anyway: many addresses, few pages each, sticky cookies, HTML only, and a rung-two detector.
The multiple-seller-account question
One r/dropshipping poster asked "How can I operate multiple Amazon stores without getting blocked?" and whether the answer is proxies. Running more than one seller account needs Amazon's permission, and an IP address does not change the identity documents, bank account and tax data Amazon links accounts on. We do not sell IPs for that and will not advise on it.
FAQ
Does Amazon block residential proxies?
Not by range, but residential addresses get the same captcha and 503 when the session behind them behaves like a scraper. A consumer-ISP range starts higher on Amazon's ladder than a datacenter range; it does not remove the ladder. Pace, cookies and headers decide how long the address stays useful.
How many requests per day can one IP make on Amazon?
Amazon publishes no limit, and any number someone quotes is a guess. The r/node figure of 8,500 requests a day from one address is far beyond what a shopper does and draws the captcha early on any range. Keep each address at a pace a person could plausibly hold.
Why does Amazon return the page without the product description?
Because Amazon fetches the description and product information sections with separate requests and serves them only to sessions it trusts. A suspicious session gets a 200 page with those sections missing, a soft block one rung before the captcha. Check for the section ids and slow that session down.
Why is scraping Amazon through residential proxies so slow?
The exit is a household connection, the pool adds a hop, and per-request rotation opens a new connection and TLS handshake through a new exit every time. Sticky sessions, higher concurrency and blocking images and scripts recover most of it. For low latency, ISP proxies fit better.
Do I need a headless browser to scrape Amazon prices?
No. Price, title, rating and review count are in the server-rendered HTML for most ASINs, and a plain HTTP client with real browser headers gets them. A browser only helps for the lazy-loaded sections.
Is it legal to scrape Amazon?
Product pages are public, Amazon's conditions of use forbid automated access, and court decisions on scraping public data differ by country and case. We are not lawyers and this is not legal advice. Stay away from data behind a login and personal data in reviews, and if you plan to sell the result, ask a lawyer.
ISP or residential proxies for Amazon price monitoring?
ISP proxies for a fixed catalogue checked on a schedule: fixed addresses, no data meter, short round trips, cookies that persist for months. Residential when you need far more addresses than you want to rent, or countries where we have no ISP stock, at the cost of a per-GB bill and slower requests.