A team buys a plan with 500 concurrent connections, points their scraper at it, and watches throughput flatline at around 40 requests per second. The proxy provider says the account is nowhere near its limit. The scraper logs show no blocks, no CAPTCHAs, no 429s. Nothing is technically broken, and yet the pipeline is delivering a fraction of what the spreadsheet promised.
This is one of the most common and least discussed failure modes in proxy-backed automation. Throughput is not something you purchase. It is something your client architecture produces, and the two biggest levers are how you pool connections and how you schedule concurrency. Get those wrong and a premium pool performs like a cheap one. Get them right and modest concurrency limits can saturate your target's tolerance long before you hit the provider's ceiling.
Throughput Is Latency Divided Into Concurrency
Start with the arithmetic, because it explains almost every disappointment.
Sustained request rate is roughly your effective concurrency divided by your average request duration. Thirty concurrent workers against a target that responds in 900 milliseconds yields about 33 requests per second. If you want 100 requests per second at that latency, you need roughly 90 workers in flight at all times, not 90 workers configured somewhere in a YAML file.
That word "effective" is where most of the loss hides. A worker that spends 400 milliseconds on a TCP handshake and TLS negotiation, 900 milliseconds waiting for the origin, and 200 milliseconds blocked on a semaphore is only doing useful work a fraction of the time. Latency through a residential exit is inherently higher than latency through a datacenter IP because there is a real consumer connection in the path. That is not a defect, it is the reason the IP looks legitimate. But it means concurrency has to compensate for latency, and connection reuse has to remove every millisecond that does not need to be spent.
What Connection Pooling Actually Does
A connection pool is a cache of already-established sockets that your HTTP client hands out to requests instead of opening something new each time. Under a proxy, the savings compound.
The handshake tax
An HTTPS request through a forward proxy involves several round trips before a single byte of payload moves: a TCP handshake to the proxy gateway, an HTTP CONNECT to open the tunnel, then a full TLS handshake with the origin server through that tunnel. On a fast local network that is negligible. Through a residential or mobile exit with 150 to 400 milliseconds of round-trip time, it can easily cost more than the request itself.
Reusing a live tunnel for a second request to the same host removes all of it. In practice, teams that switch from connection-per-request to a properly warmed pool often see per-request latency drop by 30 to 60 percent with no change in provider, pool type, or concurrency setting.
How pools are keyed
This is the detail that trips people up. A connection pool is keyed by destination, and under a proxy the key effectively includes the proxy endpoint and the credentials used. Change the session identifier in your username string and you have created a new pool key. The old socket sits idle until it is evicted, and the new request pays the full handshake tax again.
So per-request IP rotation and connection reuse are in direct tension. You cannot rotate the exit IP on every request and also reuse the tunnel, because the tunnel is the thing that pins you to that exit. The resolution is not to pick one, it is to align the unit of rotation with the unit of work.
Sticky sessions as pooling units
Treat each sticky session as a small worker with its own connection. A session that lives for 5 or 10 minutes can serve dozens of requests over one or two warm tunnels, amortising the handshake across all of them. When the session's job is finished (a category crawled, a checkout flow walked, a set of SERPs collected) you retire it and open a new one with a fresh exit.
For workflows where every request genuinely must come from a different IP, accept that you are paying handshake cost per request and budget concurrency accordingly. Do not also configure aggressive per-request rotation on top of long-lived sessions and then wonder why the pool never warms.
Concurrency Models and Where They Break
Thread pools
Simple, well understood, and memory hungry. Each thread carries a stack, and in Python the interpreter lock means you are relying on threads blocking on I/O to get parallelism. Thread pools work well up to a few hundred workers. Beyond that, context switching and memory pressure start eating the gains, and a slow proxy exit can hold a thread hostage for the full socket timeout.
Async event loops
One process, thousands of in-flight requests, cooperative scheduling. This is the right model for high-concurrency proxy work because the cost of an idle-but-open connection is close to zero. The catch is that async clients still enforce their own pool limits, and the defaults are usually conservative. A widely used async HTTP client caps total connections at 100 and per-host connections much lower. If you launch 2,000 tasks against that client, 1,900 of them are queued in your own process, not in flight, and your "2,000 concurrency" is really 100.
Worker processes and distributed queues
Once a single machine is saturated by TLS work or file descriptor limits, horizontal scaling with a shared queue is the honest answer. The tradeoff is that each worker process holds an independent pool, so total open connections against the proxy gateway multiply by worker count. Session affinity also has to move into the queue design: if request B depends on the cookie jar established by request A, they need to land on the same worker with the same exit IP.
Sizing, Timeouts, and Queue Depth
A few rules that hold up in production:
Size the pool to concurrency, not to ambition. Pool capacity should be at least equal to your in-flight worker count, ideally with modest headroom. A pool smaller than your worker count silently serialises work.
Set aggressive but layered timeouts. Separate connect timeout, read timeout, and total request timeout. A 60 second read timeout on a target that normally answers in one second means a single dead exit can occupy a worker for a minute. Five to fifteen seconds is usually right, with retries handling the rest.
Cap the retry budget globally. Uncoordinated retries are how a small degradation becomes a self-inflicted outage. When a target starts throttling, naive retry logic doubles your request volume at exactly the moment you should be backing off. Use exponential backoff with jitter and a circuit breaker per target host.
Watch keep-alive expiry on both sides. Idle tunnels get closed by the gateway, the origin, or a middlebox. Clients that hand out a dead socket produce confusing intermittent errors. Enable connection health checks before reuse, and keep idle timeouts below the shortest upstream keep-alive window.
Do not forget ephemeral ports and file descriptors. A machine churning through short-lived connections can exhaust its ephemeral port range or hit the default descriptor limit. Symptoms look like network failure but are purely local.
Where Proxies Fit In
All of this tuning assumes the network underneath behaves consistently. It often does not, and that is where pool quality moves from a procurement detail to an architectural constraint.
A pool with high churn in its exit nodes will drop your warm tunnels mid-session, forcing renegotiation and inflating tail latency. A pool with thin coverage in the country you actually need will hand you exits that are geographically distant from the target, adding round-trip time you cannot tune away. And a gateway that enforces an undocumented concurrent connection cap will queue your requests invisibly, which looks exactly like a client-side bottleneck.
This is why pool type should be chosen per workload rather than per contract. Datacenter and ISP exits give you low, stable latency and long-lived tunnels, which suits high-volume crawling of tolerant targets. Residential exits cost latency but buy legitimacy. Mobile exits sit behind carrier NAT and are the hardest to ban, at the price of the most variable throughput. Running all three behind one integration lets you route by target sensitivity instead of compromising on a single pool.
EnigmaProxy is built around that kind of routing flexibility: residential, ISP, datacenter, and mobile pools reachable through a consistent gateway, with session control that lets you hold an exit long enough for connection reuse to pay off. Because the residential network is ethically sourced from consenting participants rather than opportunistic device access, exit stability is more predictable, which matters more for pooled architectures than headline pool size ever does. If you are validating latency and exit consistency before committing a workload, running candidate endpoints through a proxy tester will tell you more in ten minutes than a datasheet will.
For teams designing around concurrency budgets, it is also worth reading plan structures carefully. Concurrency limits, session duration ceilings, and bandwidth accounting interact, and rotating residential proxy pools priced by traffic reward connection reuse in a way that per-request pricing does not.
Strategic Shifts Worth Watching
HTTP/2 and HTTP/3 change the pooling calculus. Multiplexing many streams over one connection reduces the value of large pools and increases the value of stable, long-lived tunnels. It also introduces new fingerprinting surface, since stream priority and settings frames vary by client. Expect pool design and fingerprint consistency to become the same conversation.
Adaptive concurrency is replacing fixed limits. Borrowing from congestion control, more scraping frameworks now adjust in-flight request counts based on observed latency and error rates rather than a static number. This is strictly better for both throughput and block avoidance, because it backs off before the target does.
Observability is moving down the stack. Aggregate success rate is no longer sufficient. Teams are instrumenting connect time, TLS time, time to first byte, and pool wait time separately, because those four numbers tell you whether a problem is the provider, the target, or your own client.
Cost per successful record is becoming the operative metric. As bandwidth-metered pricing dominates, wasted handshakes and failed retries show up directly on the invoice. Efficient pooling is now a cost control mechanism, not just a performance one.
Conclusion
Proxy throughput is an architectural outcome. The provider sets the ceiling, but your connection pooling strategy, concurrency model, timeout discipline, and retry budget determine how close you get to it. Align rotation with the unit of work so tunnels stay warm, size pools to actual in-flight concurrency, fail fast instead of waiting, and instrument each phase of the request separately so you can tell client bottlenecks from network ones.
Do that on top of a pool with stable exits and sensible geographic coverage, and modest concurrency will outperform brute force every time. EnigmaProxy is one option worth evaluating on those terms: multiple pool types, business-grade reliability, and the session control that pooled architectures depend on.