A data team pulls 40 million product listings a month across 14 markets. The pipeline is clean, the success rate is high, the proxies rotate properly. Then a European regulator sends a request asking which lawful basis the company relies on for the personal data sitting in those listings: seller names, reviewer handles, review text, sometimes a phone number in a business profile. Nobody on the team can answer, because nobody classified the output. That is the compliance gap most scraping operations carry, and it has almost nothing to do with whether scraping itself is legal.
Public web data collection is legitimate. Courts in the US and Europe have repeatedly declined to treat access to publicly available pages as inherently unlawful. What has changed is that privacy law does not care about access. It cares about processing. The moment your crawler stores a field that can identify a person, GDPR and CCPA obligations attach, regardless of how you obtained it or how many IPs you routed through.
This piece covers what the two regimes actually require of proxy-powered collection, where teams get caught, and how infrastructure choices (including which proxy pool you run on) either support or undermine your compliance story.
Why Public Data Is Still Regulated Data
GDPR Article 4 defines personal data as any information relating to an identifiable natural person. There is no public-domain exemption. A reviewer's display name plus their review history is personal data. A LinkedIn-style profile scraped at scale is personal data. A residential address in a classified ad is personal data. The fact that a human could have read the same page in a browser does not create a processing right.
CCPA and its CPRA amendments work differently. California carves out "publicly available information" from the definition of personal information, but the carve-out is narrower than most teams assume: it covers information lawfully made available from government records and information a consumer has made available to the general public without restricting the audience. Data behind a login, behind terms that prohibit collection, or restricted to a specific audience does not qualify. And even exempt data can lose that status if you combine it with other data to infer something the consumer did not publish.
The practical consequence: classification comes before collection, not after. If you cannot tell a regulator which fields in your dataset are personal data, you do not have a compliance program. You have a crawler.
The GDPR Obligations That Actually Bite
Lawful basis, and why consent is the wrong one
You will never obtain consent from the people whose data appears in scraped pages. That leaves legitimate interests under Article 6(1)(f) as the realistic basis for most commercial collection. Legitimate interests is not a formality. It requires a documented three-part assessment: a genuine business purpose, necessity (could you achieve the purpose with less personal data, or none?), and a balancing test against the rights of the individuals concerned.
The balancing test is where most assessments fail. Aggregated price intelligence that happens to include seller usernames tends to pass. Building person-level profiles from social data for resale rarely does. Write the assessment down, date it, and revisit it when the scope of collection changes.
Transparency without a direct relationship
Article 14 governs data collected from sources other than the individual. It requires you to inform people within a month, unless doing so would involve disproportionate effort, in which case you must publish the information instead. In practice this means a dedicated privacy notice page describing your collection categories, sources, purposes, retention, and rights process. Several EU regulators have fined companies specifically for scraping at scale without any such notice.
Data minimisation as an engineering requirement
Minimisation is the single cheapest compliance control available and the most frequently ignored. If your price monitoring use case does not need reviewer names, strip them at the parsing stage, not at the reporting stage. Drop them before they hit storage. A field that never lands in your database cannot be breached, subject-accessed, or fined.
Retention and deletion that survive an audit
GDPR expects storage limitation and CCPA now requires disclosure of retention periods per category. "We keep everything in cold storage forever" is an answer that ends badly. Set per-field retention, automate expiry, and make sure your deletion path reaches derived tables, backups, caches, and any model training sets built from the raw data.
Cross-border transfers
If EU personal data lands on infrastructure outside the EEA, Chapter V applies: adequacy decisions, standard contractual clauses, and a transfer impact assessment. This is where proxy architecture becomes a genuine legal question rather than a technical one, because your exit node geography and your storage geography are separate decisions and both matter.
What CCPA and CPRA Add for US-Facing Operations
California's regime is disclosure-and-control led rather than basis led. Three obligations dominate for data collectors.
Notice at collection. You must disclose the categories of personal information collected, the purposes, and whether it is sold or shared, including for indirectly collected data. A privacy policy that describes only what you gather from your own website visitors is incomplete if you also ingest third-party data at scale.
Consumer rights fulfilment. Access, deletion, correction, and opt-out of sale or sharing. To honour a deletion request against scraped data you need to be able to find every record relating to that consumer. That requires an identity index, which is precisely what many scraping pipelines lack because records were never keyed to people.
Service provider contracts. If a vendor processes personal information on your behalf, CPRA requires specific contractual terms. Your proxy provider is usually a network transport layer rather than a processor of your collected records, but you should be able to articulate that distinction in writing, and you should know what logs the provider retains.
Add the growing patchwork of other state laws (Virginia, Colorado, Texas, Oregon and more) and the pragmatic answer is to build to the strictest standard and apply it globally. Maintaining fourteen regional variants of a data pipeline is more expensive than maintaining one strict one.
Where Teams Get Caught
Treating anonymisation as a checkbox. Removing a name is pseudonymisation, not anonymisation, if the remaining fields still allow re-identification. Pseudonymised data remains in scope.
Inferred data blindness. Derived attributes (predicted location, purchasing intent, income band) are personal data and often more sensitive than the raw inputs.
Vendor opacity. If you cannot explain where your proxy IPs come from, you cannot answer questions about whether the end users behind them consented to their bandwidth being used. That is a supply-chain integrity issue that sits squarely inside your due diligence obligations.
Logging everything by default. Full request and response logs kept indefinitely for debugging quietly become a large personal data store with no lawful basis and no retention policy.
Ignoring robots.txt and terms while claiming good faith. Neither is privacy law, but both feed the balancing test and the CCPA "restricted audience" analysis. Documented respect for site signals strengthens your position materially.
Where Proxies Fit In
Proxies are not a compliance control, and any vendor implying otherwise should worry you. What proxy infrastructure does provide is the technical basis for several controls that regulators do examine.
Geographic scoping. Data localisation and transfer rules are easier to satisfy when you can decide which country your requests originate from and keep collection for a given market inside a defined region. Broad, granular geo-coverage lets you align crawl geography with the legal footprint of the project instead of routing everything through whatever exit happens to be available.
Session control and rate discipline. Sticky sessions and controlled concurrency let you collect at a pace that does not degrade the target service. Excessive load is a reputational and legal aggravator, and demonstrating restraint supports the necessity argument in a legitimate interests assessment.
Supply-chain integrity. This is the part that belongs in your vendor file. A network built on consented, compensated peers is a defensible input to your pipeline. One assembled from malware-bundled SDKs is not, and its provenance becomes your problem the moment anyone asks. EnigmaProxy operates multiple pool types (residential, ISP, datacenter and mobile) with ethical sourcing as a stated commitment, which is the kind of statement you want in writing from any network you depend on.
Auditability and predictable scaling. Compliance programs need documentation: which pool, which region, which retention, which contract. Running residential proxies alongside datacenter capacity from one accountable provider with business-grade reliability reduces the number of vendor relationships you have to evidence, and clear plan structures make it easier to forecast collection volume without renegotiating your data protection paperwork every quarter.
Use the infrastructure for what it is: reliable, geographically precise, ethically sourced transport. Then build classification, minimisation, retention and rights fulfilment on top of it.
Strategic Insights for 2026 and Beyond
AI training data is the next enforcement frontier. The EU AI Act's transparency obligations intersect with GDPR when training corpora contain personal data. Expect provenance documentation for datasets to become a procurement requirement, not a nice-to-have.
Regulators are moving from access questions to purpose questions. The interesting inquiry is shifting away from "were you allowed to fetch this page" toward "what did you do with the personal data afterwards, and for how long did you keep it". Teams that instrumented purpose and retention early will find these conversations short.
Compliance is becoming a sales asset. Enterprise buyers of data products now ask for lawful basis documentation and provenance during procurement. A clean answer shortens deal cycles; a vague one kills them.
Privacy-by-design is moving into the parser. The most mature pipelines now apply field-level policy at extraction: allow-lists per target, automatic redaction of free-text fields likely to contain personal data, and hashing at ingest. Retrofitting this later is significantly more expensive than building it in.
Conclusion
The legal risk in proxy-powered data collection sits downstream of the crawler. Access is rarely the problem. Undocumented lawful basis, unclassified fields, missing transparency notices, absent retention rules and no path to fulfil a deletion request are the problems, and none of them are solved by better rotation.
Build the boring layers: classify before you collect, minimise at parse time, document your legitimate interests assessment, publish an Article 14 notice, set per-field retention, and keep a real identity index so rights requests are answerable. Then choose infrastructure whose provenance you can explain to a regulator without hedging, which is where a provider like EnigmaProxy positions itself in the professional tier: multiple pool types, wide geo-coverage, and sourcing practices you can put in a vendor file. Compliance and scale are not opposing goals. In 2026 they are the same project.