Proxy feeds: from a public list to a usable address
The Hub fetches public lists of free proxies on a schedule, adds only new addresses and checks each one just like a purchased one.
It remembers rejected addresses and never spends checks on them again. The page answers one question: is a source worth keeping.
01 / The feed funnel
Public lists of free proxies are huge but almost entirely dead. The proxy feed funnel shows at which step the stream gets filtered out: from a line in a list to an address fit for mail. It shows how much real value all your sources bring over the period.
Each step shows how many were filtered out: out of hundreds of thousands of lines, only a handful are fit for mail
The other addresses were already in the fleet or had been rejected earlier. A rejected address leaves a fingerprint: the next collection won’t add it again or waste a check on it.
A new address goes through the same acceptance check as a purchased one: reachability, handshake, external address, port 25 — and only then gets a grade. Collected addresses never go into sending by themselves.
02 / The risks of free proxies
The whole email passes through a free proxy: recipient addresses, text, links. Its owner can read and alter the content — and can strip the encryption between mail servers along the way. The page warns about this in its very first line.
Free addresses are used for list cleaning and checks, not for real mailings from your working domains.
A grade D address — with port 25 closed — is barred from mail entirely, wherever it came from.
03 / Sources
For each proxy feed you see how many addresses it gave, how many of them were new, how many have been fit over all time and how many are working now. A bar shows the share of fit addresses, those waiting for a check and rejected ones.
Last collection — 2,303 lines, 11 of them new. Fit over all time: 4 — under 0.1%.
301 lines, 2 new, 2 fit over all time — under 0.1%. Addresses are read from the table’s links.
The site is now behind a browser check. The row suggests what to do: download the list by hand and upload it as a file.
The last collection gave 1,091 lines and 5 new ones, none of them fit.
The source bars are almost entirely red — that is what the truth about free lists looks like. 13,684 dead addresses have already been removed.
04 / Parsing lists
The Hub reads text files, tables with links and pages where the port is printed by a script. The method is chosen on the first collection and remembered, and if it stops finding addresses, it is chosen again.
Anything that looks like an address with a port — in a text file, a table or tags.
Some sites keep the address and port in a link to a card — the protocol is taken from there too.
The port is printed by an expression on the page; the Hub works it out without a browser, but this breaks when the site changes.
List sites block robots, so the Hub presents itself as a browser — otherwise a source looks dead even though it works. Failures are named in plain words: browser check, site didn’t respond, stopped giving addresses.
05 / The interface in detail
All screens of the proxy feeds section: the warning, the funnel, the list of sources, adding a source, parsing methods and the collection history. Above each screenshot: why the screen matters. Below it: what it shows.
The page opens with a warning: a free proxy is someone else’s machine, and the whole email passes through it. Below — the funnel for the chosen period: how many addresses all sources found, and how many of them were new, checked, rejected and fit. This is the main answer to whether free lists make any sense.
The top of the proxy feeds page: the module tabs and the period — 24 hours, 7 or 30 days; the funnel below depends on the period. Under them is the page’s main warning, worth reading before you set anything up. The whole email passes through a free proxy — recipient addresses, text, links — and its owner can read and alter the content. The conclusion is simple: free addresses are fit for list cleaning and checks, but not for real mailings from your working domains. A grade D address with port 25 closed is barred from mail entirely, wherever it came from. Below — the funnel for the chosen period.
EnlargeA grade D address with port 25 closed is barred from mail entirely.
The proxy feed funnel over 7 days — five numbers from a line in a list to a fit address. On the right of each step is its share of the previous one, so you see where the stream gets filtered out. Out of 416,251 addresses found, only 2,282 are new, almost all of them failed the check, and 7 fit ones are left. The screen honestly shows the price of free lists: lots of lines, a handful of usable addresses, and even those don’t go into sending by themselves. If a source gives no fit addresses for weeks, you can disable it. Rejected addresses are remembered by fingerprint, so the next collection won’t add them again or spend checks on them.
EnlargeA fit address from a list is not in sending yet: collected addresses don’t go into mailings by themselves.
Next — the proxy feeds themselves: each with its state, parsing method, schedule and the haul of the last collection and of all time. The bar of fit addresses shows which source is worth keeping and which can be disabled. Here too is the form for adding a new source by link.
The list of connected proxy feeds — this is where you decide which source to keep. There are four, and for each you see the state, protocol, parsing method, schedule, link and the haul of the last collection. A colored bar shows the share of fit, waiting and rejected addresses, and for free lists it is almost entirely red. On the right — the “Collect”, “Disable” and “Remove” buttons, at the top — “Collect from all enabled”; if a collection broke, the row suggests what to do. Collected addresses don’t go into sending by themselves, and rejected free ones are removed right after the check; 13,684 have already been removed.
EnlargeCollected addresses don’t go into sending by themselves. Rejected free addresses are removed right after the check, the rest by the nightly cleanup; 13,684 have already been removed.
The form for adding a new proxy feed. It needs four fields: a link to the list page or file, a name, a protocol and a collection schedule — hourly, daily or manual. The “Check and add” button downloads the page right away and won’t save a source if there are no addresses on it, so an empty link never gets into the list, and the protocol can be detected from the page or set by hand. The Hub itself turns a link to a GitHub page into a link to the “raw” file. The Hub picks how to read the page on its own at the first collection — that is the next screen.
EnlargeThe Hub itself turns a link to a GitHub page into a link to the “raw” file — otherwise it would find zero addresses there.
At the bottom — how the Hub reads pages with lists and what happened during the latest collections. The Hub picks the parsing method itself, so almost any list will do. If a collection broke, the history row names the reason in plain words.
A reference block on how the Hub reads pages with proxy lists — useful if a source suddenly stops giving addresses. There are three methods: addresses in the page text, the address and port in table links, and a port printed by a script. The method is chosen automatically on the first collection and remembered, and if it stops finding addresses, it is chosen again. At the bottom it explains why the Hub presents itself as a browser: some sites refuse robots. Failures are named in plain words: browser check, site didn’t respond, stopped giving addresses. Such rows show up in the collection history below.
EnlargeOne of the tested sites answers 406 to a request without browser headers — without the disguise the source would look dead.
The proxy feed history: the last twelve collections, each in its own row — when, which source, how it ended, how many lines it gave and how many of them were new. The color of the stripe on the left is the collection result, so a broken source stands out at once. For example, “Card table” at 16:20 gave 301 lines but only 2 of them were new, while “Ports in a script” at 13:50 ran into a browser check. At the bottom it explains why a rejected address won’t come back with the next collection. If a source keeps running into a browser check, download the list by hand and upload it as a file on the import screen.
EnlargeA rejected address is remembered: the next collection of the same list won’t add it again — otherwise the same junk would keep coming back every hour.
06 / Questions about sources
In short: what free proxies are for, why so few are fit, and what to do if a source breaks.
EnlargeThe state, parsing method and schedule of each source, its haul, and a hint if a collection broke.
Better not. The whole email passes through a free proxy, and its owner can read and alter it. Such addresses are used for list cleaning and checks.
Most lines are addresses that are already in the fleet or were rejected earlier. The Hub adds only new ones.
No. It leaves a fingerprint, and collection from sources won’t add it again. A manual import will add such an address and check it again.
No. A new address goes through the same acceptance check as a purchased one and gets a grade, and it is added to a pool separately — with the pool’s “Add proxy” button.
Download the list by hand and upload it as a file on the import screen — the source row suggests this itself.
Hourly, daily or manually — this is set for each source.
Yes. The Hub itself turns a page link into a link to the “raw” file with the list.
By the funnel and the number of fit addresses over all time. If a source gives no fit addresses for weeks, you can turn it off with the “Disable” button in its row.
07 / Next step
Collections, checks, rule triggers and people’s actions — everything that happened to your proxies is recorded in the module log.
How each collected address is checked is shown on the “Checks” page, and the whole route — from the MTA through the gate to the recipient — in the module overview.
