There’s a version of this story that fits in a tweet. Scrape job ads, let an AI rank them, email the winners. Lead generation, solved. I believed some of that going in. Then I built the thing, and the interesting part turned out to be everywhere the AI wasn’t.
So here’s the long version. How an outbound prospecting agent actually gets built, end to end, with the parts that broke and the parts that cost money. If you’re trying to automate B2B prospecting yourself, this is the stuff I wish someone had written down before I started.
Most “lead generation” tools optimise for volume. More contacts, more rows, bigger database. That’s the opposite of what a salesperson needs on a Monday morning.
What they need is a short list. The right companies, with a real reason to talk this week, and a named person with an email and ideally a phone number. Twenty good rows beat two thousand cold ones, because nobody works two thousand rows. They get ignored, and the whole system quietly dies.
So every decision below serves one goal - make the list shorter and more correct, not longer. That sounds obvious. It’s the thing almost nobody actually does, because adding rows is easy and removing the wrong ones is hard.
The trigger for the whole thing is hiring. A company that suddenly posts more job ads is usually building a team, reorganising, or under some kind of pressure. All three are reasons to reach out. It’s a public, free, weekly signal, and most people ignore it.
Two job boards get scraped every Sunday evening. Each run lands in one BigQuery table with the scrape date stamped on the row. That’s it - no fancy storage, one table, one date column. The date stamp is the whole trick, because the signal isn’t “this company has 40 openings”. The signal is “this company had 12 last week and 40 now”.
Running next to the scraper is a second, manual source. I drop screenshots of LinkedIn posts and news into a Google Drive folder - a funding round, a new plant, a “we’re hiring 50 people” post. A scenario in Make watches that folder, sends each image to Claude, and gets back two fields: the company name and the signal type, hiring or news. That writes to its own table, deduped by file name so the same screenshot can’t sneak in twice.
This is the only place the AI does real work in the entire pipeline, and it’s reading screenshots. Hold that thought.
Once you have two weekly snapshots, you diff them. For every company you compute three things: absolute growth in open roles, percentage growth, and whether the company is brand new this week - present now, absent last week.
The percentage list gets a floor. A company has to have at least five current openings to qualify, otherwise a jump from one ad to three shows up as “+200% growth” and clutters the top of the list with noise. Small guardrail, big difference.
Out of tens of thousands of scraped rows, this leaves a few hundred companies that genuinely moved. From there I cut three lists - biggest absolute growth, biggest percentage growth, and the brand-new ones - and keep the top twenty or so of each. The screenshot signals form a fourth list, and that one isn’t capped, because those are deliberate picks I made by hand, not volume to be trimmed.
Four short lists. Now the hard part.
Here’s the thing nobody shows you in the demo. The moment you try to use that list, you find out half of it is useless to your sales team. Current clients. Former clients. Companies someone emailed in March. A brand that turns out to be owned by an account you signed last year.
If you send that list as is, you’ve made things worse. You’ve got salespeople cold-pitching their own accounts. So before anything reaches a list, every candidate gets checked against the CRM.
I pull a full company export from the CRM - around two thousand accounts, with tax ID, last activity date, client status, and crucially the sales tasks planned against each one. Then I drop anyone who’s a current or former client, was active in the last ninety days, or already has a task on them. The task signal matters more than people expect. If a rep has a call scheduled for next week, that company is being worked, and it should not show up as a cold lead. Full stop.
One nuance I’m proud of - an opportunity that’s open but untouched for over ninety days gets let back in. A stalled deal nobody has moved in three months is fair game for a fresh angle. Blocking it forever just because it once existed in the CRM is how good leads go to die.
A side note on why I export the company list manually instead of pulling it through the API. The CRM’s “list all companies” endpoint times out against the automation platform’s 40-second execution limit. The task-read endpoint returns “not privileged” for the API user. Rather than fight permissions and timeouts for a week, I export the list from the UI and load it. Boring, reliable, done. Sometimes the right engineering decision is the unglamorous one.
This is where I lost the most time, and where most homemade prospecting systems quietly fail.
The job ads say “Orlen Paczka”. The CRM holds the full legal name with sp. z o.o. on the end. Same company. An exact string match catches none of it.
So you build a normaliser. It strips the legal suffixes - sp. z o.o., S.A., the full “spółka z ograniczoną odpowiedzialnością” written out, all the misspelled variants people type. It folds Polish characters down to plain ASCII, so ł becomes l and ą becomes a, because the two databases never agree on diacritics. Then it matches on prefixes, so “otto work force” lines up with “otto work force europe”. There’s even a rule for holding-group prefixes like “grupa” and “gk”, so “Grupa LUX MED” resolves to the parent.
And then there are the cases no algorithm will ever catch. Biedronka is owned by Jeronimo Martins. Hebe is too. No amount of string cleaning makes “Biedronka” look like “Jeronimo Martins”. For those you keep a hand-written alias table and you add to it when you spot a leak. It’s manual, it’s a bit embarrassing, and it’s the only thing that works until you have a hard key like a tax ID on both sides.
All of this lives as SQL and CSV files in a git repo. Not clicked into a tool’s UI. That decision matters more than it sounds. When a client asks why a company got contacted twice, I can read the diff and tell them exactly which rule let it through and when it changed. You cannot do that with logic buried in a no-code canvas.
A company name is not a lead. You can’t email “Orlen Paczka”. So the enrichment step turns names into people.
First, name to domain. Apollo’s company search takes the cleaned-up name and returns a website. Where I get a domain - and that’s a big “where”, more on it in a second - I pull HR and management contacts at that domain, capped at three people per company, deduped so the same person can’t appear twice. Then I resolve emails one at a time.
Phones are their own problem. Apollo finds emails just fine. Polish mobile numbers - basically nothing. So verified mobiles come from a second tool, Prospeo, queried against the person’s LinkedIn URL, set to return only verified mobile numbers. The nice part of that setup is it only bills when it actually finds a number. No number, no charge.
Now the uncomfortable truth. Apollo returns a usable domain for maybe one company in three. So two thirds of a clean, well-filtered list never make it to the contact stage, purely because the name-to-domain step missed. That’s the current ceiling of the whole system. It’s not the scraping, it’s not the AI, it’s a boring matching gap. That’s the next thing I’m fixing, and I haven’t cracked it yet.
Enrichment is the only part that costs real money, so it’s the only part you have to be ruthless about.
The free calls - searching for a company, searching for people at a domain - cost nothing. The expensive ones are the email match and the phone lookup, billed per result. So you cap people per company at three, you dedupe hard by domain, and you run the whole thing once a week, not on every change. That math is the difference between a few hundred credits a month and burning through a paid plan in a week.
The expensive way to learn this is the way I learned it. An earlier setup let names into enrichment before the CRM filter ran. The result was Apollo cheerfully enriching our own clients, over and over, plus a pile of duplicates. The fix wasn’t a bigger plan. It was moving the filter in front of the spend. Filter first, enrich second, always.
The boring summary. Scrapers fire Sunday evening so the data is fresh. The loaders push both boards into BigQuery a few hours later. Monday morning the rest runs - diff, filter, the three lists plus the screenshot list, enrichment, phones - and it all lands in one Google Sheet shaped exactly like the CRM’s import format. Company, person, role, email, phone, LinkedIn, source, owner, and the reason that row was picked.
A salesperson opens that sheet before their coffee’s cold and starts writing. No research. No cleanup. That’s the whole point.
The scraping and the AI are the easy ten percent. They work on day one and they make a great screenshot. If that’s all you build, you’ve built a toy.
The real work is three unglamorous things. The filter that keeps you from pitching your own clients. The name matching that makes the filter actually work across two databases that never agree. And the credit discipline that stops enrichment from quietly draining a budget. Get those right and the AI part almost doesn’t matter - mine reads screenshots and that’s about it.
Build that core in version control, not in a SaaS UI you can’t diff. The day someone asks why a lead got contacted twice, you’ll want to read the answer, not guess at it.
It took longer than I’d like to admit. It also runs itself now, every week, while I do something else. That trade was worth it.
