Project 03 / Case study
Automation and local AI · three independent repositories
Three Python tools that hand off to each other and share no code. One browses the web guided by a model running on my machine and hands the mouse to a human when a CAPTCHA appears. Another turns public profiles into leads in two different sectors, each with its own rejection rules. The third covers Spain with a grid built on the official Nomenclátor, spending less than the provider allows. All three feed the CRM from the previous case.
Winning new customers is the job nobody wants to do: search for businesses, open another tab to see whether they have a presence of their own, copy the email into a list, repeat. By the end of the day you have forty rows and the feeling of a lost day. And when someone automates this, they do it with a script that works until the page changes its layout.
I separated what breaks from what does not. Every tool has a core with its logic and a start-up layer that only presents options. When a page changes, one adapter is touched; storage and classification do not move. And the rule for what counts as a customer is written down explicitly, including the reasons why something is not one.
That the right question is not “do I use AI?” but “which decision am I delegating?”. In these three tools the model decides two small things that are very expensive to do by hand. The rest is boring code, and boring code is the kind that still runs unsupervised six months later.
Architecture
None of them depends on the others. Each has its own double-click launcher, its own database and its own exports, and all of them end up in the CRM. If one stops paying off, you switch off only that one.
The first is the big one: thirteen thousand three hundred and thirty-three lines that open Google or Bing directly in Chrome to generate candidates and then navigate with Playwright on a persistent profile. The model is local: Ollama for its compatible chat endpoint, with qwen2.5:7b. Zero paid keys, zero data leaving the machine, zero marginal cost per query.
When the model gets stuck, a human takes over. The program detects a consent screen, a CAPTCHA, a log-in prompt, a 429 error or a hard block, and in that case hands the Chrome window to the user; it checks the page again when you press ENTER, and accepts an S to skip that search. It does not try to brute-force what only a person can solve. The proxy is a single one and it is mine: no rotating addresses to get around blocks. If a site says no, it stops.
Contact searches, the session check and the profile extractor share the same Chrome profile, so the log-in is reused. Native identity of the browser that is installed, Spanish locale, Madrid time zone and six seconds of pause between searches. Storing the cookies avoids losing sessions and cuts down on needlessly fast requests.
The second is five thousand six hundred and sixty-seven lines that walk through public profiles. The interesting part is not the crawl: it is that it is designed so the sector is a parameter, rather than rewriting the program. And the third, one thousand nine hundred and six lines without a single model, is the best engineered of the three.
Criteria
Any searcher returns things that look right. The work lies in writing the rejection rules, because those are what stop a human from going through four thousand records by hand.
The profile tool ships two isolated profiles, Tatuadores and Motoclubs de España, each with its own file of search terms, its own database and its own exports. The recommended parameters live in one .bat file per profile. Moving from the tattoo market to another market was configuration, not programming. And on the command line, the profile flag always comes before the command so there is no ambiguity about which data you are working with.
In the biker profile the model requires one club signal (motoclub, motorcycle club, peña or motorcycle association, MC) and one signal of a location in Spain. An individual rider talking about bikes, a shop, a workshop, a dealership or an event do not count as a positive however much they talk about bikes. It is easier to write a list of words than a list of rejections; the second one is what saves real work.
When I refine the criteria, I do not start from zero: the tool has an option to run the data already collected back through the model and clean it, another to preview what is going to be deleted and take a backup beforehand, and a strict mode that by default preserves the email that already existed. I ruled out delete and re-collect: every record cost machine time, and a fresh collection never returns exactly the same thing.
The cleaning rules only remove what carries a category explicitly outside the sector. If an ambiguous person leaves no way to decide, they stay. In the same way, the personal email of a profile is removed from the saved contacts, and the campaign export rejects invalid domains or mailboxes that do not accept communication. A false positive costs one send; a false negative costs one opportunity. They are different costs and they have to be chosen on purpose.
The fine piece
One thousand nine hundred and six lines, six tests and the best idea in the portfolio. There is no model here: there is geometry and a budget.
The query never changes; what changes is the rectangle. Appending the city to the text (“tatuajes Madrid”) brings in the search engine's bias and makes the result depend on a phrase. Here every cell sends the same query, tatuajes, and each search is restricted with locationRestriction.rectangle to the exact perimeter of its cell. That way the coverage is complete and comparable: if a cell comes back empty, it means there are no businesses there, not that my phrase was badly written.
And this is tested. The end-to-end test intercepts the request, checks that the text sent is exactly tatuajes and that the rectangle corresponds to the cell, and then makes the assertion that matters: if I remove the pagination token, the body of the second page is identical to the first. I do not expect it to be a respected convention: it is an equality that the test breaks if anyone breaks it.
The grid is not a uniform chessboard. It is built from the Nomenclátor Geográfico de Municipios y Entidades de Población del CNIG: the national ZIP is downloaded, the program locates only the entities file (it also accepts the municipalities file or a folder, and you can drag the ZIP onto the window) and it generates an adaptive grid. It keeps population centres with a population greater than zero, avoids counting the municipality-capital-entity-centre hierarchy several times, guarantees at least one point per inhabited municipality, makes small cells where there is urban concentration and groups the rest only when they contain population. No requests are wasted covering hillside, sea or empty industrial estate.
Cost control is the part I use most in my daily work. Cells grow automatically until the worst case of three pages per cell does not exceed 4,500 calls, leaving margin against the 4,800 lockout of the plan on contract. There is a budget with a hard limit that throws an exception when it is exceeded, and note this, usage is still logged even with an unlimited budget; and the header of returned fields is trimmed to the bare minimum so that each call costs as little as possible.
Relaunching the same grid does not search anything again: it reports zero cells searched and every one skipped. And the data model separates the unique place from each time it has been seen, so a hairdresser that falls on the edge of two cells is one record with two observations, not two records. That is the difference between a database you can relaunch every month and one you have to clean every month. The test checks it: two passes leave one seed and two sightings.
Results
Figures measured over the repositories. The tests are counted one by one, not estimated.
Two of the three sources depend on somebody else's page not changing its design, and that is the expensive part of maintenance. If I started again, I would prioritise licensed APIs and leave assisted navigation only for what does not exist in any structured place: the official API tool has a fifth of the code, six tests, a controlled budget and nothing breaks. The other thing that is missing is a shared data contract between the three. Each one defines its own schema and there are stopgap transformations in between; a common schema with validation at the boundary would have removed duplicates and the odd silent failure. I already corrected that in the CRM with the blacklist per entity; in the pipeline it still shows. And the test ceiling of the big tool is another matter: one hundred and thirty-nine tests for thirteen thousand lines sounds good until you see that they are a single file.
Limits
This is not a legal disclaimer. These are design decisions that can be checked in the code.
Public data only, at a deliberately slow pace, with your own session and no automatic interactions: nothing is liked, commented on or messaged on your behalf. A single proxy, no rotation, and explicit respect for the block: if the platform says no, the program stops or passes the job to a person. Screenshots and traces exist to reproduce an error, not to file anyone.
The emails obtained are personal data and their processing is subject to the GDPR. The database is not published or sold: it is part of a confidential portfolio, the code is private and it can be reviewed in conversation with anyone who asks.