Terug naar blog
Automation

The no-API problem: scraping, headless browsers, and email as an API

Door Amogh3 min lezen

The pitch for automation always assumes a clean API on both ends: system A has one, system B has one, you connect them, done. In practice, a huge share of the systems small and mid-sized businesses actually run — legacy booking platforms, niche scheduling tools, payment portals built for a different era — expose nothing but a web page a person is expected to click through by hand. That's the no-API problem, and it's one of the most common reasons "just automate it" turns out to be harder than it sounds.

The two workarounds, and when each applies

Headless browser capture. When a system has no API but does have a web interface, a headless browser (we use Browserless) can sign in with real session credentials and read exactly what a person would see — screenshots, page content, structured data pulled out of the rendered HTML. We reached for this in three different client situations that had nothing else in common except the missing API:

  • Automated invoicing, where a booking platform had no API to expose the supporting figures needed for each instructor's pay period, so a headless browser signed in with session cookies and captured them automatically.
  • Automated class-staffing emails, where the only way to see which classes still needed a helper was to log into the booking platform and look — now done by a scheduled headless-browser check, four days a week, with the results turned into one clean team email.
  • Automated schedule scrape → website sync, where a public WordPress schedule had to mirror a booking platform with no feed at all — solved by scraping the calendar on a schedule and republishing the results automatically.

Email as an integration point. Sometimes the system doesn't even have a usable web interface to scrape reliably, but it does send confirmation emails — and those emails are structured enough to parse. In a booking-to-lifecycle marketing automation, a course platform's payment-confirmation emails became the actual data source: a Gmail trigger catches each one, parses it into structured fields, and uses that to update the CRM and enroll the customer in the right Klaviyo welcome sequence. The closed platform never had to change. The email it was already sending became the API it never offered.

Why this matters beyond the specific hack

Neither pattern is glamorous, and neither is what you'd design if you were building the system from scratch. But most businesses aren't starting from scratch — they're running on tools they chose years ago, some of which will never ship an API, and replacing them outright is often more disruptive than working around the gap. The no-API problem isn't a reason automation is impossible; it's a reason "do you have an API" is the wrong first question. The better one is "what does a person currently look at, and can we look at the same thing programmatically."

The tradeoff worth naming honestly

Headless-browser and email-parsing approaches are inherently more fragile than a real API integration — if the source platform changes its page layout or email template, the automation needs updating. That's a real cost, and it's worth weighing against the alternative, which is usually a person doing the same lookup by hand, indefinitely, with the same fragility just distributed across human attention instead of code. In every case above, the workaround removed hours of recurring manual work; it didn't remove the underlying platform's limitations, and it was never pitched as doing so.

If a system you rely on has no API and you've been told automation "isn't possible" because of it, that's usually the wrong conclusion — it just means the integration point is a screen or an inbox instead of an endpoint.

Benieuwd wat AI realistisch gezien uit handen kan nemen?

Neem je tijd terug. Wij automatiseren de handmatige taken, zodat jij de controle behoudt. Plan vandaag nog een gesprek in.

Plan een kennismaking

30 minuten · gratis · vrijblijvend

Of stuur ons een bericht