How to Build a Job Board: Data Sources Compared
Before you write a single line of matching or ranking logic for a job board, you have to decide where the jobs come from. That decision shapes everything downstream — your schema, your update cadence, your legal exposure, and how much of your team's time goes into maintenance instead of product. There isn't a universally correct answer; it depends on how many sources you need and how much ongoing engineering time you're willing to spend.
The Options
- A. Scrape ATS platforms directly. An ATS (Applicant Tracking System) is the software an employer uses to post and manage its job listings — Greenhouse, Lever, Ashby, Workday, and Workable are examples. Each one exposes a public or semi-public JSON API (often called that platform's "boards API," since it's the same endpoint that powers the employer's own public jobs board page) behind their hosted careers pages. You call each one directly, per company, per ATS — see the field-by-field walkthroughs for Greenhouse, Lever, Ashby, Workable, and Workday if you're evaluating this path.
- B. Use a single-ATS-specific tool or API. Some platforms (or third-party wrappers around them) offer a slightly more convenient client for one specific ATS — still limited to that one platform's data and conventions.
- C. General web scraping of career pages. For companies that don't use a major ATS, or that customize their careers page heavily, you scrape the rendered HTML directly — no structured API to lean on.
- D. Use a normalized, multi-source job data API. A third party has already built and maintains adapters across ATS platforms, normalizes the schema, deduplicates, and gives you one endpoint.
Comparison
| Setup time | Ongoing maintenance | Data freshness | Coverage breadth | Legal / ToS risk | |
|---|---|---|---|---|---|
| A. Scrape ATS APIs directly | Days per ATS (auth-free JSON, but you write parsing + polling for each) | Moderate — breaks when a platform changes response shape or adds fields | Depends on your polling interval; no push notifications from any of them | One ATS at a time; multiplies linearly with each platform you add | Low for public boards APIs, but check each platform's terms before high-volume automated use |
| B. Single-ATS tool/API | Fast for that one platform | Low for that platform, but you're still stuck at one ATS | Same polling constraints as A, abstracted slightly | Narrow — locked to whichever ATS the tool wraps | Depends on the tool's own terms and how it accesses the source |
| C. Scrape raw career pages | Slow — every company's HTML is different, no shared structure | High — page redesigns silently break your scraper with no warning | Poor unless you poll aggressively, and even then you're parsing rendered markup | Broadest in theory (any company, ATS or not) but fragile in practice | Highest — no public API contract, more likely to run into ToS friction or anti-bot measures |
| D. Normalized multi-source data API | Fast — one schema, one client, no per-ATS adapter code | Low — the provider absorbs upstream breakage | Depends on the provider's refresh cadence, but consistent across sources | Broad by design, spans multiple ATS platforms and sources under one schema | Low — you're consuming a service built for this, not scraping directly |
Why Nobody Wins on Every Row
Direct ATS scraping (A) is reasonable if you only need one or two companies' jobs and you're comfortable owning the parsing code. The APIs are free, unauthenticated, and well-documented enough to reverse-engineer in an afternoon — see our walkthrough of the Greenhouse boards API for what that looks like in practice.
The problem shows up at scale. Each ATS has a different response shape, pagination behavior, and location/salary conventions, with no shared vocabulary for department or seniority. Spanning Greenhouse, Lever, Ashby, and Workable means maintaining four adapters, and a breaking change to any one of them becomes your on-call problem.
General career-page scraping (C) is the least defensible long-term. There's no contract with the source — a CSS class rename or a client-side rendering change can silently zero out your data. It's also the option most likely to run into legal gray areas, since you're parsing pages built for humans, not machines.
A normalized data API (D) trades a subscription cost for not having to build or maintain that adapter layer yourself. It's the right call once multi-source coverage becomes a requirement — the tradeoff is that your freshness and coverage are bounded by the provider's own pipeline rather than something you control end to end.
A Rough Decision Rule
- One company's jobs, low volume, willing to maintain a parser: scrape that ATS directly.
- Several companies on the same ATS: still viable to scrape directly, but budget real maintenance time.
- Multiple ATS platforms, or companies without a major ATS at all: general scraping gets expensive fast, and a normalized API starts looking a lot cheaper than the engineering time you'd spend replicating it.
- Need deduplication across sources (the same job posted on the company site and reposted elsewhere): worth reading how that problem is actually solved before you build it yourself — see deduplicating job listings across sources.
None of these are permanently right or wrong — they're a function of how many sources you need today and how much of that maintenance burden you want to own next year.