# Job Postings Datasets: What's Available and How to Get One > Free and commercial job postings datasets compared: Kaggle scrapes, USAJOBS, JOLTS, O*NET, vendors, building your own, and a checklist for judging quality. Source: https://cleanjobdata.com/articles/job-postings-dataset Published: 2026-11-01 TL;DR: Job postings datasets come in four kinds: free scraped snapshots (such as Kaggle's LinkedIn Job Postings 2023 to 2024), official sources (USAJOBS for US federal postings; JOLTS, O*NET and Indeed Hiring Lab for aggregates and taxonomies), commercial vendors (Lightcast, Coresignal, Revelio Labs, TheirStack), and datasets you build yourself from public ATS endpoints. Government statistics like JOLTS are survey estimates, not individual postings. Judge any dataset on deduplication, freshness, expired-job detection, salary coverage, location normalization and license terms before you build on it. You can get a job postings dataset four ways: download a free scraped snapshot or an official government source, license one from a commercial vendor, build your own from public ATS endpoints, or export one from a job data API. The right choice depends on whether you need individual postings or aggregate counts, history or fresh data, and the right to store and redistribute it. Many "job postings datasets" turn out to be something else, such as survey statistics, an occupation taxonomy, or a one-off scrape with unclear rights. This guide sorts them, then gives a checklist for judging any dataset before you commit. Sources were checked on November 1, 2026. ## At a Glance | Source | What it is | Individual postings? | Freshness | Terms | |---|---|---|---|---| | Kaggle "LinkedIn Job Postings (2023 - 2024)" | Community scrape of LinkedIn | Yes | Static snapshot | Check the dataset's license; LinkedIn hasn't licensed it | | Other Kaggle scrapes (Indeed, Glassdoor) | Small single-role or single-country scrapes | Yes | Static | Varies by uploader | | USAJOBS Search and Historic JOA APIs | Official US federal job announcements | Yes, federal only | Current and historical | USAJOBS API terms | | BLS JOLTS | Monthly establishment survey | No, estimates | Monthly | US government statistics | | O*NET database | Occupation taxonomy with job titles | No | Periodic releases | CC BY 4.0 | | Indeed Hiring Lab | Job postings indices | No, indices | Weekly | CC BY 4.0 | | Commercial vendors | Multi-source postings, often with history | Yes | Ongoing | Contract | | Build your own | Snapshots of public ATS boards | Yes | As often as you run it | Each source's terms | | CleanJobData | ATS and career-site postings via API or daily files | Yes | Continuous | Plan licence | ## Free and Public Datasets ### Kaggle scrapes The best-known free dataset is "LinkedIn Job Postings (2023 - 2024)" on Kaggle, which its description puts at roughly 124,000 postings. It's useful for coursework, prototyping a parser or testing a classifier. Two cautions. It's a static snapshot, so it can't power anything live. And it was scraped from LinkedIn, whose terms prohibit scraping; whatever license the uploader selects, it doesn't come with LinkedIn's permission. Check the license field on the dataset page and don't build a commercial product on it. The same applies to the smaller Indeed and Glassdoor scrapes on Kaggle, which usually cover one role or one country. ### USAJOBS [USAJOBS](https://developer.usajobs.gov/api-reference/job-apis) is the cleanest free source of real postings, limited to US federal jobs. The Search API covers current announcements and needs a key from a request form. The Historic JOA endpoint returns past announcements with fields such as `positionTitle`, `hiringAgencyName`, `positionOpenDate`, `positionCloseDate`, `minimumSalary`, `maximumSalary` and `positionLocations`. Its tutorial examples call it without a key. ```python import requests def historic_joa(series: str, start: str, end: str): """Yield past USAJOBS announcements for one occupational series (dates as MM-DD-YYYY).""" page = 1 while True: res = requests.get( "https://data.usajobs.gov/api/historicjoa", params={ "PositionSeries": series, "StartPositionOpenDate": start, "EndPositionOpenDate": end, "PageSize": 100, "PageNumber": page, }, timeout=60, ) res.raise_for_status() body = res.json() yield from body["data"] if page >= body["paging"]["metadata"]["totalPages"]: break page += 1 for joa in historic_joa("2210", "01-01-2025", "12-31-2025"): # 2210 = IT management print(joa["positionTitle"], joa["minimumSalary"], joa["maximumSalary"]) ``` The USAJOBS terms don't allow reselling or redistributing API data as a standalone feed, so this works for analysis and internal tools, not for a data product. ### Government statistics: JOLTS The [Job Openings and Labor Turnover Survey](https://www.bls.gov/jlt/) (JOLTS) from the US Bureau of Labor Statistics estimates job openings, hires and separations each month from a sample of about 21,000 business establishments. It's the standard source for "how many openings are there", but it contains no postings: no titles, no companies, no descriptions. Use it to sanity-check trends you see in postings data, not as a postings dataset. ### Taxonomies and indices: O*NET and Indeed Hiring Lab The [O*NET database](https://www.onetcenter.org/database.html) (version 31.0 at the time of writing) describes US occupations and includes alternate and reported job titles for each O*NET-SOC code. It's free under CC BY 4.0 in CSV, Excel, JSON and SQL formats. It's the reference you map postings to, not postings itself. See our [job title normalization guide](https://cleanjobdata.com/articles/job-title-normalization) for how to use it. Indeed's Hiring Lab publishes its [job postings index](https://github.com/hiring-lab/job_postings_tracker) on GitHub under CC BY 4.0: daily values, updated weekly, measuring postings relative to February 1, 2020, by country, sector and some regions. It's excellent for trend lines and contains no individual jobs. ## Commercial Vendors Commercial datasets are where you get millions of individual postings with history. Each vendor describes its data differently; here's what each says on its own site: - **[Lightcast](https://kb.lightcast.io/en/articles/6957446-job-posting-analytics-jpa-methodology)** collects postings from "over 220,000 current and historical sources" by scraping job boards and company websites. It runs a two-step deduplication (within a source, then across sources over a 60-day window) that removes up to 80% of what it collects, and codes postings to its own occupation, title and skills taxonomies. It's aimed at labor market analysis. - **[Coresignal](https://coresignal.com/)** advertises 486M+ deduplicated active and historical job postings from multiple sources, with data since 2020, delivered through an API or as flat files (JSONL or CSV). - **[Revelio Labs](https://www.reveliolabs.com/)** describes a job postings dataset updated daily, sourced from company websites, major job boards and staffing firms, as part of a wider workforce data platform. - **[TheirStack](https://theirstack.com/en/job-posting-api)** says it aggregates jobs from over 367,000 websites, including career pages, job boards and ATS platforms, removes duplicates across them, and adds technology tags for company and lead research. Some vendors, TheirStack among them, list LinkedIn and Indeed among their sources, so ask each vendor what rights you get to store, display and redistribute the data. Coresignal and TheirStack publish self-serve plans; see [Best Job APIs in 2026](https://cleanjobdata.com/articles/best-job-apis-2026) for the figures we verified. For the others, pricing is by quote. ## Building Your Own If you need a few hundred employers, you can build a dataset from public ATS job boards. Greenhouse, Lever, Ashby and others expose read endpoints that return an employer's open jobs without a key. The [ATS job postings API list](https://cleanjobdata.com/articles/ats-public-job-postings-api-list) covers which platforms have them. A minimal daily snapshot looks like this: ```python import datetime import json import requests BOARDS = ["airbnb", "stripe"] # example Greenhouse board tokens def snapshot(path: str): seen_at = datetime.datetime.now(datetime.timezone.utc).isoformat() with open(path, "w") as out: for token in BOARDS: res = requests.get( f"https://boards-api.greenhouse.io/v1/boards/{token}/jobs", params={"content": "true"}, timeout=30, ) res.raise_for_status() for job in res.json()["jobs"]: out.write(json.dumps({ "source": "greenhouse", "board": token, "id": job["id"], "title": job["title"], "location": job["location"]["name"], "url": job["absolute_url"], "updated_at": job["updated_at"], "seen_at": seen_at, }) + "\n") snapshot(f"jobs-{datetime.date.today()}.ndjson") ``` That's the easy part. The work is everything after: finding board tokens, adding an adapter for every ATS, diffing snapshots to detect expired jobs, [deduplicating](https://cleanjobdata.com/articles/deduplicating-job-listings-across-sources) reposts, [normalizing locations](https://cleanjobdata.com/guides/job-location-normalization) and [extracting salary](https://cleanjobdata.com/guides/salary-data-extraction). Budget for maintenance, not just the first run. ## CleanJobData: API and Bulk Files CleanJobData collects postings from employer ATS platforms and company career sites, and it does not include Indeed, LinkedIn or Glassdoor data. You can get it two ways: - **The API.** Paid plans include a licence to sync jobs into your own database, so you can build a dataset with incremental pulls (`published_after` or `max_age`) and catch closed jobs with the [expired jobs endpoint](https://cleanjobdata.com/docs/api/jobs/expired). See [pricing](https://cleanjobdata.com/pricing) and [Syncing Job Data in Depth](https://cleanjobdata.com/guides/syncing-job-data-in-depth). - **Enterprise bulk files.** [Enterprise](https://cleanjobdata.com/enterprise) adds daily flat-file dumps as gzipped NDJSON, unlimited API requests, and a licence to redistribute, resell and train AI models on the data. Each record carries the same normalized schema: `title`, full `description`, structured `locations`, `salary_min`, `salary_max` and `salary_currency` where pay was published, `experience_level`, `remote_type`, `employment_type`, `published`, `is_active`, `expired_at` and a `company` object. ## Checklist: How to Evaluate a Job Postings Dataset Ask for a sample, then test it against this list. Vendor fact sheets rarely answer these questions; the data does. | Check | What to look for | How to test a sample | |---|---|---| | Deduplication | One record per real job, not per site or per repost | Count groups sharing normalized company, title and location | | Freshness | Both a posted date and a first-seen date | Compare `published` to when the record appeared | | Expired detection | A closed flag or closed date | Open 20 random "active" apply URLs and see how many are dead | | Salary coverage | Numeric min, max, currency and period, posted vs estimated | Share of records with numeric salary, by country | | Location normalization | Country codes or IDs, multi-location support, remote flags | Look for "Remote" or "Multiple Locations" stored as a city | | Description | Full text, not a snippet | Compare length against the live posting | | Source and apply URL | The employer's own posting where possible | Share of apply URLs on employer or ATS domains | | Company identity | A stable ID or domain, not just a name | Spot-check "Amazon" vs "Amazon Web Services" vs "AWS" | | Taxonomy | Occupation or seniority mapping, with null when unknown | Check coverage and how unknowns are represented | | License | Storage, display, redistribution and AI training rights | Read the contract, not the marketing page | A quick script gets you most of the numbers on any NDJSON sample. Adjust the field names to the dataset: ```python import json import re from collections import Counter def norm(s): return re.sub(r"[^a-z0-9]+", " ", (s or "").lower()).strip() rows = [json.loads(line) for line in open("sample.ndjson")] n = len(rows) keys = Counter( (norm((r.get("company") or {}).get("name")), norm(r.get("title")), norm(r.get("location"))) for r in rows ) dupes = sum(c - 1 for c in keys.values() if c > 1) def share(pred): return f"{100 * sum(1 for r in rows if pred(r)) / n:.1f}%" print("records:", n) print("likely duplicates:", f"{100 * dupes / n:.1f}%") print("numeric salary:", share(lambda r: r.get("salary_min") is not None)) print("has description:", share(lambda r: len(r.get("description") or "") > 500)) print("marked inactive:", share(lambda r: r.get("is_active") is False)) ``` Read the results relative to each other: a dataset with 2% duplicates and 30% salary coverage in the US is in a different league from one with 25% duplicates and salaries that are mostly estimates. ## FAQ ### Where can I download a free job postings dataset? Kaggle has free scraped snapshots, such as "LinkedIn Job Postings (2023 - 2024)", and USAJOBS offers current and historical US federal postings through its APIs. Check the license before any commercial use; scraped datasets don't carry the source site's permission. ### Is JOLTS a job postings dataset? No. JOLTS is a monthly Bureau of Labor Statistics survey of about 21,000 establishments that estimates openings, hires and separations. It has no individual postings, titles or companies. ### Which job postings dataset has the most history? Commercial vendors usually have the deepest history. Coresignal advertises data since 2020, and Lightcast draws on more than 220,000 current and historical sources. Free sources such as Kaggle snapshots cover short windows. ### Can I redistribute a job postings dataset? Only if your license says so. Most free and API sources restrict redistribution, and USAJOBS explicitly forbids reselling its data as a standalone feed. CleanJobData's Enterprise plan includes a licence to redistribute, resell and train AI models. ### Does CleanJobData offer bulk job data downloads? Yes. The Enterprise plan includes daily flat-file dumps as gzipped NDJSON, and paid API plans allow syncing jobs into your own database. See [Enterprise](https://cleanjobdata.com/enterprise) for bulk access.