CleanJobData

How to Scrape iCIMS Job Listings

CleanJobData Engineering

TL;DR

iCIMS has no public JSON jobs endpoint. You page through the HTML search results on a company's career subdomain to collect job IDs, then fetch each job page and read its schema.org JobPosting JSON-LD block. Pagination uses a zero-based pr parameter, page size varies by company, and some browser user agents are rejected with a 405.

Most applicant tracking systems give you a JSON endpoint. iCIMS doesn't. Its career sites are server-rendered HTML, and the reliable structured data sits in a schema.org JobPosting JSON-LD block on each job page. That makes iCIMS a scraping job rather than an API integration, so it's worth knowing where it's fragile.

Finding the Career Site Host

Many iCIMS customers use a host of the form https://careers-<name>.icims.com, for example careers-daddario.icims.com. Others put the same pages behind their own domain, so if you don't see an icims.com address, look for requests to one in the network tab or the page source. There's no public directory of iCIMS customers, so you need each company's address.

Listing Jobs

The listing is an HTML search page. Two parameters matter: ss=1 (a search with no filters) and in_iframe=1, which returns the stripped-down version of the page without the site chrome.

curl "https://careers-tephseal.icims.com/jobs/search?ss=1&in_iframe=1"

Each job on the page is a link of the form https://careers-<name>.icims.com/jobs/<id>/<slug>/job, so the job IDs come from those href values. The page contains no locations and no descriptions, only the IDs and titles.

Pagination

Add pr=<n> to get another page. The page number is zero-based: pr=0 is the first page (and the same as leaving pr off), pr=1 the second, and so on.

curl "https://careers-tephseal.icims.com/jobs/search?ss=1&in_iframe=1&pr=1"

Three things to know:

  • Page size varies by company. One board returned 20 jobs per page, another 50. Don't hard-code a number.
  • The pager text says how many pages there are. Each page contains text like Page 2 of 9, which you can read to know when you're done.
  • Don't test for rel="next" as plain text. The page includes an HTML comment that mentions rel="next", so a simple text search reports a next page even on the last one. Look for a real <link rel="next" …> tag instead. It's present on every page except the last. Asking for a page past the end returns no job links, which is another reliable stop signal.

A 175-job board came to 9 pages (eight of 20 jobs and a last page of 15).

The Job Page and Its JSON-LD

Fetch each job with its ID:

curl "https://careers-tephseal.icims.com/jobs/2678/job?in_iframe=1"

The page includes one <script type="application/ld+json"> block containing a JobPosting:

{
  "@type": "JobPosting",
  "title": "Events & Visitor Experience Coordinator (Part-Time)",
  "description": "<p>…</p>",
  "datePosted": "2024-09-20T04:56:47.477Z",
  "validThrough": "2027-09-20T04:56:47.477Z",
  "employmentType": "PART_TIME",
  "jobLocationType": "TELECOMMUTE",
  "occupationalCategory": "Project Management",
  "directApply": true,
  "hiringOrganization": { "name": "D'Addario & Company", "sameAs": "http://www.daddario.com" },
  "jobLocation": [
    { "address": { "addressLocality": "Farmingdale", "addressRegion": "NY", "addressCountry": "US",
                   "streetAddress": "595 Smith Street", "postalCode": "UNAVAILABLE" } }
  ]
}

In our checks, every job page we sampled had this block. Everything you need, apart from the ID, comes from it.

Fetching Jobs (Node/TypeScript)

// Some browser user agents (e.g. a Windows one) get a 405. This one works.
const HEADERS = {
  "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36",
  accept: "text/html",
};
 
async function getHtml(url: string) {
  const res = await fetch(url, { headers: HEADERS });
  if (!res.ok) throw new Error(`iCIMS ${res.status} for ${url}`);
  return res.text();
}
 
export async function listJobIds(tenantHost: string): Promise<string[]> {
  const ids = new Set<string>();
  for (let page = 0; page < 200; page++) {
    const html = await getHtml(
      `https://${tenantHost}/jobs/search?ss=1&in_iframe=1${page ? `&pr=${page}` : ""}`,
    );
    const before = ids.size;
    for (const m of html.matchAll(/href="https:\/\/[^"\/]+\/jobs\/(\d+)\//g)) ids.add(m[1]);
 
    const hasNext = /<link\b[^>]*\brel="next"/i.test(html); // the tag, not the comment
    if (!hasNext || ids.size === before) break;
  }
  return [...ids];
}
 
function readJobPosting(html: string) {
  for (const m of html.matchAll(/<script[^>]*type="application\/ld\+json"[^>]*>([\s\S]*?)<\/script>/g)) {
    try {
      const json = JSON.parse(m[1]);
      if (json["@type"] === "JobPosting") return json;
    } catch {
      /* ignore blocks that aren't valid JSON */
    }
  }
  return null;
}
 
const clean = (v?: string) => (v && v.toUpperCase() !== "UNAVAILABLE" ? v : undefined);
 
export async function fetchIcimsJobs(tenantHost: string) {
  const ids = await listJobIds(tenantHost);
  const jobs = [];
  for (let i = 0; i < ids.length; i += 5) {
    const batch = ids.slice(i, i + 5);
    const settled = await Promise.allSettled(
      batch.map((id) => getHtml(`https://${tenantHost}/jobs/${id}/job?in_iframe=1`)),
    );
    settled.forEach((r, idx) => {
      if (r.status !== "fulfilled") return;
      const jp = readJobPosting(r.value);
      if (!jp) return;
      const places = Array.isArray(jp.jobLocation) ? jp.jobLocation : jp.jobLocation ? [jp.jobLocation] : [];
      jobs.push({
        id: batch[idx],
        title: jp.title,
        company: jp.hiringOrganization?.name,
        website: jp.hiringOrganization?.sameAs,
        posted: jp.datePosted,
        expires: jp.validThrough,
        employmentType: jp.employmentType,
        telecommuteFlag: jp.jobLocationType === "TELECOMMUTE", // a hint only, see below
        category: jp.occupationalCategory,
        locations: places.map((p: any) => ({
          city: clean(p.address?.addressLocality),
          region: clean(p.address?.addressRegion),
          country: clean(p.address?.addressCountry),
        })),
        html: jp.description ?? "",
        url: jp.url ?? `https://${tenantHost}/jobs/${batch[idx]}/job`,
      });
    });
  }
  return jobs;
}

Fields Worth Knowing

  • title, description (HTML) and url are what you'd expect.
  • datePosted and validThrough are ISO timestamps, but only the date part is trustworthy: the time of day for the same job differed between two requests a few minutes apart in our tests, so use the date and ignore the time. The posting above was published in 2024 and doesn't expire until 2027, so a job can look old while still being open.
  • employmentType is an uppercase code such as FULL_TIME or PART_TIME.
  • jobLocation can be a single object or an array, so handle both. Each place has an address with addressLocality, addressRegion, addressCountry, streetAddress and postalCode. Countries in the pages we checked were ISO codes such as US.
  • jobLocationType can be TELECOMMUTE, but don't rely on it for remote detection. The only job that carried it in our checks was an in-person-sounding role with no mention of remote work in its description, and it was missing from the job that described itself as "a hybrid role". Read the title and description for "remote" or "hybrid" instead, and treat the flag as at most a hint.
  • hiringOrganization carries the company name, and its sameAs is the company's website.
  • occupationalCategory is a job category label such as "Project Management".
  • baseSalary is part of the schema.org format but was absent on all 14 jobs we sampled, so don't count on it.

Gotchas

Address placeholders. postalCode and postOfficeBoxNumber are often the literal string UNAVAILABLE. Filter it out, as the clean() helper above does, or it ends up in your location text.

User-agent filtering. iCIMS rejects some browser user agents. A Windows Chrome user agent got a 405 in our tests, while a macOS one, curl's default and python-requests all got a 200. If you see a sudden 405, change the user agent before you assume you're blocked, and back off and retry on 405 and 429.

Every job needs its own request. The list gives IDs only, so a board of 175 jobs is 9 listing requests plus 175 detail requests. Keep concurrency low and skip IDs you've already stored on repeat runs.

Missing JSON-LD means a lost job. If a job page has no parseable JobPosting block, there's no structured data to fall back on, and it's easiest to skip it and log it. That never happened in our sample, but you should handle it.

Old postings can stay open. Compare datePosted and validThrough with today's date before treating a job as fresh.

No webhooks or published limits. It's pull-only. Poll and compare job IDs between runs to detect closures.

When This Is Enough

For one company with a modest number of jobs, the approach above works and is free. The fragile parts are the HTML pagination and the request volume.

When It Isn't

Across many iCIMS customers, plus every other platform, the maintenance adds up: hosts, page sizes, blocked user agents and placeholder values. Compare this with Workday and Greenhouse, and see how we normalized four ATS platforms.

CleanJobData reads iCIMS career sites along with other platforms and returns one schema, with locations resolved to cities, states and countries, an employment type and remote type, and company details. Salary is only there when the job page publishes it. Try it in the Playground or see the API Documentation.