Job Title Normalization: A Practical Guide

Job title normalization maps a raw title such as Sr. SWE II (Backend) - Remote to a clean title, a seniority level and a standard occupation such as O*NET-SOC 15-1252.00, Software Developers. The approach that holds up in production is a pipeline: clean the string, extract seniority with explicit rules, then match the remaining core title to a taxonomy, with embeddings or an LLM reserved for titles your rules can't place.

This guide walks through each step with code, covers the main taxonomies, and ends with how to measure whether it works. It's written for teams building job search, analytics or matching on top of postings data.

At a Glance

Why Job Titles Are Messy

Employers write titles for candidates and search engines, not for data pipelines. In any large sample you'll see:

• Abbreviations: Sr., Jr., SWE, Mgr, PM (product, project or program?), RN.
• Level codes: Engineer II, L5, Level 3, Band 7. They mean different things at each company.
• Noise: locations, Remote, pay, shift times, requisition IDs, URGENT, emoji, and German gender tags such as (m/w/d).
• Internal or branded titles: Member of Technical Staff, Customer Happiness Hero.
• Compound titles: Senior Manager, Early Career Programs contains both a senior and an entry-level signal.
• Words that change meaning by context: Staff Engineer is senior, Staff Nurse is not. Associate is often junior, Associate Director is not. Principal Engineer is a level, a school principal is an occupation. Lead Engineer is seniority, Lead Generation Specialist is not.
• Many languages: Ingeniero de Software Senior, Softwareentwickler, 資深工程師.

Every step below exists to deal with one of these.

Step 1: Clean the String

Cleaning removes noise without guessing at meaning. Keep the original title too; you'll want it for display and debugging.

import re
 
ABBREVIATIONS = {
    r"\bsr\b\.?": "senior",
    r"\bjr\b\.?": "junior",
    r"\bmgr\b\.?": "manager",
    r"\bswe\b": "software engineer",
    r"\beng\b\.?": "engineer",
    r"\bdev\b": "developer",
}
 
NOISE = [
    r"\((?:m|w|f|d|x|h)(?:\s*/\s*(?:m|w|f|d|x|h))+\)",  # (m/w/d), (h/f)
    r"\breq(?:uisition)?\s*#?\s*\d+\b",                   # Req #12345
    r"\b(?:remote|hybrid|on-?site)\b",                    # work mode lives in its own field
    r"\$\s?\d[\d,]*(?:k)?(?:\s*-\s*\$?\d[\d,]*(?:k)?)?",  # pay in the title
    r"\burgent(?:ly)?\b|\bimmediate start\b",
]
 
def clean_title(raw: str) -> str:
    t = raw.lower()
    for pattern in NOISE:
        t = re.sub(pattern, " ", t)
    t = re.sub(r"\s+[-|–—@]\s+.*$", " ", t)   # drop " - Berlin", " | Acme" suffixes
    for pattern, full in ABBREVIATIONS.items():
        t = re.sub(pattern, full, t)
    t = re.sub(r"[^\w\s/&+#.]", " ", t)       # keep c++, c#, .net
    return re.sub(r"\s+", " ", t).strip()
 
clean_title("Sr. SWE II (Backend) - Remote")   # 'senior software engineer ii backend'
clean_title("Softwareentwickler (m/w/d) | Acme")  # 'softwareentwickler'

The suffix rule is deliberately blunt. Check it against your data: some titles use a dash for the specialty (Engineer - Payments), and you may prefer to keep that part.

Step 2: Extract Seniority

Seniority should come out before taxonomy matching, for two reasons. Taxonomies like O*NET-SOC don't encode it (a senior and a junior software engineer share one code), and seniority words confuse similarity search.

Use ordered rules: check the most specific patterns first, require context for ambiguous words, and return None when nothing matches. An unknown level is a real state, not a synonym for mid-level.

import re
 
# Order matters: the first match wins.
SENIORITY_RULES = [
    ("EX", r"\b(chief \w+ officer|c[etfo]o|vice president|vp|head of|managing director|founder)\b"),
    ("SE", r"\b(associate|assistant|deputy) director\b"),  # before the bare "associate" rule
    ("EX", r"\bdirector\b"),
    ("SE", r"\b(senior|principal|lead)\b(?! generation)"),
    ("SE", r"\bstaff (?:\w+ ){0,2}(engineer|scientist|developer|architect)\b"),  # not "staff nurse"
    ("EN", r"\b(junior|entry level|graduate|intern(ship)?|trainee|apprentice|associate)\b"),
]
 
def seniority(clean: str) -> str | None:
    for level, pattern in SENIORITY_RULES:
        if re.search(pattern, clean):
            return level
    return None
 
seniority("senior software engineer ii backend")  # 'SE'
seniority("staff nurse")                           # None
seniority("associate director of finance")         # 'SE'
seniority("software engineer")                     # None

These rules are illustrative, not complete. Two choices worth making on purpose:

• Numeric levels. II or L5 can be mapped per company if you know its ladder. Across companies they're unreliable, so leave them out of the general rules.
• Multiple signals. For a title like Senior Manager, Early Career Programs, return every level that matched as well as the first one, so downstream code can decide.

The Experience Level Mapping guide covers how to treat null levels in filters and UI.

Step 3: Map to a Taxonomy

A taxonomy gives you a stable ID to group, count and filter on. Two are free and widely used.

O*NET-SOC (US)

The O*NET database (version 31.0 at the time of writing) is built on the O*NET-SOC taxonomy, aligned with the US Standard Occupational Classification. Codes look like 15-1252.00 (Software Developers), and O*NET lists reported titles for each occupation; for 15-1252.00 they include Software Engineer, Application Developer, DevOps Engineer and Software Architect. Its Alternate Titles file holds tens of thousands of lay titles keyed to codes, with columns for O*NET-SOC Code, Title, Alternate Title, Short Title and Source(s). It's licensed CC BY 4.0 and downloadable as CSV, Excel, JSON or SQL.

ESCO (EU)

ESCO is the European classification of skills, competences and occupations: 3,039 occupations and 13,939 skills in 28 languages (version 1.2.1). Each ESCO occupation maps to exactly one ISCO-08 code, so you can roll up to the international standard. It's the better fit if your postings are European or multilingual.

Matching: dictionary first, then similarity

Start with an exact lookup against the alias table. It's fast, explainable and handles the common titles that make up most of your volume.

import re
import pandas as pd
 
# O*NET "Alternate Titles" file, downloaded in text (tab-separated) format
alt = pd.read_csv("Alternate Titles.txt", sep="\t")
alias_to_code = {}
for row in alt.itertuples(index=False):
    alias_to_code.setdefault(clean_title(row[2]), row[0])   # Alternate Title -> O*NET-SOC Code
for code, title in alt[["O*NET-SOC Code", "Title"]].drop_duplicates().itertuples(index=False):
    alias_to_code.setdefault(clean_title(title), code)
 
SENIORITY_WORDS = r"\b(senior|junior|lead|principal|staff|head of|entry level|intern|i{1,3}|iv)\b"
 
def core_title(clean: str) -> str:
    return re.sub(r"\s+", " ", re.sub(SENIORITY_WORDS, " ", clean)).strip()
 
def exact_match(raw: str) -> str | None:
    return alias_to_code.get(core_title(clean_title(raw)))

For titles the dictionary misses, embed the core title and take the nearest taxonomy entries as candidates:

import numpy as np
from sentence_transformers import SentenceTransformer
 
model = SentenceTransformer("all-MiniLM-L6-v2")
labels = list(alias_to_code.keys())
label_vecs = model.encode(labels, normalize_embeddings=True)
 
def candidates(raw: str, k: int = 5):
    q = model.encode([core_title(clean_title(raw))], normalize_embeddings=True)[0]
    scores = label_vecs @ q
    top = np.argsort(-scores)[:k]
    return [(labels[i], alias_to_code[labels[i]], float(scores[i])) for i in top]
 
candidates("backend platform engineer")

Set a score threshold from your labeled data (Step 4) and return no code below it rather than the closest wrong answer.

Using an LLM for the long tail

An LLM helps most with titles that are ambiguous without context, such as Associate or Consultant. Keep it constrained: give it the top candidates from the embedding step plus a short slice of the description, ask it to choose one code or none, and reject anything outside the list.

// callModel is whatever LLM client you use; it takes a prompt and returns text.
async function pickOccupation(title, description, candidates, callModel) {
  const options = candidates.map((c) => `${c.code}: ${c.label}`).join("\n");
  const prompt = [
    "Choose the occupation code that best fits this job, or answer NONE.",
    `Title: ${title}`,
    `Description excerpt: ${description.slice(0, 1500)}`,
    "Options:",
    options,
    "Answer with only the code or NONE.",
  ].join("\n");
 
  const answer = (await callModel(prompt)).trim();
  const allowed = new Set(candidates.map((c) => c.code));
  return allowed.has(answer) ? answer : null; // never trust a code outside the list
}

Cache results by cleaned title. Most titles repeat, so you pay for each distinct title once rather than per posting.

Step 4: Evaluate It

Normalization without measurement drifts quietly. A small, honest evaluation set catches most problems.

1. Label a sample. Pull 300 to 500 titles, stratified by source, country and language, and label the correct code and seniority by hand, including "none" where no code fits.
2. Report accuracy and coverage separately. Coverage is the share of titles that got any code; accuracy is how many of those are right. A system that answers everything at 70% accuracy is often worse than one that answers 85% of titles at 95%.
3. Score at more than one level. For O*NET-SOC, check the major group (first two digits) as well as the full code. Getting "Computer and Mathematical" right but the detailed occupation wrong is a different failure from getting the group wrong.
4. Keep a regression set of the tricky titles you've fixed, and run it on every rule change.

REGRESSION = [
    ("Staff Nurse - Night Shift", None),
    ("Staff Software Engineer", "SE"),
    ("Associate Director, Finance", "SE"),
    ("Associate Product Manager", "EN"),
    ("Lead Generation Specialist", None),
    ("Software Engineer", None),
]
 
failures = [(t, exp, seniority(clean_title(t)))
            for t, exp in REGRESSION if seniority(clean_title(t)) != exp]
assert not failures, failures

Watch the null rate in production too. A sudden jump usually means a new source with a title format your cleaning step doesn't handle.

What CleanJobData Does With Titles

CleanJobData returns title as the employer published it and adds experience_level, one of EN, MI, SE or EX, inferred from the title when jobs are normalized. experience_levels lists every level that matched, and experience_level is null when no level can be inferred. Matching covers several languages, not only English. You can filter on it directly:

curl "https://api.cleanjobdata.com/jobs?title=data%20engineer&experience_level=SE,EX" \
  -H "Authorization: Bearer $CLEANJOBDATA_API_KEY"

CleanJobData doesn't assign O*NET or ESCO occupation codes, so if you need occupations, run Step 3 on the titles you pull. Details are in the Experience Level Mapping guide and the Jobs API reference.

FAQ

What is job title normalization?

It's the process of mapping raw job titles to a consistent form: a cleaned title, a seniority level and usually a standard occupation code such as O*NET-SOC or ESCO. It makes postings from different employers comparable for search, filtering and analytics.

Should I use O*NET-SOC or ESCO?

Use O*NET-SOC for US-focused data and ESCO for European or multilingual data. ESCO occupations map to ISCO-08, so you can roll either up to an international standard if you need to compare across regions.

Can an LLM normalize job titles on its own?

It can, but it's slower, costlier and less consistent than rules plus a dictionary. Use it for the titles those methods can't place, give it a fixed list of candidate codes, and reject answers outside that list.

How do I extract seniority from a job title?

Use ordered rules that check specific patterns first, require context for ambiguous words like "staff" and "associate", and return null when nothing matches. Don't treat a missing level as mid-level.

Does CleanJobData normalize job titles?

It keeps the employer's original title and adds experience_level (EN, MI, SE or EX) inferred from it, or null when no level can be inferred. It doesn't assign occupation codes.