career-ops

Scrapes jobs from LinkedIn, Indeed, ZipRecruiter, and Google Jobs using JobSpy, deduplicates against the career-ops processed-urls database, and writes fresh results to the inbox for review.

What the Pipeline Does

• Scrape — Runs JobSpy for each term × location × site combo.
• Deduplicate — Removes jobs already in processed-urls.tsv, scan-history.tsv, and the current pipeline.md LINKS section (case-insensitive on URL).
• Write CSV — Saves fresh jobs to artifacts/inbox/jobspy_YYYYMMDD_HHMM.csv.
• Update pipeline.md — Writes a clean table of the most recent 50 new jobs with a raw LINKS section.
• Append processed URLs — Adds new job URLs to data/processed-urls.tsv.

LinkedIn Safety

LinkedIn is capped at linkedin_max_per_run results per run (default: 20) to avoid rate-limiting. The daily total is tracked via linkedin_max_per_day in the config (informational).

Notes

• No external APIs required — JobSpy scrapes job boards directly.
• search-profile.json is the single source of truth for search config.
• All paths are relative to the script location — no hardcoded paths.
• Errors are reported silently (no tracebacks) to keep cron output clean.