CareerBot: Autonomous Job Discovery Pipeline
An autonomous, zero-human-input pipeline that discovers Data Science jobs on LinkedIn, identifies the correct recruiter using a multi-layered search strategy, and verifies their email via Hunter.io APIs.
Problem
Manual execution of repetitive search and verification tasks is unscalable and prone to data loss, requiring an autonomous, fault-tolerant system to streamline the workflow.
Approach
Built a scheduled, fault-tolerant Python agent using real Chrome browser sessions with randomized delays to bypass LinkedIn anti-bot systems. Engineered a 3-layer recruiter discovery system (Local Cache → Direct Poster → Fallback Search) and integrated Hunter.io APIs with automatic key rotation. Implemented robust state management (`run_state.json`) for exact step-level crash recovery.
Outcome
Achieved zero-human-input daily execution that deduplicates jobs and extracts key skills to build a high-quality pipeline.
Reduced API quota usage and browser navigations by 70% using a deterministic local recruiter cache.
Built a resilient architecture capable of pausing and resuming mid-run during network failures without duplicating data.
Automatically resolves and verifies recruiter emails into a classified taxonomy (Valid, Accept All) ready for direct outreach.