Job scraper for LinkedIn, storing results in SQLite
Top 89.6% on sourcepulse
This Python application addresses the frustration of sifting through irrelevant job postings on LinkedIn by scraping, filtering, and storing job data locally. It targets job seekers looking for a more efficient and personalized way to manage their job search, offering features like keyword filtering, duplicate removal, and a web interface for tracking application status.
How It Works
The project utilizes Python libraries like Requests
and BeautifulSoup
to scrape job postings from LinkedIn based on user-defined search queries and filters specified in a config.json
file. It then processes these postings to remove duplicates and filter out irrelevant jobs based on keywords in titles and descriptions. The cleaned data is stored in a SQLite database. A Flask-based web interface allows users to view, sort, and update the status of job postings (applied, rejected, interview, hidden).
Quick Start & Requirements
pip install -r requirements.txt
python main.py
python app.py
config.json
file with proxy settings, headers, OpenAI API key (for cover letter generation), resume path, and search queries.Highlighted Details
Maintenance & Community
Licensing & Compatibility
Limitations & Caveats
6 months ago
1 day