Senior Scraping Engineer (Web scraping & Anti-bot)
Indexed description
The platform will support both cloud deployment and on-premises deployment in client infrastructure (on-prem, Kubernetes).
We are now looking for an experienced Tech Engineer for our web scraping development team who will lead and drive architectural decisions, and shape the future of our cutting-edge scraping infrastructure.
Key Responsibilities
Architecture Design
- Data-plane: high-parallel crawler, session management, anti-bot bypass, CAPTCHA solver
- Control-plane: API, authorization, billing, quotas, monitoring
- Infrastructure: microservices, Docker/K8s, CI/CD, observability
- Establish development and release processes
- Code quality control and technical decision oversight
- Browser and network-level fingerprinting (JA3, TLS, CDP)
- Protection bypass: Cloudflare, Akamai, Qrator, ServicePipe, Variti, and others
- CI/CD setup, Helm charts, scaling configuration
- Monitoring organization (Prometheus, Grafana, Loki) and SLA processes
- 4-6+ years in backend, data engineering, automation, or high-load systems
- Strong production web scraping experience with protected / dynamic / JS-heavy websites
- Python async stack; Go experience is a plus
- Experience with Playwright, Puppeteer, Selenium, Scrapy, browser automation, CDP
- Strong understanding of HTTP, TLS, HTTP/2, cookies, sessions, headers, proxies
- Redis, queues/message brokers, Docker, CI/CD, observability
- Kubernetes experience is preferred; deep K8s/SRE is a plus, not a hard blocker within this budget
- Ability to write clear technical documentation in English
- Ownership mindset: reliability, monitoring, incident debugging, cost control
- Participate in developing the company's key product from scratch
- High autonomy and influence on architecture and team decisions
- Budget for conferences, courses, and work tools
- Competitive compensation package
- Remote work flexibility
Apply now and become the technical leader who shapes our platform's future!
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search