Lead Data Engineer Job at WorkHQ, Los Angeles, CA

b3NUSk9DWnQ3NFU5Z3lhazBuU21ocDFiRHc9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Shift work,

Similar Jobs

Uniquely Feline Cat Sitting

Cat Sitter Job at Uniquely Feline Cat Sitting

 ...great opportunity for individuals who have a passion for animals and enjoy spending time with them. Duties: -Provide in-home pet sitting services for various types of animals, including cats, fish, and small mammals -Follow specific instructions provided by pet... 

Ace IT Careers

Entry-Level QA Tester (Remote) Job at Ace IT Careers

 ...Join our dynamic team as an Entry-Level QA Tester (Remote) at Ace IT Careers ! This is a fantastic opportunity for recent graduates or anyone eager to launch their career in software quality assurance. You will work remotely with leading employers dedicated to quality... 

Oracle

Consulting Technical Manager Job at Oracle

 ...areas of regulatory compliance, identity governance, web access management, information security, data security and enterprise security...  ....We're committed to including people with disabilities at all stages of the employment process. If you require accessibility assistance... 

C&S Wholesale Grocers

Experienced, multi-stop Class A CDL Driver Job at C&S Wholesale Grocers

 ...Benefits start on Day one! Tuition Reimbursement Guaranteed $1000/week during training period Paid every Wednesday Utilizing the specialized skills developed with experience and the C&S Professional Driver Training Program, you will operate a Commercial... 

SW16 Logistics LLC

SW16 Amazon Delivery Driver Job at SW16 Logistics LLC

 ...DSP dedicated to providing exceptional delivery service to our community.We are a customer...  ...deliver Amazon packages. Our Delivery Drivers strive to get every Amazon order to the...  ...part of a team Must work at least one weekend day per week. Saturday or Sunday Ability...