AWS Databricks Engineer- Remote-6 Months - Must be authorized to work in the U.S. without sponsorship.

AWS Databrick Data Engineer
Remote
6months Contract/Contract to Hire
 
Must be authorized to work in the U.S. without sponsorship. No EAD''s
 
An AWS Databricks Data Engineer to design, build, and maintains scalable data pipelines and lakehouse architectures using Apache Spark, Python, and SQL. Optimize cloud storage, ensure data quality, and integrate seamlessly with other AWS services

Core Responsibilities

  • Pipeline Development: Design and build batch and streaming data pipelines using PySpark, Delta Lake, Autoloader and Delta Live Tables (DLT) to ingest data sets into Databricks.
  • AWS Integration: Build cloud-native data solutions leveraging AWS services (e.g., S3 for storage, IAM for access management, and AWS Glue or Lambda for serverless tasks
  • Data Governance & Security: Configure and manage access controls using Databricks Unity Catalog to ensure compliance and monitor lineage
  • Orchestration & CI/CD: Automate pipeline deployments using Databricks Workflows, Apache Airflow, and CI/CD tools (e.g., GitHub, GitLab). 
  • Cross-Functional Collaboration: Partner closely with stakeholders to build robust feature stores and prepare datasets for various consumption needs

 

Typical Qualifications & Technical Skills

  • Industry: Healthcare Payer industry experience. Have worked on MMIS data sets – claims, provider, member enrollment and similar data sets
  • Experience: 3-5 years of hands-on data engineering experience, specifically on Databricks.
  • Programming: High proficiency in Python (specifically PySpark) and advanced SQL.
  • Big Data & Cloud: Strong understanding of Apache Spark, Data Lakehouse architecture, and working within a production AWS environment.
  • Databricks Ecosystem: Familiarity with the Databricks platform ecosystem, including notebooks, Delta Lake, and Unity Catalog.
Back to blog

Other Jobs To Apply

Upto $200K/Year | Inpatient 7-On/7-Off | Dual-Certified PMHNP/FNP | Remote

Lead M&A Delivery

CAD Implementer (REMOTE-US)

Scientific Account Manager - Stem Cell Solutions- REMOTE

Regional Safety Leader - Remote Power & Energy

Sr. Program Manager (Healthcare)

Senior/Lead Software Engineer

Hubspot Marketing Consultant (Remote)- CONTRACT

Remote Sales Administrative Assistant

Remote eBay Expert

Electrical or Mechanical Commissioning Agent (Travel Required/Remote)

Senior Big Data Engineer: Snowflake + dbt (Remote)

Remote Senior Bioinformatics Manager, Genomics & Clinical

Business Process Analyst at General Dynamics: 100% Remote (LA)

Solution Architect – Agentic AI

NetSuite ARM Implementation Consultant - Senior Associate

PMT Engineer - Analytics and Automation

Payments & Product Partnerships Counsel, Americas

Financial Consultant I - Principal Connection

Dist Design Project Coord - Sr

Market Commentary Writer and Social Editor at Wealthfront: 100% Remote (San Francisco, CA, New York City, NY)

Global Total Rewards Lead

Remote Payroll Specialist | Flexible Hours & Growth

Entry-Level Auto Technician: Hands-On Tire & Maintenance

Senior Associate, Risk and Structuring Analytics

Services Project Coordinator

Software Developer/Senior Software Developer

Manager, Professional Services- Streaming Media

Varonis Careers - Technical Account Manager

Manager, Accounts Payable

Monday.com Consultant[remote]

Remote Electrical Engineer — Critical Power Systems

Platform Engineering & Reliability Specialist (Azure)

Remote Patient Scheduling Specialist (Part-Time, 20hrs/wk)

Remote Retirement Inside Sales Specialist

Remote SAP MM Consultant – Entry Level

Client Support Specialist - Greater Philadelphia, PA Area (REMOTE)

Customer Service Specialist I at Aon Plc: 100% Remote (Fort Washington, PA, US)

Services Project Coordinator

Appeals Nurse Consultant (Remote)

Servicing Travel Advisor III - Chase Travel (Remote - Tempe, AZ)

Programmatic Solutions Manager, Uber Advertising

Manager, Flight Maintenance - Traveling

Customer Service / Pizza Maker (01068) 28803 Northwestern Hwy, Southfield, MI

Guidewire Architect - PC/CC/BC

Notary Stamp Order Online | West College Corner, IN

Senior Analyst, Program Cost Controls (Remote)

Sr. Tech Business Developer, Just Walk Out Technology by Amazon - AWS Solutions

Medical Consultant

SENIOR DIGITAL SALES EXECUTIVE (REMOTE/USA) - KSLA (GDM - GRAY MEDIA GROUP)