Senior Data Engineer - Databricks Specialist

Hays IT Contracting is a collaboration based on B2B principles. Our company adapts IT specialists to the most interesting technological projects on the market. Join the circle of 500 satisfied contractors and work for Hay’s clients! For our client we are currently looking for:

Senior Data Engineer – Databricks Specialist

Location: remote (Poland) with one visit per month in Warsaw

Contract type: B2B (with Hays Poland)

Length: first contract for 12 months

Start date: ASAP/1 month 
Rate: 170-220 pln/h (net+vat)

General information:  You will be a hands-on technical lead responsible for designing, building, and owning end-to-end data solutions on our Azure Databricks platform.

Tasks:

• Own end-to-end delivery of data engineering solutions: from scoping and architecture through development, testing, deployment, support and continuous improvement.
• Design and build scalable Databricks-based solutions using Workflows, Jobs, Notebooks, Spark Declarative Pipelines (SDP), Delta Lake and Medallion Architecture (Bronze/Silver/Gold).
• Implement CDC/CDF patterns, data ingestion frameworks and appropriate orchestration mechanisms (scheduled, file-arrival, continuous or API-driven).
• Manage and govern data assets through Unity Catalog, including access management, permissions, service principals and data governance policies.
• Deliver solutions through CI/CD pipelines using Declarative Automation Bundles (DAB), GitHub Actions or Azure DevOps, following engineering best practices and clean branching strategies.
• Collaborate with business stakeholders and BAs to translate requirements into scalable technical solutions.
• Define and promote engineering standards around code quality, testing, documentation, reusability and production support.
• Work closely with QA throughout the delivery lifecycle, ensuring acceptance criteria, data quality validation and testing are built into the solution from design through production release.
• Mentor team members and support performance tuning, troubleshooting and operational excellence across the platform.

What we expect from you:

• Minimum 5 years of experience in a similar position
• Proficiency in English at B2+ level
• Technical knowledge and experience:
• Databricks & Data Engineering
  • Strong hands-on experience with Databricks Workflows, Jobs, PySpark, Python and Spark Declarative Pipelines (SDP).
  • Practical knowledge of CDC/CDF implementation, Delta Lake, Unity Catalog and Lakehouse architecture.
  • Experience configuring pipeline triggers, orchestrating workloads and managing production-grade data pipelines.
  • Experience with Databricks Asset / Declarative Automation Bundles (DAB), source control, pull requests and code review processes.
• Azure & Data Platform
  • Azure Data Lake Storage Gen2 (ADLS Gen2), Azure Key Vault and Azure SQL/T-SQL.
  • Strong understanding of Medallion Architecture, data platform design principles and modern data engineering tooling.
  • Python ecosystem for data engineering, including PySpark, Delta and testing frameworks.
• Engineering Practices
  • CI/CD using GitHub Actions or Azure DevOps.
  • Infrastructure as Code (Terraform or Pulumi).
  • Test-Driven Development, automated testing and software design best practices (including SOLID principles).
  • Agile delivery methodologies.
  • AI-assisted development, using AI tools throughout analysis, implementation, testing, code review and documentation activities.
  • Spec-driven development, including working against agreed specifications, testable acceptance criteria and traceability between requirements, implementation and delivered outcomes.
• Nice to have :
  • Degree in Computer Science, Mathematics or related field.
  • Databricks certification (Associate or Professional).
  • Experience with Structured Streaming, Kafka or Event Hub integrations.
  • Data Warehousing and Kimball dimensional modelling.
  • Power BI, including semantic models and DirectLake connectivity.
  • Financial services domain knowledge (investment, fund accounting or regulatory reporting).
  • Experience with Logic Apps, Power Automate, MongoDB or other NoSQL technologies.
  • MCP (Model Context Protocol), LLM API integrations (OpenAI, Anthropic) and AI-powered data engineering solutions.
  • Experience building Agentic AI workflows and production-grade AI-enabled automation.

What will the recruitment process look like:

• Your CV will be verified by Hays Recruiter
• Recruiter will contact you by phone - a 15-minute conversation about the project and your experience
• Technical conversation with the client (2-3 stages)
• Offer
• Welcome to the project!
  
   

Hays Poland sp. z o.o. is an employment agency registered in a registry kept by Marshal of the Mazowieckie Voivodeship under the number 361.