Senior Data Engineer

We’re expanding our team!
DS STREAM is growing! We are looking for  SENIOR Data Engineers .

You will need at least one of these

• At least 5 years of commercial experience in Data Engineering with focus on large scale data pipelines.
• Proven track record using advanced solutions in Azure cloud and Databricks
• Previous exposure to GCP infrastructure and tools (Airflow, Pub/Sub, BigQuery, VertexAI)
• Knowledge of Spark Structured Streaming
• Proficiency in Python and SQL
• Ability to build pipelines using Github or similar tools  

Nice to have

• Ability to build enterprise API integrations with FastAPI or similar
• Experience in leading/mentoring engineers

[REMOTE] Senior Data Engineer   

We're expanding our AI Data Engineering Team at DS STREAM! 

This is a 100% remote role WITHIN POLAND .

Project description:

This initiative centers on a mature, high-availability data hub architected within Azure Databricks, utilizing Unity Catalog, Spark Structured Streaming, and Asset Bundles to ensure robust performance and governance. The solution is built upon the Medallion Architecture framework, providing a scalable foundation that is now being expanded to ingest, transform, and seamlessly integrate new data sources into the existing ecosystem.

You will get

• 100% remote work with the possibility to use an office in Warsaw
• Flexible working hours and engaging company events
• Access to Azure, GCP and Databricks certifications
• Possibility to use an in-house incubator to develop your own ventures
• A supportive community of 50+ senior data consultants with extensive experience in Data Warehouse and Big Data
• and much more!

,[Development of a program for processing large volumes of data , Ingesting data from APIs , Developing Databricks pipelines in PySpark / SQL, Handling Spark Structure Streaming pipelines , Designing and managing objects in Unity Catalog according to Medallion Architecture ] Requirements: Python, Docker, SQL, Databricks, PySpark, CI/CD, GCP, Airflow, BigQuery, API, FastAPI, Azure Databricks, Azure, AWS, Kubernetes Tools: Agile, Scrum. Additionally: Remote work, Competitive compensation, Sport subscription, Training budget, Private healthcare, Flat structure, Small teams, International projects, Free coffee, Bike parking, In-house trainings, In-house hack days, Modern office, Startup atmosphere, No dress code, Free beverages.