We are seeking a highly skilled Databricks Platform Engineer with strong experience in data engineering. The candidate will have a deep understanding of both data platforms and software engineering, enabling them to effectively integrate and operate the platform within a broader IT ecosystem.
This role requires a hands-on individual contributor who takes full ownership of deliverables end-to-end, including design, development, testing, deployment, and ongoing support.
- Manage and optimize Databricks data platform including workspace setup, cluster policies, job orchestration, Unity Catalog, cost controls, multi-tenancy.
- Design, write and maintain APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
- Ensure high availability, security, and performance of data systems which includes access control, secrets management, RBAC, monitoring, alerting, RLS, incident handling, performance tuning.
- Provide valuable insights about the data platform (Databricks) usage which includes cost attribution, usage analytics, workload patterns, telemetry.
- Implementing new features of Databricks, including serverless, Declarative Pipelines, Agents, lakebase , etc.
- Design and maintain system libraries (Python) used in ETL pipelines and platform governance (Databricks).
- Optimize ETL Processes - Enhance and tune existing ETL processes for better performance, scalability, and reliability.
MUST
- Minimum 10 Years of experience in IT/Data.
- Minimum 5 years of experience as a Databricks Data Platform Engineer.
- 3+ years of experience in designing, writing, and maintaining APIs used for Platform automation, Serverless workflows, Deployment pipelines, release management and repository management
- Bachelor's in IT or related field.
- Infrastructure & Cloud: Azure, AWS (expertise in storage, networking, compute).
- Programming: Proficiency in PySpark for distributed computing.
- minimum 4 years of Python experience for ETL development.
- SQL: Expertise in writing and optimizing SQL queries, preferably with experience in databases such as PostgreSQL, MySQL, Oracle, or Snowflake.
- Data Warehousing: Experience working with data warehousing concepts and Databricks platform.
- ETL Tools: Familiarity with ETL tools & processes
- Data Modelling: Experience with dimensional modelling, normalization/denormalization, and schema design.
- Version Control: Proficiency with version control tools like Git to manage codebases and collaborate on development.
- Data Pipeline Monitoring: Familiarity with monitoring tools (e.g., Prometheus, Grafana, or custom monitoring scripts) to track pipeline performance.
- Data Quality Tools: Experience implementing data validation, cleaning, and quality frameworks, ideally Monte Carlo.
- Containerization & Orchestration: Docker, Kubernetes.
- Infrastructure as Code (IaC): Terraform.
- Understanding of Investment Data domain (desired).
Luxoft DXC
2000
Luxoft, a DXC Technology Company is a global digital strategy and software engineering firm with over 17,000 international employees within its 58 offices in 29 countries. It is headquartered in Zug, Switzerland.
In January 2019, Luxoft was acquired by U.S. company DXC Technology. Luxoft partnered with LG Electronics to create a next-generation Autonomous Mobility concept vehicle that integrates consumers' personalized digital lifestyles into a driving experience. Luxoft enabled Switzerland's first Blockchain based e-vote platform with the City of Zug and Hochschule Luzern's Blockchain Lab.
Luxoft, a DXC Technology Company is a world-renowned company. It has been present on the Polish market for over 13 years. We have offices in Krakow, Warsaw, Wroclaw, and Gdansk. We employ over 2,000 professional experts carrying out projects for over 100 clients from the financial, automotive, medical, tourist industries, etc. We work for many international clients, including the USA, Great Britain, and Switzerland.
So far, Luxoft Poland has made a name for itself as a company that offers work on innovative projects, we offer various experiences in the field of IT, opportunities for rapid development, an extensive training program, and attractive benefits for employees.
At present, 62% of Luxoft Poland employees come from Poland, and 38% from around 50 countries, including Ukraine, Brazil, India, Turkey, Spain, Portugal, Italy, Romania, USA, etc.
At Luxoft, a DXC Technology Company almost 80 percent of employees are experts with the "Senior" experience level, with at least five years of experience. We care about our employees, so every day we try to provide them with the best possible conditions for work and development.
Technology is our passion! We focus on top engineering talent means that you will be working with the best industry professionals from around the world. Because of that, Luxoft is a global family with an epic atmosphere – we love what we do!