Hybrid Data Architect with Databricks, Guidewire
Dice · Bloomington, IL, Vereinigten Staaten Von Amerika · Hybrid
Details zum Jobangebot
Dice is the leading career destination for tech experts at every stage of their careers. Our client, PamTen Inc, is seeking the following. Apply via Dice today!
Experience with Insurance Domain , Must have experience with Guidewire.
Required Skills:
Experience with Insurance Domain , Must have experience with Guidewire.
Required Skills:
- 15+ years of experience in the IT/Technology and having at least 3+ years of experience in Azure/Databricks
- Design and architect Databricks-based solutions that align with business objectives, ensuring scalability, performance, and security.
- Designed, Development and implemented Data mesh using Delta Lake.
- Provide technical leadership and guidance to the Databricks development team, ensuring best practices are followed throughout the project lifecycle.
- Collaborate with cloud infrastructure teams to design and optimize the underlying infrastructure for Databricks workloads on platforms such as AWS or Azure
- Develop efficient data ingestion and ETL pipelines using Databricks, Apache Spark, and other relevant technologies.
- Integrate Databricks with data lakes and data warehouses to ensure seamless data access and analytics.
- Continuously monitor and optimize Databricks workloads for performance and cost-effectiveness.
- Implement and maintain security measures, including access controls, encryption, and compliance standards, to protect sensitive data.
- Create documentation and provide training to internal teams on Databricks best practices and usage.
- Stay up to date with the latest developments in Databricks and related technologies to recommend and implement improvements.
- Good knowledge on SQL Queries & Stored Procedures.
- Strong programming skills in PySpark, Python, writing complex queries in T-SQL
- Experience in transferring data from RDBMS to Data bricks using ADF.
- Expertise in using Spark SQL with various data sources like JSON, Parquet and XML.
- Experience in creating tables, partitioning, bucketing, loading, and aggregating data using Spark SQL/Python.
- Experience with Insurance Domain.
