Senior Databricks Data Engineer | AWS, PySpark & SQL
Senior Databricks Data Engineer | AWS, PySpark & SQL
Detalles del trabajo
País
["Sao Paulo"]
Rol de puesto
Technology Lead - Brazil
Trayectoria profesional
Estado / Región / Provincia
Sao Paulo
Ubicación de trabajo
Brazil
Habilidades
Others (Please specify in the Requisition notes)
Dominio
Delivery
Grupo de interés
Infy Brazil
Compañía
INFOSYS TECNOLOGIA DO BR
ID de solicitud
155913BR
Habilidades técnicas 1
Habilidades técnicas 2
Habilidades técnicas 3
Habilidades técnicas 4
Habilidades técnicas 5

Infosys Brazil is looking for professionals with a Senior Databricks Data Engineer | AWS, PySpark & SQL profile to join our team.


Required Qualifications

  • Strong experience in Data Engineering and Databricks environments.
  • Advanced proficiency in Python, PySpark, and SQL.
  • Experience with Databricks and Delta Lake, including jobs, workflows, Medallion Architecture, Delta Live Tables/Lakeflow, and Asset Bundles.
  • Strong knowledge of relational and dimensional data modeling.
  • Experience developing ETL/ELT, CDC, and batch data pipelines.
  • Hands-on experience with AWS services such as Amazon S3, AWS Glue, Amazon EMR, Amazon Athena, AWS Lambda, AWS DMS, and AWS Step Functions.
  • Experience with Azure Synapse, including SQL Pools and data pipelines.
  • Knowledge of Git and CI/CD practices applied to Data Engineering.
  • Experience with data quality, testing, reconciliation, and automated validation processes.
  • Ability to work independently and collaborate effectively with technical and business stakeholders.
  • Strong communication and interpersonal skills.

Preferred Qualifications

  • Experience with Data Warehouse-to-Lakehouse migration programs.
  • Knowledge of Unity Catalog, including permissions, data lineage, governance, and data contracts.
  • Experience with DataStage, particularly analyzing existing jobs for reverse engineering.
  • Knowledge of automated testing, Data Quality gates, and data validation between legacy and modern environments.
  • Experience translating business rules from legacy ETL tools into Spark-based solutions.

Roles and Responsibilities

  • Design, develop, test, and maintain scalable batch data pipelines using PySpark and Python.
  • Build and operate Databricks and Delta Lake solutions based on Bronze, Silver, and Gold architecture layers.
  • Develop data transformations and support relational and dimensional data modeling.
  • Migrate and rebuild legacy ETL pipelines using Spark, including CDC and batch ingestion from relational databases.
  • Support the reverse engineering of Azure Synapse pipelines and SQL Pools.
  • Implement automated testing, data quality controls, reconciliation processes, and validation between legacy and new environments.
  • Apply Git and CI/CD practices to data pipeline development and deployment.
  • Work with AWS data services to build reliable and scalable data solutions.
  • Collaborate with technical teams and business stakeholders to ensure successful solution delivery.

Additional Information
Employment Type

  • Full-time, permanent position (CLT).


Work Model

  • Hybrid.


Benefits

  • Transportation allowance.
  • Meal voucher.
  • Childcare assistance.
  • Life insurance.
  • Funeral assistance.
  • Medical insurance.
  • Dental insurance.


Career Opportunities

  • Opportunity to work on global projects and international environments.
  • Exposure to Software Quality Assurance and project governance practices.
  • Collaboration with multicultural and cross-functional teams.
  • Professional development within a global technology organization.


Language Requirements

  • Advanced English proficiency required for interaction with international teams and stakeholders.

Additional Required Qualifications

• Bachelor’s degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• This position may require relocation and/or travel to work/project location.
• Candidates authorized to work for any employer in the United States without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role now or in the future.

Additional Details

About Us
Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem.

EEO
Infosys provides equal employment opportunities to applicants and employees without regard to race; color; sex; gender identity; sexual orientation; religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability.

Compartir

Trabajos similare

Conéctate con nosotros