Engenheiro de Dados - Híbrido

Saia na frente:

CANDIDATE-SE

Vaga ofertada por parceiro do Jobbol:
Job Title: Sr Data EngineerJob Type: Long-time based job opportunityLocation: Hybrid in Sao Paulo BrazilJob Description:Job Title: Data EngineerJob Summary: We are seeking a skilled Data Engineer to design, develop, and maintain scalable data pipelines and workflows.
The ideal candidate will have strong expertise in Python, SQL, Client, and Airflow, with experience in building ETL/ELT solutions and optimizing data infrastructure.
This role involves collaborating with data analysts, scientists, and business stakeholders to ensure data availability, reliability, and efficiency.
Key Responsibilities: Design, build, and maintain scalable ETL/ELT pipelines to process large volumes of structured and unstructured data.Develop and optimize SQL queries and transformations within Client to enable efficient data storage and retrieval.Implement workflow orchestration using Apache Airflow to automate data processing tasks.Write efficient, reusable, and scalable Python scripts for data extraction, transformation, and loading (ETL).Monitor and troubleshoot data pipelines to ensure high availability and performance.Collaborate with data teams to define data modeling best practices and maintain a clean, well-structured data warehouse.Work with cloud platforms (AWS, GCP, or Azure) to integrate data sources and manage cloud-based data infrastructure.Ensure data security, governance, and compliance with industry best practices.
Required Skills & Qualifications: Strong programming skills in PythonExpertise in SQL for data querying, transformation, and performance tuning.Hands-on experience with Client (schema design, performance optimization, Client features like Snowpipe, Streams, and Tasks).Experience with Apache Airflow for scheduling and orchestrating data pipelines.Knowledge of ETL/ELT processes and best practices in data engineering.Experience with cloud platforms (AWS, GCP, or Azure) and their data services.Familiarity with data modeling (Star Schema, Client Schema, etc.) and data warehouse concepts.Experience with version control systems like Git and CI/CD pipelines.
Preferred Skills: Experience with big data processing frameworks (Spark, Databricks).Knowledge of Kafka, Kinesis, or other real-time data streaming tools.Familiarity with containerization (Docker, Kubernetes) for deploying data pipelines.Understanding of Data Governance, Data Quality, and Data Security principles.
Education & Experience: Bachelor s or Master s degree in Computer Science, Data Engineering, or a related field.
6-10+ years of experience in which 3+ years of experience in a data engineering role.