Daniel B.Data Engineer
Skills
Data Engineer with over a decade of experience in the Information Technology sector, specializing in database management and Big Data environments. Expertise includes handling ETL/ELT processes leveraging technologies such as Apache Spark, Hadoop, and Hive.
Senior Data Engineer
6/1/2022 - 2/1/2024
Responsible for designing, developing, and maintaining robust solutions for data management and analysis. Designed and implemented efficient data pipelines for data collection, processing, and storage, while configuring and optimizing databases to ensure performance and scalability. Developed and maintained integrations between disparate data sources to guarantee data consistency and integrity, and collaborated with development and analysis teams to meet integration needs. Designed efficient data models to support analytical and business requirements, performed data transformations, and cleaned data to ensure its quality. Implemented process automation to improve operational efficiency and established monitoring systems to proactively identify and resolve issues. Worked closely with data scientists, software engineers, and other stakeholders to ensure successful project delivery, while implementing security practices and measures to protect sensitive data and ensuring compliance with relevant regulations related to data privacy and security. Developed proficiency in Python, SQL, and Pandas, as well as Big Data tools like Hadoop, Spark (PySpark, SparkSQL). Demonstrated expertise in data modeling and database design, and extensive experience in cloud environments such as AWS, Azure, and Google Cloud, specializing in data-focused services.Consulting, Data & Analytics - Financial Services - Latin America South
10/1/2021 - Present
Accumulated significant expertise as a Data Engineer, excelling in the planning and execution of data treatments, analyses, and transformations. Mastery was achieved in constructing and optimizing data pipelines utilizing Python, Spark, and SQL. Developed robust data processing workflows and ensured high-quality data management within the analytics platforms. Leveraged Spark's capabilities for large-scale data processing and used SQL for efficient data querying and transformation. The role required a strong understanding of data warehousing principles, which facilitated the design and implementation of efficient data storage solutions.Database Administrator
7/1/2021 - 12/1/2021
Developed proficiency in SQL and AWS RDS, ensuring seamless database management and monitoring related processes. Demonstrated expertise in AWS cloud services, implementing and managing RDS (Relational Database Service) to optimize database performance and reliability. Utilized advanced SQL techniques to perform routine database maintenance, troubleshooting, and optimization tasks. Ensured database security, integrity, and availability, adhering to best practices and industry standards.Geoprocessing Support Analyst
9/1/2020 - 2/1/2021
Developed proficiency in database consultation and customer service, focusing on producing solutions in SQL and GIS. Spearheaded project development using the ESRI tool - ArcGIS Pro and ensured the maintenance of scripts to support ongoing operations.Database Analyst
9/1/2018 - 9/1/2020
Acquired expertise in database monitoring and optimizing performance through routine database loading procedures. Contributed significantly to database documentation processes, ensuring comprehensive and accessible records. Gained proficiency in database systems and tools to streamline related processes and improve overall efficiency.
Information Systems at Anhanguera Educacional
2017 - 2021MBA - Strategic Business Management at Anhembi Morumbi University
2021 - 2022Postgraduate Diploma in Cloud Computing Projects and Architecture at Anhanguera Educational
2023 - 2024Database Administrator at Senac DF
2021 - 2021Postgraduate Degree in Computing and Information and Communication Technologies (ICT) at Anhanguera Educacional
2023 - 2023
Apache Airflow: Orchestrating Your First Data Pipeline at Alura
1/1/2024Python for Data Science: Functions, Packages, and Pandas at Alura
11/1/2023Apache Spark Training with Python at Alura
3/1/2023Microsoft Certified: Azure AI Fundamentals at Microsoft
3/1/2022Microsoft Certified: Azure Fundamentals at Microsoft
3/1/2022Microsoft Certified: Azure Data Fundamentals at Microsoft
3/1/2022Big Data & Analytics at FIAP
3/1/2022Database Administrator at Senac DF
8/1/2021Microsoft Power BI - Basic at Senac DF
6/1/2021Oracle One Training: Programming Logic at Alura
5/1/2021Scrum Foundation Professional Certificate - SFPC™ at CertiProf
7/1/2020Lifelong Learning at CertiProf
7/1/2020
Daniel is available for hire
Hire Daniel B.
All Howdy Candidates are vetted for skills and english proficiency.
