Luiz V.Platform Architect
Skills
An experienced Data Engineer with over seven years in data engineering and three years in web development, specializing in data platform architectures, ingestion, transformation, modeling, quality, and governance. Holds a Master's degree in Computing from Universidade Federal Fluminense and possesses proficiency in Big Data tools, SQL, Python, and Spark. Proficient in architecting and implementing data platform architectures and pipelines, with certifications in major platforms and languages such as GCP, AWS, Python, and MySQL.
Technical expertise spans across cloud platforms AWS and GCP, and programming languages including Python, GO, Bash, PHP, SQL, and PL/SQL. Skilled in handling a variety of database technologies such as MySQL, Redshift, BigQuery, Athena, Presto, PostgreSQL, Hive, MongoDB, and DynamoDB. Adept in leveraging various technologies like Spark, Glue, Nifi, Pentaho, EMR, Dataproc, the Hadoop suite, HDFS, S3, Google Storage, Airflow, Luigi, Crontab, Step Functions, Kafka, Pub/Sub, Docker, docker-compose, Looker, Tableau, GoodData, Selenium, and BeautifulSoup. Further demonstrates technical skills through contributions documented in a GitHub repository.
Sr. Data Engineer
1/1/2023 - 11/1/2023
Responsible for implementing DataMesh tools and processes for the People Latam team, including the configuration, adaptation, and maintenance of the data platform to enable seamless data ingestion and transformation. Developed proficiency in using Google Cloud Platform (GCP), with particular expertise in BigQuery and Google Cloud Storage (GCS). Leveraged Python extensively for data manipulation and transformation processes.Data Engineer
2/1/2021 - 2/1/2022
Led the project and implementation of a comprehensive data platform encompassing a data lake, warehouse, quality monitoring, streaming, and data mesh architecture. Utilized a robust stack including GCP, Dataproc, Airflow, Google Storage, BigQuery, MySQL, PostgreSQL, and MongoDB. Developed advanced data engineering skills and built scalable, resilient data pipelines. Enhanced data processing capabilities and ensured data integrity across the platform. Leveraged cloud technologies to optimize data storage and retrieval, and implemented efficient workflows for data ingestion and transformation.Data Engineer
12/1/2018 - 3/1/2021
Implementation of the data platform, encompassing lake, warehouse, and quality modules, was successfully executed. Demonstrated proficiency in handling ingestion and transformation pipelines, ensuring robust documentation through the use of catalog and data dictionary tools. Technical expertise included working extensively with GCP and AWS as cloud service providers, and leveraging orchestration capabilities with Airflow. Managed data processing using EMR, and database management systems such as Oracle and MySQL. Additionally, showcased adeptness in utilizing NiFi for data flow automation, AWS Glue for ETL processes, and Redshift as the data warehousing solution.Data Engineer
12/1/2015 - 12/1/2018
Part of the data engineering team, implementing ETLs for the data lake and modeling the data warehouse in Looker. Developed proficiency in AWS, MySQL, and Redshift, with extensive experience using Hadoop in an on-premises environment. Demonstrated expertise in data visualization and modeling in Looker.Web Developer
12/1/2012 - 11/1/2015
Developed comprehensive web systems with an emphasis on PHP and MySQL. Acquired proficiency in PHP and Python, while utilizing Locaweb for hosting solutions. Demonstrated robust capabilities in managing and querying databases with MySQL and PostgreSQL.Web Developer
9/1/2011 - 12/1/2012
Developed and maintained web systems with a focus on PHP and MySQL, utilizing tools and services from Hostgator. Gained expertise in PHP and ASP.Net, ensuring robust and scalable backend solutions. Applied comprehensive MySQL skills to manage and optimize database performance.
Economics at Pontifical Catholic University of São Paulo
2014 - 2020Computer Science at Federal Fluminense University - UFF
2021 - 2022
Google Cloud Platform Big Data and Machine Learning Fundamentals at google
4/1/2021Data Lakes and Data Warehouses with GCP at google
3/1/2021Streaming Analytics Systems on GCP at google
2/1/2021Batch Data Pipelines on GCP at google
1/1/2021
Luiz is available for hire
Meet Luiz V.
All Howdy Candidates are vetted for skills and english proficiency.
