Edgar Morel

Project: Enhancement of analytical platform

• Migration of key business data across multiple environments, ensuring data accuracy and consistency.
• Automated migration workflows using Python, Oracle PL/SQL, SQL, and Java, preserving complex data relationships.
• Developed solutions with Oracle APEX for software development, data analysis, and reporting, enhancing decision-making processes.
• Optimization of queries through in-depth analysis, significantly improving overall efficiency.

Technologies used: Oracle, SQL, PL/SQL, Python, Git, Java, APEX.
Project Outcome: Enhancement of centralized data warehouse for optimizing and speeding report retrieval and data migration of key production tables.

Project : Analytics Platform for Cervantes Institute - Spanish Government

• Led ETL development at the Cervantes Institute (IC), enhancing data processes across departments
• Managed ETL processes for On-Prem legacy databases in Academic, Culture, Administration, Accounting, and Marketing departments
• Developed an automated PySpark-SQL-YAML-based ETL module that reads from Hadoop HDFS files, creates Dimensions and Fact Tables, constructs DataMarts, handles schemas automatically, simplifies SQL maintenance, tracks data lineage, utilizes the Great Expectations and Pytest libraries for QA automation and logging, generates logs for ETL tracking, and simplifies documentation
• Created a Golden Record Module using PySpark and Python, and implemented Master Data Management (MDM) to ensure data consistency, accuracy, and usability across all areas of the Cervantes Institute.

Technologies used: Cloudera,CDH,Apache Nifi,PySpark, Python, SQL, MSSQL,Oracle 11g,Hadoop,HDFS,Hive,PowerBI,Apache Atlas,Trino,IBM Cloud Pak for Data,Iceberg,Presto/Trino, Avro,Parquet, QA: PyTest, Great Expectations (GE)

Project Outcome: A Data Analytics Platform that is used by more than 300 people on Cervantes Institute offices on more than 40 countries and 100 centres.

Project : ETL development, Data Warehouse maintenance and Data Mart development.

• Developed and maintained large-scale data pipelines using Databricks, PySpark, and Delta Lake to process over 100 TB of data.
• Implemented robust ETL processes, data quality checks (Great Expectations), and QA automation (PyTest).
  Migrated legacy pipelines and functions to Azure Data Factory, integrating with Azure services like Blob Storage and Data Lake Gen 2.
• Optimized job parallelization using Azure Batch, ensuring efficient data processing.
• Collaborated in agile environments (Scrum/Kanban) and produced reports with Power BI for data analytics and decision-making.

Technologies used: AWS, AWS Glue, Hive, Trino, Starburst, Delta Lake, Python, PySpark, Databricks, Ascend.io, Power BI, PyTest, Great Expectations (GE), Azure Data Factory (ADF), Azure Functions, Azure Batch, Azure Blob Storage, Azure Data Lake Storage Gen 2.

Project Outcome: Creation and maintenance of pipelines that ingests more than 100 Terabytes of data from multiple sources.

Project: Data Warehouse maintenance and Data Mart development.

• ETL development from multiple sources (DBMS, API, Web)
• Reporting solution development using Oracle PostgreSQL, Excel, and Tableau
• Machine Learning models for Antichurn and Recommendation systems.

Technologies used: Oracle, MSSQL, PostgreSQL,Tableau, QGIS,R,Python,SQL

Project Outcome: Development of new Data Marts, machine learning models, and analytical tools for use by senior management and the marketing department.

Project: Data Warehouse maintenance and Data Mart development.

• ETL process from multiple sources (DBMS,Web)
• Data modeling and implementation of anti-churn campaigns
• Customer segmentation using machine learning techniques
• Go To Market expansion using Hot Spot Analysis with MapInfo Pro, SQL, and PostGIS
• Communication of findings and results to Management and Directors
• Participated in migration of On Premise Data Warehouse to Off Premise Data Lake.

Technologies used: Oracle, Python, R, SQL, MapInfo, PostGis, Excel, Tableau

Project Outcome: Development of analytical tools adopted by over 50 users across senior management and commercial sectors.

Project: Data Warehouse maintenance and ETL monitoring

• Maintenance of existing pipelines
• Monitor of existing pipelines.

Technologies used: Oracle 11g, SSIS, MSSQL , PLSQL, Excel

Project Outcome: Maintenance of data pipelines serving over 30 stakeholders, including senior management and commercial departments.

Project: ERP maintenance

• Maintenance of ERP modules
• Creation of new features for ERP.

Technologies used: Oracle 11g, PLSQL, Excel, Java Web

Project Outcome: Development of an analytical platform utilized by 10 professionals in commercial and purchasing departments.