Kate Drogaieva: Data Engineer Portfolio
Data WarehouseInsurance
Data Warehouse Modeling |
DevOps
|
CI/CD
pipeline to deploy DW schema changes in Redshift based on AWS CodePipeline, CodeBuild, FlyWay and JUnit for testing
CI/CD pipeline
based on GitHub actions, FlyWay and DBT test
Data Pipelines
|
Extracting
from AWS Aurora into staging tables in MS SQL Server, transforming in SQL and Pentaho
Data Integration, loading into AWS Redshift with post processing in stored
procedures
Extracting from
AWS Aurora loading into Redshift via Fivetran;
transforming in Matillion and Redshift Stored Procedures
|
|
|
This DBT
package provides a materialization that builds advanced version of slowly
changing dimension type 2 (scd2)
|
|
Simplified
version of insurance policy transactions modeling and transforming in Postgres
database
|
DBT SCD2 from historical data and incremental changes not in order |
Simplified
version of insurance policy transactions modeling and transforming in Snowflake
Downloading
market daily data from Yahoo! Finance's API, reporting growing stocks using structural
breaks in DBT Python model and Snowflake SQL.
Loading
stocks fundamental data from GuruFocus using Rest API
based on recent changes detected from scrapped webpages.
Data Feeds and Analysis (advanced SQL) |
Combining data
from three different transactional systems, implementing several levels of data
aggregations, applying capping, cumulative multiplication.
Datasets for
modeling insurance rates across California for Auto, Home, and Landlord
products.
Helpdesk
tickets surveys monthly summaries and sentiments
Helpdesk
tickets surveys monthly summaries and sentiments
Helpdesk
tickets monthly forecast
Data GovernanceImplementation of Atlan Data catalog |
Tableau: Product Performance Dashboards50+ very complex calculations based on analysis transactional data, analytical function, different levels of aggregation and sophisticated capping rules. The calculation is performed in Redshift store procedures, views and Tableau dashboards. |
|
|
|
|
|
|
|
|
|
|
Looker Studio: Incidents Management
Dashboards
Helpdesk
tickets trends and executive summaries |
|||
Python and Machine Learning |
|
|
|
|