👋 Hello, I'm

Shashank Dey

Associate Data Engineer & Creative Problem Solver

I design, develop, and optimise large-scale ETL applications and streaming architectures using Databricks, AWS, Spark, and Snowflake. Passionate about building innovative solutions that make a difference.

View My Work Get In Touch

Professional Experience

My journey through the tech industry

Jan 2025 - Present

Data Engineer | Associate

Goldman Sachs

- Streamlined Snowflake ingestion by integrating with Databricks Autoloader, resulting in $100K annual savings in computation costs
- Optimised streaming architecture performance and reliability, achieving 50% faster execution through targeted optimisations and robust monitoring / alerting systems
- Upgraded DBR versions across all workflows, resolving compatibility issues for diverse libraries / modules by developing custom init scripts

Jul 2022 - Dec 2024

Data Engineer | Analyst

Goldman Sachs

- Developed and managed automated streaming ingestion workflows for 50+ diverse services, processing over 250 datasets with Databricks Autoloader and robust data re-processing capabilities
- Engineered and executed large-scale ETL pipelines, ingesting 150+ Delta tables into Snowflake and successfully migrating 250+ Iceberg tables to Delta Lake
- Automated the generation of critical Customer & Account extracts for FDIC reporting and developed data mappings (Alloy) between logical models and physical datasets

Jan 2022 - Jun 2022

Data Engineer | Intern

Sigmoid Analytics

- Built a Live Tweets Ingestion engine using Twitter API fetching Covid Tweets to build a sentiment analysis model
- Used Kafka and Spark streaming to ingest the data in real-time, python for data cleansing and transformations, and MongoDB to store the data
- Encrypted the data with pymongo, orchestrated on Airflow, and drew insights using matplotlib and seaborn

Featured Projects

Some of my personal projects, more to come in future!!

🐦

Covid Tweet Analyser

A Live Tweets Ingestion engine using Twitter API fetching Covid Tweets to build a sentiment analysis model

Python Kafka Spark MongoDB Twitter API

GitHub

📍

Nearby Places

A mobile application that helps you find nearby places based on your current location

Java Android Google Maps API

GitHub

⛅

Weather App

A web application to fetch the current weather of a city input from user

HTML CSS JavaScript OpenWeatherMap API

GitHub View

🎵

Music Player

A music player application that allows you to play music infinitely on loop

HTML CSS JavaScript

GitHub View

👁️

Face and Eye Detection

A real-time face and eye detection system capturing facial features from images using Haar Cascades

Haar Cascades Python Jupyter Notebook

GitHub

Skills & Technologies

Tools and technologies I work with

Languages

Python

Expert

C++

Advanced

JavaScript

Intermediate

HTML / CSS

Advanced

REST API

Advanced

Database & Cloud

MongoDB

Advanced

PostgreSQL

Advanced

AWS

Intermediate

Snowflake

Expert

Airflow

Expert

Tools & Technologies

Databricks

Expert

Spark

Expert

Streaming

Expert

Git

Advanced

VS Code

Advanced

Get In Touch

Let's connect and discuss Tech

📧

Email

Email Address

💼

LinkedIn Profile

📱

Phone

+91 7014051345

💻

GitHub

GitHub Profile

📍

Location

Hyderabad, India