Suggestions
Afaque Ahmad
Data Engineer @ QuantumBlack, McKinsey & Company - NUS
Afaque Ahmad is a Senior Data Engineer I at QuantumBlack, AI by McKinsey, based in Singapore.1 He has been with the company since September 2019, progressing from Data Engineer I to his current role.1 Afaque is passionate about big data and solving challenging problems.1
Professional Experience
Afaque's key accomplishments at QuantumBlack include:
- Designing and developing a data lake on AWS, creating over 15 pipelines processing 50K+ orders daily1
- Developing a Customer360 Feature Store on Hadoop, processing over 1TB of data for 100M customers1
- Creating a data quality framework to detect and report issues1
Skills and Expertise
Afaque has expertise in:
- Data lake design and architecture
- Large-scale feature engineering
- ETL pipeline development
- SQL query optimization
- Reporting automation
Additional Activities
Besides his primary role, Afaque:
- Creates video lectures on algorithms to help his peers1
- Started a YouTube channel in August 2023 focusing on Data Engineering content1
Certifications
Afaque holds several certifications, including:
- Machine Learning in Production (DeepLearning.AI, September 2024)1
- AWS Certified Cloud Practitioner (May 2021)1
- Astronomer Certified Apache Airflow Fundamentals (April 2023)1
- The Complete dbt (Data Build Tool) Bootcamp (December 2022)1
Afaque Ahmad is clearly a skilled and dedicated data engineer with a strong background in big data technologies and a passion for sharing knowledge in the field of data engineering.
Highlights
These 12 videos on Apache Spark will completely revolutionize your understanding forever.
-
Spark Query Plans: https://lnkd.in/gspmeQ-U
-
Spark DAGs: https://lnkd.in/gT63KYSn
-
Spark Memory Management: https://lnkd.in/g_Duijqw
-
Spark Executor Tuning: https://lnkd.in/ggRS8b7u
-
Shuffle Partitions: https://lnkd.in/gb4ccdje
-
Data Partitioning: https://lnkd.in/gCQHqWac
-
Bucketing: https://lnkd.in/gKMbwrbf
-
Caching: https://lnkd.in/gJPKpPF4
-
Data Skew: https://lnkd.in/geb4cFcp
-
Salting: https://lnkd.in/gRquuyZf
-
AQE & Broadcast Joins: https://lnkd.in/gM3es6xe
-
Dynamic Partition Pruning: https://lnkd.in/gD4be_5F
Please share/repost to help spread the word :)
#dataengineering #spark #apachespark #performancetuning #sparkoptimization
