Suggestions
Leo Chashnikov
Senior Software Engineer at Meta
Leo Chashnikov is a Senior Software Engineer at Meta (formerly Facebook) with over 10 years of experience in software development.1 He currently specializes in Data Engineering, focusing on constructing and optimizing data processing pipelines.1
Some key points about Leo Chashnikov:
Professional Background
- Works as a Software Engineer at Meta, with a focus on Data Engineering12
- Has been in the software development field for over 10 years1
- Previously worked on projects involving Big Data analysis using Scala and Spark2
Expertise and Interests
- Experienced in Scala, Python, and Java programming languages5
- Knowledgeable in Big Data processing and Functional Programming5
- Holds certifications in Kubernetes Application Development (CKAD) and Data Science2
Contributions and Thought Leadership
- Actively shares insights on Data Engineering through articles and posts3
- Wrote an article titled "Navigating the Data Engineering Landscape: From Raw Data to Insights"3
- Provides guidance on transitioning from Software Engineering to Data Engineering1
Personal Background
- Originally from Ukraine, now based in London6
Leo Chashnikov is passionate about technology and has transitioned from an interest in computer games to developing them and programming in general.4 He now leverages his expertise to contribute to Meta's data infrastructure and shares his knowledge with the broader tech community.
Highlights
I've heard about 1 Billion Row Challenge (#1BRC) couple of days ago. The idea is quite simple - you have a file with 1 billion weather measurements, you need to read and aggregate it as fast as possible. This challenge was proposed by Gunnar Morling on January 1st, and became quite popular since then. You can find all the details here - https://lnkd.in/e3s6KJhF.
I'm far from being able to seriously participate there, but I decided to at least go through some of the solutions, find optimizations that are broadly applicable in other contexts, while still being easy to read and understand.
Just to make sure that I understand these ideas well enough I've tried making some notes in this "explainer" post, along with some chunks of code - https://lnkd.in/evfrQ5rv Any feedback is welcome, and I hope to read and watch many posts from top contestants on how they approached the problem and reached their solution.
Always feel uncomfortable sharing some posts I wrote, but that one I actually even like, so why not.. Tried compiling a small list of "usual suspects" for optimization opportunities in Apache Spark applications. By no means exhaustive, but might serve as a good starting point or checklist
