Sign In

Suneel Jakka

Software Engineer | Data | Analytics | AI/ML | Cloud | CRM

Suneel Jakka is a highly accomplished professional with a diverse educational background and a rich experience in the field of computer science and engineering.

He holds a Doctor of Philosophy - PhD in Computer Science from the University of the Cumberlands, a Data Mining Certificate in Statistical Data Mining from Stanford University, a Masters in Computer Applications from Osmania University, and a Bachelors in Computer Science from Sri Venkateswara University.

Suneel has a proven track record of working in reputable organizations, showcasing his expertise and leadership skills.

His professional journey includes roles such as Adjunct Professor at Southern New Hampshire University, Lead Engineer at Boeing, Staff Software Engineer/Lead at WarnerMedia, Principal Engineer/Architect at AT&T, Senior Engineering Manager at IBM, Principal Engineer at Infosys, Senior Software Engineer Team Lead at Fujitsu, Software Engineer at Wipro Limited, and Software Engineer at Birlasoft (India) Limited.

Highlights

Nov 20 · linkedin

I had to explain about stages, tasks & jobs within Apache Spark last week during one of my presentations. Here is my explanation regarding the same. (Next post will be about more internals for each of them)

Job: A job is the highest-level unit of execution in Spark. It represents a complete computation triggered by an action (e.g., count(), saveAsTextFile()). Jobs consist of multiple stages.

Stage: A stage is a set of tasks that can be executed in parallel. Stages are created based on the transformations in the Spark application's execution plan. A stage boundary is typically defined by operations that require data to be shuffled across the network, like reduceByKey() or groupByKey().

Task: A task is the smallest unit of work in Spark. Tasks are the actual computations that are performed on the data partitions. Each stage consists of multiple tasks, where each task operates on a subset of the data. Tasks within a stage can be executed in parallel across different executor nodes. Here's a simplified sequence:

When you perform an action, Spark creates a job.

The job is broken down into stages based on transformations and actions in the execution plan.

Each stage is divided into tasks that can be executed concurrently on different nodes in the cluster.

Tasks within a stage operate on partitions of the data in parallel.

This parallelism and distribution of tasks across stages and nodes contribute to Spark's ability to process large datasets efficiently in a distributed environment.

Aug 19 · linkedin

I was explaining the importance of data modeling other day in one of the discussions. Data modeling is important for both Data Warehouses and Data Lakes because it enhances data organization, accessibility, performance, and usability. For data warehouses, it supports efficient querying and reporting, while in data lakes, it aids in exploration and scalability. The specific approach to data modeling might vary between these two environments due to their distinct purposes and characteristics, but the underlying principles of good data modeling remain essential.

Related Questions

What are Suneel Jakka's main research interests?
How long has Suneel Jakka been an adjunct professor at Southern New Hampshire University?
What are some notable projects Suneel Jakka has worked on?
What is Suneel Jakka's educational background?
How does Suneel Jakka's work at Southern New Hampshire University differ from his work at Queensland University of Technology?
S
Add to my network

Location

Dallas-Fort Worth Metroplex