Stack vs Heap: Where Does My Code Live?

Starving in Abundance: The Forgotten Memory

In my Operating Systems class, I learned about ‘Memory Structure’. Terms like Stack and Heap appeared on exams, and I memorized them just enough to pass.

But honestly, until I graduated, I never took memory seriously. We live in an era where personal laptops come with 16GB or 32GB of RAM as standard. For undergraduate-level assignments, running out of memory was virtually impossible. Plus, with languages like Java handling cleanup via the Garbage Collector (GC), I felt no need to care about memory addresses.

Drunk on the abundance of hardware and the convenience of high-level languages, I jumped into the professional world having completely forgotten the ‘Sense of Memory’—a fundamental skill for any developer.

The Painful Lesson from Production

When practicing for coding tests in school, I mostly dealt with arrays holding a few integers. The input size rarely exceeded 100,000. But production was different. Even in a small startup, the Database (DB) held millions of real customer records.

In production, to handle this massive data conveniently, we use a tool called ‘ORM (Object Relational Mapping)’. It is a grateful tool that lets you manipulate data like Java objects without writing raw SQL queries. (Hibernate is the standard in the Java world).

The problem was that this tool was ‘too convenient’.

With just one button press, it fetches DB data into a List. I failed to gauge the weight of the data hidden behind that convenience. I wrote a single line saying ‘Get Customer Info’, but I had unknowingly scooped up tens of thousands of customer records and their order histories into the memory.

The result was catastrophic. The 32GB RAM filled up instantly. The server suffered from ‘Severe Lag’ as the GC tried frantically to free up space, and eventually, it froze.

Familiar Stack, Alien Heap

Staring at the error logs vomited by the server, my eyes locked onto one phrase.

java.lang.OutOfMemoryError: Java heap space

The word ‘Stack’ was quite familiar. I learned it ad nauseam in Data Structures class, and it is the name of the site developers visit daily (Stack Overflow). I knew as common sense that writing a bad recursive function causes the Stack to explode.

But the culprit killing my server in production was never the Stack. The error logs always pointed to the ‘Heap’.

‘The Stack is not full, but the Heap space is insufficient?’

A question flashed through my mind. I know the Stack, but what exactly is the Heap that crashes production so often? Is it the same as the Heap data structure? Why is my code tormenting the Heap instead of the Stack?

That question led me back to dusty textbooks and Google. And I learned a crucial fact: The ‘Memory (RAM)’ where my code lives is not a single studio apartment. It is a ‘Segregated Space’ strictly divided by purpose.

The Workbench and The Warehouse

Let’s return to our ‘Digital Fulfillment Center’ worldview. Here, ‘RAM’ is the space where the Worker (CPU) spreads out items to work. But for efficiency, this space is strictly divided into two operations.

1. Stack: The Worker’s Personal Workbench

• Feature: A narrow, high desk right in front of the Worker (CPU).
• Usage: Temporarily holds variables (local variables, parameters) needed for the ‘current task (function)’.
• Lifespan: When the task (function) is done, the desk is wiped clean (Pop). Management is incredibly easy and fast.
• Metaphor: Ingredients on a cutting board. Once the chopping is done, they are moved immediately.

2. Heap: The Public Warehouse

• Feature: A massive warehouse behind the workbench. It has lots of space, but it takes time to retrieve items.
• Usage: Stores data that is large or needs to be kept for a long time (Objects, Instances).
• Lifespan: It remains until the user throws it away or the Janitor (Garbage Collector) cleans it up.
• Metaphor: The refrigerator or pantry. Ingredients stay there even after cooking is finished.

[Code Verification] Proving Two Worlds with Errors

Is memory really divided like this? We can prove the existence of these two spaces by intentionally causing errors with code.

1. Exploding the Stack (StackOverflowError)

I said the Stack is a ‘Workbench’. The workbench is small. If a function keeps calling itself (recursion) without ending, the paperwork piles up to the ceiling and eventually collapses.

public class StackTest {
    public static void recursiveCall(int depth) {
        // Infinite Recursion: 
        // The function never ends and keeps piling up on the Workbench (Stack)
        System.out.println("Stack Depth: " + depth);
        recursiveCall(depth + 1);
    }

    public static void main(String[] args) {
        recursiveCall(1);
    }
}

Result: It runs for a few thousand times and then dies with a StackOverflowError. Even if the Heap space is empty, if the Stack (Workbench) is full, the program dies.

2. Exploding the Heap (OutOfMemoryError)

Now, let’s recreate my production nightmare. Similar to loading tens of thousands of objects via ORM, I will keep creating massive lists and shoving them into the Warehouse (Heap).

import java.util.ArrayList;
import java.util.List;

public class HeapTest {
    public static void main(String[] args) {
        List<byte[]> warehouse = new ArrayList<>();
        
        while (true) {
            // Keep creating 1MB data and stacking it in the Warehouse (Heap)
            // Real-world scenario: Fetching massive rows from DB without paging
            warehouse.add(new byte[1024 * 1024]);
        }
    }
}

Result: java.lang.OutOfMemoryError: Java heap space. This is not a Workbench (Stack) problem. The Warehouse (Heap) has no more space to store items. This is the moment we visually confirm how our code torments the Heap.

The Janitor: Garbage Collector (GC)

Here lies the critical difference. The Stack is emptied ‘automatically’ when the function ends. You don’t need to care. But the Heap keeps accumulating trash unless someone cleans it.

In older languages like C, developers had to manually clean up using the free() command. If you forgot, the warehouse would fill with trash and explode (Memory Leak). Modern languages like Java, Python, and JS have hired a professional cleaner called the ‘GC (Garbage Collector)’.

‘Hey, nobody is using this object anymore?’ The GC periodically roams the Heap, finds ownerless objects, and throws them away. Thanks to this, we don’t have to write memory deallocation code.

But there is no free lunch. The moment the GC does a ‘Grand Cleaning’, all operations in the fulfillment center stop temporarily (Stop-the-world). The phenomenon where a game suddenly lags or the server freezes for a second is often due to this cleaning time.

Closing Thoughts: Eyes to See the Invisible

After understanding Stack and Heap, the code on my monitor started to look different. Previously, when I saw new Student(), I just thought, ‘I made an object’. But now I see it clearly.

‘A box has just entered the Warehouse (Heap). This will eat up memory until I remove it (or the GC comes).’

Since developing this ‘Eye to See the Invisible’, I escaped from vague fears. When the server vomits an OutOfMemory error, I don’t panic and hit the reboot button like before. Instead, I calmly ask, ‘Which object conquered the Heap?’ and open the analysis tools. Because once you know the cause, you can fix it.

We have now conquered the space where code lives (Memory). The warehouse is perfectly ready. Then, who are the actual ‘Workers’ moving and assembling items in this warehouse? What is the difference between working alone and working together (Multi-Tasking)?

In the next post, let’s journey into the world of ‘Process and Thread’—the flower of Operating Systems and the eternal homework of backend developers.