Dataclass vs namedtuple performance: cause and fix

Unexpected slowdown when using Python dataclasses often appears in production pipelines that also manipulate pandas DataFrames, where the class objects are instantiated millions of times. The overhead stems from the generated init and attribute handling, leading to higher latency compared to lightweight namedtuple instances.

# Example showing the issue
import timeit
from collections import namedtuple
from dataclasses import dataclass

# Simple payload
Payload = namedtuple('Payload', ['a', 'b', 'c'])

@dataclass
class PayloadDC:
    a: int
    b: int
    c: int

# Benchmark creation of 1,000,000 objects
nt_time = timeit.timeit('Payload(1, 2, 3)', globals=globals(), number=1_000_000)

dc_time = timeit.timeit('PayloadDC(1, 2, 3)', globals=globals(), number=1_000_000)

print(f"namedtuple creation: {nt_time:.6f}s")
print(f"dataclass creation: {dc_time:.6f}s")
# Output shows dataclass ~2x slower than namedtuple

Dataclass generates an init method, repr, eq and stores attributes in a dict by default, which adds per‑instance overhead. namedtuple is a thin tuple subclass with fixed slots, so attribute access is essentially index lookup. This behavior follows CPython’s implementation details for classes versus tuple subclasses and often surprises developers who expect dataclass to be as lightweight as a tuple. Related factors:

  • dict allocation per instance
  • Descriptor overhead for type checking
  • Lack of slots in plain dataclass

To diagnose this in your code:

# Quick check with timeit to spot the slowdown
import timeit
print('Running a micro‑benchmark...')
print(timeit.timeit('Payload(0,0,0)', globals=globals(), number=500_000))
print(timeit.timeit('PayloadDC(0,0,0)', globals=globals(), number=500_000))

Fixing the Issue

If raw speed matters, switch to a namedtuple or enable slots on the dataclass.

# Simple one‑liner fix
Payload = namedtuple('Payload', ['a', 'b', 'c'])

For production code that still wants the dataclass ergonomics, use the slots flag to eliminate the per‑instance dict:

from dataclasses import dataclass

@dataclass(slots=True)
class PayloadDC:
    a: int
    b: int
    c: int

The slots version removes the dictionary overhead while keeping generated methods and type hints. Add a sanity check in CI to ensure the dataclass remains fast enough for your workload:

import timeit, sys

def benchmark(cls, n=1_000_000):
    return timeit.timeit(f'{cls.__name__}(1,2,3)', globals=globals(), number=n)

nt = benchmark(namedtuple('Tmp', ['a','b','c']))
dc = benchmark(PayloadDC)
assert dc <= nt * 1.5, f'Dataclass too slow: {dc}s vs {nt}s'

This approach keeps the readability of dataclasses while preventing the silent performance penalty in high‑throughput scenarios.

What Doesn’t Work

❌ Adding slots after the class definition: slots must be defined at class creation time, otherwise Python ignores them.

❌ Replacing dataclass with a regular class and manually writing init: you re‑introduce boilerplate and still keep the dict overhead.

❌ Using pandas.apply to convert rows to dataclass instances: apply is vectorised for pandas but still incurs the same per‑instance cost, negating the intended speed gain.

  • Using a plain dataclass for millions of records without slots.
  • Assuming dataclass is always as fast as a tuple because of its syntactic sugar.
  • Neglecting to benchmark; relying on intuition rather than measurement.

When NOT to optimize

  • Small scripts: Creating a few hundred objects has negligible impact.
  • One‑off data migrations: Execution time is dominated by I/O, not object creation.
  • Prototyping in notebooks: Clarity outweighs micro‑optimisation during exploration.
  • Already using slots: If you already declared slots=True, the extra speed gain is minimal.

Frequently Asked Questions

Q: Can I keep dataclass features and still get namedtuple speed?

Yes, declare @dataclass(slots=True) to drop the per‑instance dict.

Q: Is the slowdown only in object creation?

Creation is the biggest hit; attribute access also slows slightly without slots.

Choosing the right lightweight container can shave seconds off a pipeline that processes millions of rows. While dataclasses bring readability, slots give you the best of both worlds without the hidden cost. Always benchmark the critical path before committing to a data structure.

→ Why Python dict hash collisions degrade lookup performance → Why attrs outperforms dataclass in object creation speed → Why numpy object dtype hurts pandas performance → Fix How cffi vs ctypes impacts performance