How Go 1.24's Swiss Tables saved us hundreds of gigabytes (opens in new tab)

Go 1.24 initially caused a Go runtime regression that increased RSS across Datadog services, but some high-traffic workloads ultimately used substantially less memory. The reduction came from Go 1.24’s Swiss Tables map implementation, which made a large, mostly read-only routing cache more compact. Profiling also revealed opportunities to reduce memory further by removing redundant data from the cached values.

The Unexpected Memory Reduction

• Datadog observed roughly 500 MiB less live heap in the shardRoutingCache map after upgrading to Go 1.24.
• With GOGC=100, that translated to approximately 1 GiB less total memory usage.
• Even after accounting for an expected 400 MiB RSS increase from the mallocgc regression, the service achieved a net reduction of about 600 MiB.
• The improvement was most visible in high-traffic environments because they contained larger routing caches.

The shardRoutingCache Data Structure

• The cache maps routing keys to shard information:

  map[string]Response
  

• Each Response contains:

  • ShardID int32
  • ShardType
  • RoutingKey string
  • LastModified *time.Time

• The map is populated mainly during service startup by querying a database.

• It is rarely modified afterward, making its memory layout and initial allocation particularly important.

• The routing key is stored both as the map key and again inside the value, creating potential redundancy.

Estimating Memory per Entry

• On a 64-bit system, a map key’s string header occupies 16 bytes.
• The value requires approximately:
  • 4 bytes for ShardID
  • 8 bytes for ShardType
  • 16 bytes for the RoutingKey string header
  • 8 bytes for the LastModified pointer
• The value totals 36 bytes before alignment, or roughly 40 bytes with padding.
• Including the key header, each key-value pair requires about 56 bytes, excluding the separately allocated string and time.Time data.

Go 1.23 Bucket-Based Maps

• Go 1.23 maps used hash tables organized into an array of buckets.
• The number of buckets was always a power of two, and each bucket contained eight slots.
• Reads and writes required scanning the slots in the selected bucket to find a matching key or an empty position.
• When a bucket filled, Go added linked overflow buckets, which increased memory usage and made lookups more expensive.
• Map growth occurred when the average load factor exceeded 13/16, or 6.5 of 8 slots.
• The map then allocated twice as many buckets.
• To avoid a large latency spike, growth was incremental: old and new bucket arrays coexisted while entries were gradually moved during subsequent writes.

Why Workload Shape Matters

• The routing cache is populated in a startup-heavy phase and then primarily read.
• Such a workload benefits from a compact map representation because it does not need frequent insertions or growth.
• Differences in cache size and traffic patterns explain why the memory improvement was significant in some environments but not uniform across the fleet.

Go 1.24’s Swiss Tables implementation can substantially reduce memory usage for large, stable maps, even when another runtime change causes RSS growth. Teams should profile real production heaps after Go upgrades and inspect large structs for duplicated strings, unnecessary pointers, and other avoidable per-entry overhead.