.NET Continuous Profiler: Memory usage
Datadog’s .NET memory profiler helps identify excessive garbage collection, allocation hotspots, and objects that remain in memory after collection. It combines CLR events, operating-system thread metrics, sampled allocation data, stack traces, and weak handles to provide production-friendly memory insights. The approach favors low overhead, though some capabilities depend on the .NET version. ## Measuring Garbage Collector CPU Impact - The profiler uses CLR events to monitor garbage collection phases. - In server GC mode, the CLR creates two high-priority threads per heap/core to process collections in parallel. - Since .NET 5, these threads are named `.NET Server GC` and `.NET BGC`. - At each profile export, the profiler retrieves these threads’ CPU usage from the operating system. - It records the result as a sample with a native stack containing a `Garbage Collector` frame. - This uses a pull model: the exporter periodically requests the CPU measurement because no suitable event or dedicated profiler thread exists. - Before .NET 5, GC thread CPU usage could not reliably be identified because `GCCreateConcurrentThread` did not include thread IDs. ## Sampling Allocations - Per-allocation callbacks such as `ICorProfilerCallback::ObjectAllocated` provide detailed data but significantly slow allocation fast paths. - `GCSampledObjectAllocation` and `ObjectsAllocatedByClass` reduce some costs but do not provide call stacks for individual allocation sites. - Datadog instead listens to `AllocationTick`, emitted for roughly every 100 KB allocated. - Each event includes: - The object’s `ClassID` and type information. - The allocation address. - The object size and total allocation size since the previous tick. - The allocation kind: SOH (`0`), LOH (`1`), or POH (`2`). - Generic type names are reconstructed through the .NET profiling API. - Because allocation events are synchronous, the current thread is responsible for the allocation; the profiler walks that thread’s stack to capture the allocation call site. - This produces sampled allocation data for each heap category without imposing the cost of observing every allocation. ## Tracking Objects That Survive Garbage Collection - An allocation address alone cannot track an object indefinitely because compacting garbage collections can move objects. - Datadog uses weak handles, created through `GCHandle.Alloc`, which move with objects and do not keep them alive. - The profiler added this functionality through the .NET 7 `ICorProfilerInfo13` API and its `LiveObjectsProvider`. - For every sampled allocation, it creates a weak handle and records the object’s creation time. - After each garbage collection: - Handles for unreachable objects are removed and destroyed. - Handles for surviving objects remain and are included in the next profile. - This lets users inspect representative objects that persist after collection and investigate potential memory leaks. ## Practical Recommendation Use allocated-memory profiles to find endpoints and types responsible for excessive allocation, then examine surviving-object samples for retention or leak investigations. GC CPU data is especially useful for diagnosing applications whose high CPU usage is driven by frequent or expensive garbage collections.
Read original(opens in new tab)