PHP 8: Observability baked right in
PHP’s observability mechanisms failed to keep pace with Zend Engine improvements in PHP 7 and PHP 8, especially the introduction of JIT. Existing hooks imposed significant runtime costs, created compatibility and stability problems, and limited tracers such as Datadog’s ability to evolve. PHP 8 addressed these issues by introducing a new observer API designed specifically for modern, lower-overhead runtime instrumentation. ## Observability Before PHP 8 ### The `zend_execute_ex` VM Hook - Extensions could override `zend_execute_ex` to intercept every PHP-defined function and method call. - This moved PHP calls onto the native C stack, whose limited size (`ulimit -s`) could cause stack overflows and process crashes. - Every userland call was intercepted, even when an extension only needed to observe a subset, adding overhead to call-heavy applications. - The compiler could no longer use optimized distinctions between userland and internal calls, such as `DO_UCALL` and `DO_ICALL`. - Extensions had to manually forward the hook to other extensions, creating “noisy neighbor” problems, unexpected behavior, and possible crashes. - The hook was incompatible with PHP 8’s JIT compiler. ### Custom Opcode Handlers - Extensions could replace handlers for function-call opcodes, avoiding the native-stack problem associated with `zend_execute_ex`. - These handlers still required careful forwarding to neighboring extensions, which was historically unreliable. - Handlers could mutate VM state—for example, preventing the original opcode from running—making reliable cooperation between multiple extensions impossible in some cases. - Generators could not be fully instrumented through custom opcode handlers. - Like `zend_execute_ex`, custom opcode handlers were incompatible with the PHP 8 JIT. ### Zend Extension Hooks - Zend Extensions had privileged access to engine-level function-call begin and end handlers. - This approach caused the compiler to emit `EXT_FCALL_BEGIN` and `EXT_FCALL_END` around every function call. - The additional opcodes introduced too much overhead for production-grade tracing. ### AST Injection Experiments - Researchers explored injecting observability nodes into the abstract syntax tree during compilation. - These nodes could invoke tracing functions before and after calls. - However, injecting instrumentation around every function call was expected to have overhead comparable to Zend Extension hooks. - No production-ready tracers using this approach were known at the time. ## The Need for a New Observer API - Existing hooks forced observability tools to interfere deeply with VM execution or compiler output. - Their limitations included excessive overhead, stack-safety risks, incomplete generator support, extension conflicts, and JIT incompatibility. - These constraints prevented tools such as the Datadog PHP tracer from taking full advantage of PHP 8. - In response, the authors and the PHP internals community developed and shipped the observer API in PHP 8, providing a foundation for more modern and efficient tracing, profiling, and debugging. PHP 8’s observer API was necessary because older instrumentation techniques were either unsafe, too slow for production, difficult to compose, or incompatible with the JIT. A runtime-level observability mechanism designed alongside the engine is a more sustainable approach than modifying VM hooks, opcodes, or compiled syntax from extensions.
Read original(opens in new tab)