aws·Jun 16, 2026·3 min readCurated summary

Amazon S3 annotations: attach rich, queryable context directly to your objects | Amazon Web Services

Amazon S3 annotations let organizations attach rich, mutable context directly to objects without rewriting the underlying data. Each object can have up to 1,000 named annotations, with a combined limit of 1 GB, in formats such as JSON, XML, YAML, or plain text. The annotations move with objects during copying and replication, can be queried through S3 Metadata tables, and are intended to support large-scale analytics and AI-driven workflows. ## Richer Metadata at Object Scale - Annotations address limitations of existing S3 metadata features: - System metadata describes properties such as size and storage class. - Object tags support operational tasks but are limited to 10 immutable tags. - User-defined metadata is limited to approximately 2 KB of headers and must be supplied during upload. - Annotations can be modified or deleted independently without rewriting the object. - Each annotation has a unique name and can hold up to 1 MB, with up to 1 GB total per object. - Supported content includes structured formats such as JSON, XML, and YAML, as well as plain text. ## Supporting AI and Industry Workflows - AI-generated transcripts, summaries, classifications, ratings, and technical specifications can remain attached to the relevant data. - Media companies can store transcripts, moderation results, subtitles, licensing information, and media specifications alongside video assets. - Financial institutions can attach investment summaries and sentiment analysis to research documents for autonomous discovery. - Life sciences organizations can add regulatory status, cohort information, and approval chains to clinical-trial data. - Annotations remain available for objects in archival storage classes without restoring the objects or paying retrieval charges. ## Object Lifecycle and Synchronization - Annotations automatically move with objects during copy, replication, and cross-Region transfers. - S3 removes annotations when the associated object is deleted. - This reduces dependence on separate metadata databases or sidecar files and avoids synchronization overhead. - Different teams or enrichment processes can maintain separate annotations on the same object without overwriting one another. ## Working with Annotations - IAM or bucket policies must grant: - `s3:PutObjectAnnotation` - `s3:GetObjectAnnotation` - The main APIs are: - `PutObjectAnnotation` to create or update an annotation - `GetObjectAnnotation` to retrieve one annotation - `ListObjectAnnotations` to view all annotations on an object - `DeleteObjectAnnotation` to remove an annotation - Reusing an annotation name with `PutObjectAnnotation` updates its content. - For multipart uploads, annotations are added after the upload is completed. - The example uses separate `mediainfo` JSON and `ai_summary` text annotations on a video object. ## Querying Through S3 Metadata - When S3 Metadata is enabled, annotations automatically populate managed annotation tables. - These tables can be queried with Amazon Athena and other analytics engines. - The S3 Tables MCP server provides a standardized interface for AI models and agents to query annotation data using natural-language workflows. - This enables discovery across large collections of S3 objects without retrieving the objects themselves. S3 annotations are most useful when object context is large, frequently updated, and needed for cross-object search or AI automation. Organizations should use them to keep business and machine-generated metadata directly associated with the data while using S3 Metadata tables for scalable querying.

Read original(opens in new tab)

MetadataAI AgentsAmazon S3Json+3

figma·Dec 15, 2025·2 min readCurated summary

Chat, Are We Cooked? How Language Has Become the New Metric of Virality | Figma Blog

Social media algorithms are reshaping language by turning words into signals of virality. Linguist Adam Aleksic argues that platforms influence not only how people speak, but also how they understand identity, culture, and one another. While this linguistic creativity is not inherently harmful, users should recognize how algorithms can strip terms from their original contexts and spread ideological ideas. ## Products Shape Communication and Identity - Interfaces communicate values as well as functionality, reflecting Marshall McLuhan’s idea that “the medium is the message.” - Tinder’s swiping system commoditizes dating and reinforces associations such as “right” being positive and “left” being negative. - Dating platforms encourage users to perform and categorize themselves according to available features: - Hinge prompts users to present a structured story. - Grindr encourages identity categories such as “bear” or “twink.” - These design choices can affect both outward self-expression and people’s internal understanding of identity. ## Algorithms Accelerate Linguistic Change - Online communities create slang, while recommendation algorithms expose it to much larger audiences. - This produces “context collapse”: users encounter words outside the communities and cultural settings where they originated. - Platform metadata now includes every word users write or say, not just hashtags, making language itself a mechanism for discovering and promoting content. - Influencers can popularize expressions whose origins their audiences may not recognize. ## From Ballroom Slang to Mainstream Vocabulary - Terms such as “slay,” “serve,” “queen,” “cooked,” “ate,” “bussin’,” and “it’s giving” have roots in African American and queer ballroom culture. - Historically, slang spread gradually through connected social groups; algorithms now accelerate that process dramatically. - As words become mainstream, their original connotations may weaken or change. - Aleksic emphasizes the importance of understanding these origins rather than treating viral slang as context-free vocabulary. ## Memes as Cultural and Ideological Vehicles - Aleksic is less concerned with linguistic change itself than with the ideas carried by newly popular terms. - Expressions from online subcultures, including incel communities, can enter mainstream usage through meme circulation. - Terms such as “looksmaxxing” and “-pilled” may spread because memes make unfamiliar or troubling ideas more accessible. - Memes can effectively “Trojan-horse” concepts into broader culture, though critical awareness can reduce their potential harm. Users should enjoy the flexibility and creativity of evolving language while remaining attentive to where viral terms come from, what meanings they lose, and which beliefs they may carry into the mainstream.

Read original(opens in new tab)

MetadataUser InterfaceAlgorithmsSocial Media Algorithms+2

toss·Oct 30, 2025Original article

Toss People: Designing a structure (opens in new tab)

Data architecture is evolving from a reactive "cleanup" task into a proactive, end-to-end design process that ensures high data quality from the moment of creation. In fast-paced platform environments, the role of a Data Architect is to bridge the gap between rapid product development and reliable data structures, ultimately creating a foundation that both humans and AI can interpret accurately. By shifting from mere post-processing to foundational governance, organizations can maintain technical agility without sacrificing the integrity of their data assets. **From Post-Processing to End-to-End Governance** * Traditional data management often involves "fixing" or "matching puzzles" at the end of the pipeline after a service has already changed, leading to perpetual technical debt. * Effective data architecture requires a culture where data is treated as a primary design object from its inception, rather than a byproduct of application development. * The transition to an end-to-end governance model ensures that data quality is maintained throughout its entire lifecycle—from initial generation in production systems to final analysis and consumption. **Machine-Understandable Data and Ontologies** * Modern data design must move beyond human-readable metadata to structures that AI can autonomously process and understand. * The implementation of semantic-based standard dictionaries and ontologies reduces the need for "inference" or guessing by either humans or machines. * By explicitly defining the relationships and conceptual meanings of columns and tables, organizations create a high-fidelity environment where AI can provide accurate, context-aware responses without interpretive errors. **Balancing Development Speed with Data Quality** * In high-growth environments, insisting on "perfect" design can hinder competitive speed; therefore, architects must find a middle ground that allows for future extensibility. * Practical strategies include designing for current needs while leaving "logical room" for anticipated changes, ensuring that future cleanup is minimally disruptive. * Instead of enforcing rigid rules, architects should design systems where following the standard is the "path of least resistance," making high-quality data entry easier for developers than the alternative. **The Role of the Modern Data Architect** * The role has shifted from a fixed, corporate function to a dynamic problem-solver who uses structural design to solve business bottlenecks. * A successful architect must act as a mediator, convincing stakeholders that investing in a 5% quality improvement (e.g., moving from 90 to 95 points) provides significant long-term ROI in decision-making and AI reliability. * Aspiring architects should focus on incremental structural improvements, as any data professional who cares about how data functions is already operating on the path to data architecture.

MetadataData GovernanceData ArchitectureData Modeling+4

datadog·Jan 9, 2024·4 min readCurated summary

.NET Continuous Profiler: Under the hood

Datadog’s .NET profiler is designed for continuous, low-overhead production monitoring rather than occasional diagnostic runs. It collects CPU, wall time, exceptions, lock contention, and allocation data, aggregates it into compact `.pprof` files, and links profiles to traces and services through runtime metadata. The post introduces the architecture and emphasizes preserving application performance as a central design requirement. ## What a Continuous Profiler Does - Profiling analyzes runtime performance and method call stacks. - It complements APM, which focuses on request latency, throughput, and errors. - The profiler also measures: - CPU usage - Wall time and method duration - Exceptions - Lock contention - Memory allocations and potential leaks - Unlike tools such as PerfView, dotTrace, dotMemory, and Visual Studio profilers, Datadog’s profiler is intended to run continuously in production with negligible overhead. - Continuous profiling avoids the need to recreate production traffic, security settings, hardware, and load in a separate environment. ## Datadog’s .NET Profiler Architecture - The profiler is composed of specialized profilers for different resource types. - Each profiler includes: - A sampler that collects raw data - A provider that exposes the collected samples - An aggregator combines samples from all profilers. - An exporter serializes the data into Google’s `.pprof` format and uploads it through the Datadog Agent. - Datadog’s backend processes the profiles for visualization and analysis. ## Sample Aggregation and Storage Each sample contains: - A call stack made up of method frames - Key-value labels, such as thread identifiers - A numeric value vector representing measurements like CPU consumption or wall time Samples with identical call stacks and labels are merged, and their numeric values are added together. This reduces duplication and produces smaller profile files—for example, repeated exceptions from the same code path and thread can be stored as one aggregated sample. The aggregation and `.pprof` serialization code is implemented in Rust and shared across Datadog’s Ruby, PHP, and other runtime profilers. ## Connecting Profiles to Traces and Services - Each uploaded profile includes process ID, host name, and runtime ID metadata. - The runtime ID uniquely identifies a .NET service running within a process. - This is important because a single .NET process can host multiple services, such as separate IIS applications running in different AppDomains. - The tracer communicates the mapping between runtime IDs, AppDomains, and service names. - Service names come from `DD_SERVICE`; if it is unset, the process name is used. - Datadog sends one profile per runtime ID every minute, so multiple profiles from one process may share a timestamp while representing different services. - Runtime IDs allow the backend to associate profiles with the correct traces and spans. ## Making .NET Call Stacks Easier to Read The .NET profiling API can expose compiler- and runtime-generated names that differ from the original source code. Datadog rewrites these frames to make visualized call stacks more understandable. - Constructors named `.ctor` are displayed using the class name. - Compiler-generated anonymous methods are rendered as the enclosing method followed by `_AnonymousMethod`. - Lambdas and local methods use an enclosing-method name with the `_Lambda` suffix. - Nested named methods such as `<DefiningMethodName>g__InnerMethodName|yyy_zzz` are displayed as `DefiningMethodName.InnerMethodName`. - Compiler-generated state-machine methods such as `MoveNext` are mapped back to the original source-level type and method names. ## Native and Managed Implementation Considerations - The team considered using Microsoft’s `TraceEvent` NuGet package to receive and parse CLR events in C#. - That approach would execute managed profiling code on the same CLR as the application being profiled. - Allocations made by the profiler could therefore increase garbage-collector pressure. - The post begins discussing how this performance concern influenced the implementation, but the provided excerpt ends before that design is explained. A production profiler must not only collect useful data but also minimize the memory and CPU costs of collecting it. Datadog’s architecture addresses this through specialized samplers, aggregation, compact serialization, runtime-aware trace association, and source-oriented call-stack cleanup.

Read original(opens in new tab)

MetadataDatabase DesignDatadogDotnet+4