Gpu Computing | Techlist.io

awsJan 26, 20263 min readCurated summary

AWS Weekly Roundup: Amazon EC2 G7e instances, Amazon Corretto updates, and more (January 26, 2026) | Amazon Web Services

AWS’s January 26, 2026 roundup highlights new GPU infrastructure, Java updates, container optimization, expanded observability, and more flexible Amazon Connect workflows. The main launch is EC2 G7e, powered by NVIDIA Blackwell GPUs and designed for demanding AI inference, spatial computing, and scientific workloads. AWS also announced regional expansions and upcoming community and re:Invent-focused events. ## Amazon EC2 G7e Instances - Generally available in US East (N. Virginia) and US East (Ohio). - Powered by NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs. - Deliver up to 2.3× the inference performance of G6e instances. - Provide twice the GPU memory and support configurations with up to eight GPUs and 768 GB of total GPU memory. - Can run medium-sized models of up to 70 billion parameters using FP8 precision on a single GPU. - Target generative AI inference, spatial computing, and scientific computing. ## Amazon Corretto Security Updates - AWS released January 2026 quarterly security and critical updates for supported OpenJDK versions. - New releases include: - Corretto 25.0.2 - Corretto 21.0.10 - Corretto 17.0.18 - Corretto 11.0.30 - Corretto 8u482 - Updates provide current security patches and performance improvements for Java applications. ## Amazon ECR Layer Sharing - Amazon Elastic Container Registry now supports cross-repository layer sharing through blob mounting. - Common image layers can be reused across repositories rather than uploaded repeatedly. - This can speed up image pushes and reduce storage costs by storing shared layers once. ## CloudWatch Database Insights Expansion - On-demand Database Insights is now available in: - Asia Pacific (New Zealand) - Asia Pacific (Taipei) - Asia Pacific (Thailand) - Mexico (Central) - The machine-learning-powered feature helps identify database performance bottlenecks and recommends remediation steps. ## Amazon Connect Guided Experiences - Step-by-Step Guides now support conditional logic and real-time data updates. - Managers can configure interfaces that show or hide fields, change default values, and modify required fields based on earlier inputs. - Automatic refreshes from Amazon Connect resources help agents work with current information. ## Upcoming AWS Events - **Best of AWS re:Invent:** A free virtual event on January 28–29 featuring curated announcements, technical sessions, leadership insights, and live Q&A. - **AWS Community Day Ahmedabad:** A free, community-led conference on February 28, 2026, with technical talks, demos, networking, and real-world use cases. - AWS encourages builders to use the AWS Builder Center to discover additional virtual and in-person events. AWS customers working with AI should consider evaluating G7e instances, while Java teams should apply the latest Corretto updates. ECR users can also benefit from shared layers to improve container delivery efficiency and reduce costs.

Read original

Gpu Computing Generative AI AWS Amazon Ec2+4

discordOct 8, 20253 min readCurated summary

From Single-Node to Multi-GPU Clusters: How Discord Made Distributed Compute Easy for ML Engineers

Discord argues that distributed machine learning becomes practical when developer experience is treated as a first-class engineering problem. Ray provided the distributed-computing foundation, while Discord built a platform around it with a CLI, Dagster and KubeRay orchestration, and the X-Ray observability interface. This transformed GPU-intensive ML from manual experimentation into reproducible production pipelines, enabling Ads Ranking to move to multi-GPU neural networks and produce major business gains. ## Scaling Beyond Single-Node ML - Discord’s ML systems grew from simple classifiers to complex models serving hundreds of millions of users. - Teams needed: - Multiple GPUs for training - Datasets larger than a single machine - More compute than existing infrastructure could provide - Ray addressed the distributed-computing challenge, but Discord still needed a standardized internal platform to make it easy to use. ## Problems with Ad-Hoc Ray Clusters - Early ML engineers manually created Ray clusters using open-source documentation. - This led to: - Inconsistent cluster configurations - Uneven resource management - No centralized scheduling - Limited monitoring - Multiple teams independently rebuilding infrastructure solutions - Discord concluded that Ray needed an internal platform layer rather than direct, manual use. ## A Parameterized CLI for Cluster Creation - Discord replaced numerous GPU-specific YAML templates with one parameterized template. - Engineers specify requirements such as: - GPU type - Worker count - Memory - The CLI generates Kubernetes configuration, security settings, and hardware-specific resource requests. - It manages the full cluster lifecycle, including creation and deletion. - This made multi-GPU environments available through a single command and standardized deployments across teams. ## Automated Orchestration with Dagster, KubeRay, and Ray - Discord combined three systems: - **Dagster** defines workflows, dependencies, schedules, and validated configuration. - **KubeRay** dynamically provisions Ray clusters on Kubernetes with the appropriate namespace, service account, and GPU node pool. - **Ray** executes distributed training, evaluation, and batch inference. - The workflow is: 1. An engineer launches or schedules a Dagster pipeline. 2. Dagster submits the job specification. 3. KubeRay creates the required Ray cluster. 4. Ray distributes the workload across GPUs. 5. Logs and metrics flow back to Dagster and monitoring systems. - The approach provides predictable, reproducible jobs with centralized visibility. - Discord’s ad relevance model now trains daily without engineers manually editing cluster configurations. ## Centralized Observability with X-Ray - Discord built X-Ray as a web UI for monitoring Ray infrastructure. - It displays: - Active clusters - Cluster ownership - Machine types - Current status - Engineers can inspect dashboards and launch interactive notebooks for experimentation from one place. ## Ads Ranking as a Production Test - Ads Ranking determines which Quest advertisements are most relevant to individual users. - Before Ray, the system relied on XGBoost and lacked: - Model sharding - Multi-GPU support - Scalable, frequent retraining - Ray enabled sharded neural networks trained on multi-GPU clusters. - Reported results included: - Twice as many players joining Quests - Ad coverage increasing from roughly 40% to nearly 100% - A production pipeline that retrains daily and continuously delivers new model versions Discord’s experience suggests that distributed ML succeeds when powerful infrastructure is paired with simple interfaces, automated orchestration, and strong observability. Organizations adopting Ray should build comparable platform tooling around it rather than expecting ML engineers to manage clusters, scheduling, and monitoring themselves.

Read original

Gpu Computing Kubernetes Xgboost Ray+3

dropboxAug 27, 20252 min readCurated summary

Hack Week 2025: How these engineers liquid-cooled a GPU server

Dropbox engineers used Hack Week 2025 to build a liquid-cooling system for GPU servers, anticipating the rising heat and power demands of AI workloads. Their prototype, assembled from radiators, fans, a pump, reservoir, tubing, manifolds, and sensors, reduced temperatures by 20–30°C during stress tests compared with air cooling. The project also enabled quieter operation and may help Dropbox use less data-center space and energy as GPU requirements increase. ## Building a Custom Liquid-Cooling System - The team built the system from scratch after failing to source a complete commercial setup in time. - It replicated key data-center cooling components: - Radiators and fans for heat dissipation - A pump and reservoir to circulate coolant - Tubing and manifolds - Sensors to verify flow and monitor performance - Engineers tested the cooling loop before connecting expensive GPUs, then integrated it with a GPU server. ## Thermal and Noise Improvements - Liquid cooling lowered CPU and GPU temperatures by approximately 20–30°C during demanding torture tests. - Because the liquid loop handled the primary heat-generating components, the team could: - Remove some fans - Run others at lower speeds - Reduce noise and potentially power consumption - Remaining airflow was directed toward lower-heat components such as DIMMs and network cards. - The team considered using an airflow baffle to cool those components more precisely. ## Preparing for AI-Era Infrastructure - High-end GPUs increasingly consume more power and generate more heat, potentially making air cooling insufficient. - Liquid cooling could allow Dropbox to: - Fit more powerful servers into existing data-center footprints - Avoid spreading servers across additional space - Reduce cooling energy and operating costs - Although vendors do not yet universally require liquid cooling for top-tier GPUs, the engineers expect that requirement to become more common. - Dropbox’s growing focus on AI workloads provided additional motivation to investigate the technology early. ## Hack Week as an Experimentation Platform - The project received funding and organizational support from Dropbox’s infrastructure teams. - Hack Week gave engineers time to explore a long-term infrastructure problem outside their normal work. - The team’s Learn Fast award recognized the project’s emphasis on experimentation and rapid learning. - Working in person helped the engineers exchange ideas, troubleshoot quickly, and collaborate with colleagues across the company. Dropbox plans to expand testing with additional liquid-cooling labs in multiple data centers. The prototype is an early step toward infrastructure capable of supporting increasingly powerful, GPU-intensive AI systems.

Read original

Gpu Computing Ai Infrastructure Liquid Cooling Thermal Management+3