It's all about the angle: Your photos, re-composed
Google’s new Auto frame feature in Google Photos re-composes photos by changing the apparent camera viewpoint after capture. It combines 3D scene reconstruction with generative AI to adjust framing, reduce wide-angle distortion, and generate background areas that were outside the original frame. The result is a more natural perspective while preserving the visible content and subjects’ identities. ## Why Traditional Editing Falls Short - Cropping and zooming cannot change the original camera perspective or parallax. - They also cannot reveal areas hidden outside the captured frame. - This is especially limiting for portraits, selfies, and “almost perfect” photos with poor framing or lens distortion. ## Two-Stage 3D-Aware Editing - **3D scene and camera estimation** - A machine-learning model creates a 3D point map from the 2D image using monocular depth estimation. - Each pixel is associated with an estimated 3D surface point. - The system estimates the original focal length and reconstructs human faces and bodies carefully to reduce identity-distorting artifacts. - Semantic analysis detects subjects’ face positions and 3D orientations to determine better camera parameters. - **Generative completion and retouching** - Classical 3D rendering produces an image from the modified camera position, orientation, and focal length. - Moving the virtual camera exposes “holes” where the original image contained no information. - A latent diffusion model fills these missing regions and corrects the rendered result. - Classifier guidance with regional scaling helps preserve original content while allowing generation in newly revealed areas. ## Automatic Framing and Distortion Correction - The system automatically selects camera parameters for improved portrait composition. - It can adjust both camera pose and intrinsic properties such as focal length. - For wide-angle selfies, it detects perspective distortion that enlarges features close to the lens. - By effectively moving the virtual camera backward and adjusting the optics, it produces more natural facial proportions. ## Availability in Google Photos - The technology is now integrated into the Auto frame feature. - It applies to eligible photos containing people. - Users receive the re-composed image as a second Auto frame candidate. - The enhancement requires only a single action and automatically changes the viewpoint while retaining the original scene. The feature demonstrates how combining geometric 3D reconstruction with controlled generative inpainting can perform edits that conventional cropping and retouching cannot. For portraits and selfies, Auto frame offers a practical way to improve composition and perspective after the moment has passed.
Read original(opens in new tab)