Google has incorporated the entire Earth into a large model for real-time observation and daily updates.

The ChatGPT moment for Earth has arrived!

Google DeepMind has launched the AlphaEarth Foundations model (hereinafter referred to as AEF), enabling high - precision mapping of the Earth.

Demis Hassabis, a Nobel laureate and the CEO of Google DeepMind, said: It will provide scientists with near - real - time Earth observations.

The model mainly solves two major problems: data overload and information inconsistency. Simply put, there is a large amount of observational data but a small amount of high - quality labeled data, which brings difficulties to global map drawing.

AEF is like a virtual satellite. It integrates petabytes of massive Earth observation data such as optical satellite images and climate simulations. It can generate a general geospatial representation using multi - source and discrete data, allowing scientists to create Earth maps on demand and effortlessly achieve global mapping and monitoring.

The satellite embedding dataset generated by it is now used by more than 50 global organizations, including the United Nations Food and Agriculture Organization and Harvard Forest.

In fact, not only scientists but also ordinary people can easily understand the changes of the Earth through AlphaEarth Foundations.

From the Age of Discovery to the modern era, it took humans 500 years to map the Earth. Now, the emergence of AEF makes it a reality to remap the Earth every day.

This made Bilawal Sidhu, a former Google Maps researcher, exclaim: Google has taken an important step towards building an Earth - version ChatGPT.

Let's see how it does it.

High - precision tracking of the Earth

Multi - source information fusion and continuous time modeling

Now there are countless devices such as satellites, radars, drones, and weather stations, which collect data on the Earth's images, temperature, humidity, terrain, etc. every day.

However, different countries and institutions may have different labeling standards for observational data. Moreover, the Earth's surface is constantly changing (such as deforestation), but on - the - spot labeling is often updated only once every few years, resulting in a disconnection between the labeled data and the actual situation.

Simply put, Earth observation data is like "countless photos of the Earth", but there are few labels that show "what is specifically in the photos", which directly restricts the precision and efficiency of global map drawing.

The value of AEF lies in that it can still make good use of massive observational data and draw more accurate maps even when labels are scarce.

AEF is an embedding field model, and its core is the spatiotemporal precision encoder (STP). STP captures long - distance geographical associations through spatial self - attention, analyzes temporal dynamics through temporal - axis self - attention, and then combines convolution operations to retain local fine features.

Put more simply, it compresses the complex information on the Earth's surface (such as terrain, vegetation, climate, etc.) into a set of "digital codes" (embedding vectors). These codes can accurately reflect the spatiotemporal characteristics of the Earth's surface, allowing for both a global view and detailed analysis.

Another major innovation of AEF is its ability to model time continuously.

It separates the support period of observational data (the time range when the data is actually collected) from the validity period of map generation (the time range to be mapped). Even if there is no direct observational data within the validity period, it can generate continuous results through interpolation or extrapolation.

For example, if there are only satellite images from 2018 and 2020, the model can reliably infer the surface state of a certain area in 2019, which solves the limitation of traditional models that can only process data at fixed time points.

In data processing, AEF achieves deep fusion of multi - source information.

It can receive more than 10 types of inputs, such as optical satellites, radars, lidars, climate data, and even geographical texts, breaking the barriers between data types. It converts these different types and resolutions of data into compatible features through a unified encoding method.

For example, the vegetation color in optical images, the surface structure information from radars, and the temperature data from weather stations will all be integrated into the same embedding vector, which is only 64 bytes.

The training mechanism of the model is also quite distinctive. It uses student - teacher models and contrastive learning strategies. At the same time, it introduces text alignment training to associate geographical text descriptions with the embedding vectors of corresponding regions, further enriching the semantic information of features.

Finally, the embedding field generated by AEF accurately analyzes the Earth's land and coastal waters in the form of 10x10 - meter grids. At the same time, it creates a highly compact summary for each grid. The storage space required is only one - sixteenth of that of other AI systems, reducing the analysis cost.

Moreover, these embedding vectors can be directly used for various mapping tasks without retraining. Whether it is land cover classification, crop identification, or forest change detection, high - precision results can be achieved through simple transfer learning.

Excellent performance

Compared with traditional methods and other AI mapping systems, AEF always maintains the highest accuracy when performing various tasks in different time periods, including identifying land use and estimating surface attributes.

For example, in a land cover classification task, the balanced accuracy of AEF reaches 0.82, while that of the second - best model is only 0.69.

AEF performs particularly well in scenarios with scarce labeled data, with an average error rate 24% lower than that of the tested models, demonstrating excellent learning efficiency.

From the perspective of specific application scenarios, AEF performs comprehensively in thematic mapping (such as land cover classification), biophysical variable estimation, and change detection.

For example, in annual land cover change identification, it can accurately distinguish types such as forests, farmlands, and cities, and the error in capturing subtle changes is much lower than that of traditional methods.

Satellite embedding dataset

In Google Earth Engine, the satellite embedding dataset driven by AEF is one of the largest datasets of its kind, containing more than 14 trillion embedding footprints per year. It has now been released in the Earth Engine data catalog.

This series of annual embedding data has been adopted by multiple global organizations, including the United Nations Food and Agriculture Organization, Harvard Forest, and the Group on Earth Observations. They use this dataset to create powerful custom maps.

For example, the "Global Ecosystem Atlas" project aims to create the first comprehensive resource for mapping and monitoring global ecosystems. This project is using this dataset to help countries classify unmapped ecosystems into categories such as coastal scrub and hyper - arid deserts.

Nick Murray, the global scientific leader of the Global Ecosystem Atlas, said:

The satellite embedding dataset is revolutionizing our work. It helps countries map unknown ecosystems, which is crucial for accurately identifying key areas for conservation efforts.

Animated unsupervised clustering showing the segmentation of the satellite embedding dataset from coarse to fine

The MapBiomas team in Brazil is testing this dataset to gain a deeper understanding of agricultural and environmental changes across the country. It provides an important basis for conservation strategies and sustainable development initiatives in key ecosystems such as the Amazon rainforest.

AEF also helps solve key issues such as food security, deforestation, and water resources.

In response, some netizens said: AI models are becoming public infrastructure.

After all, only by understanding the Earth can we better protect it.

Paper address:

https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphaearth-foundations-helps-map-our-planet-in-unprecedented-detail/alphaearth-foundations.pdf

Reference links

[1]https://deepmind.google/discover/blog/alphaearth-foundations-helps-map-our-planet-in-unprecedented-detail/

[2]https://x.com/GoogleDeepMind/status/1950563700286398965

[3]https://x.com/demishassabis/status/1950667643771326784

[4]https://x.com/bilawalsidhu/status/1950580970907648234

This article is from the WeChat public account "QbitAI". Author: Wen Le. Republished by 36Kr with permission.