alterlab-geomaster

GeoMaster

全面的地理空间科学技能，涵盖 GIS、遥感、空间分析和地球观测机器学习，涉及 70 多个主题，包含 8 种编程语言的 500 多个代码示例。

何时使用此技能

当用户希望执行以下操作时，使用此技能：

• 处理卫星或航空影像（Sentinel、Landsat、MODIS、SAR、高光谱），并计算光谱指数
• 处理栅格和 DEM——地形指标、镶嵌、重投影、分区统计
• 搜索并加载云原生数据（STAC、COG、Planetary Computer、Earth Engine、Copernicus Data Space）
• 对地球观测数据训练或应用机器学习，或运行水文、海洋、大气空间工作流

不适用场景

场景	改用
纯矢量处理——sjoin、buffer、overlay、dissolve、分级统计图——不涉及栅格或地球观测	alterlab-geopandas
天球坐标、FITS 图像、天空 WCS	alterlab-astropy
模拟流体动力学本身（Navier-Stokes、浅水方程求解器）	alterlab-fluidsim
为人文研究绘制历史资料地图，或对档案地图进行地理配准	alterlab-digital-humanities

安装

为 GDAL 支持的技术栈选择一种包管理器。现代 pip wheel 包含 rasterio/fiona/pyproj/shapely 所需的 GDAL/GEOS/PROJ，因此纯 pip（uv）环境无需系统级 GDAL 即可覆盖核心库。请勿在同一环境中混用 conda-GDAL 和 pip rasterio/fiona——两者附带的 GDAL 二进制文件不同，ABI 不匹配会导致段错误。独立的 gdal Python 绑定不包含二进制文件（需要匹配的系统级或 conda libgdal）；PDAL 和 rsgislib 同样没有可靠的 pip wheel——请通过 conda 安装（选项 B）。

# Option A — uv/pip env (recommended here): wheels bundle GDAL/GEOS/PROJ.
# rasterio/fiona cover most GDAL needs; standalone `gdal` -> use Option B.
uv pip install rasterio fiona shapely pyproj geopandas
uv pip install torchgeo earthengine-api
uv pip install scikit-learn xgboost torch-geometric
uv pip install osmnx networkx folium keplergl
uv pip install cartopy contextily mapclassify
uv pip install xarray rioxarray dask-geopandas
uv pip install pystac-client planetary-computer odc-stac rio-cogeo
uv pip install "laspy[lazrs]" open3d      # PDAL: use conda (no pip wheel)

# Option B — conda env (best for PDAL / rsgislib / system GDAL tooling)
conda install -c conda-forge gdal rasterio fiona shapely pyproj geopandas \
    rsgislib pdal python-pdal postgis libspatialite

快速入门

从 Sentinel-2 计算 NDVI

import rasterio
import numpy as np

with rasterio.open('sentinel2.tif') as src:
    red = src.read(4).astype(float)   # B04
    nir = src.read(8).astype(float)   # B08
    ndvi = (nir - red) / (nir + red + 1e-8)
    ndvi = np.nan_to_num(ndvi, nan=0)

    profile = src.profile
    profile.update(count=1, dtype=rasterio.float32)

    with rasterio.open('ndvi.tif', 'w', **profile) as dst:
        dst.write(ndvi.astype(rasterio.float32), 1)

使用 GeoPandas 进行空间分析

import geopandas as gpd

# Load and ensure same CRS
zones = gpd.read_file('zones.geojson')
points = gpd.read_file('points.geojson')

if zones.crs != points.crs:
    points = points.to_crs(zones.crs)

# Spatial join and statistics
joined = gpd.sjoin(points, zones, how='inner', predicate='within')
stats = joined.groupby('zone_id').agg({
    'value': ['count', 'mean', 'std', 'min', 'max']
}).round(2)

Google Earth Engine 时间序列

import ee
import pandas as pd

ee.Initialize(project='your-project')
roi = ee.Geometry.Point([-122.4, 37.7]).buffer(10000)

s2 = (ee.ImageCollection('COPERNICUS/S2_SR_HARMONIZED')
      .filterBounds(roi)
      .filterDate('2020-01-01', '2023-12-31')
      .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 20)))

def add_ndvi(img):
    return img.addBands(img.normalizedDifference(['B8', 'B4']).rename('NDVI'))

s2_ndvi = s2.map(add_ndvi)

def extract_series(image):
    stats = image.reduceRegion(ee.Reducer.mean(), roi.centroid(), scale=10, maxPixels=1e9)
    return ee.Feature(None, {'date': image.date().format('YYYY-MM-dd'), 'ndvi': stats.get('NDVI')})

series = s2_ndvi.map(extract_series).getInfo()
df = pd.DataFrame([f['properties'] for f in series['features']])
df['date'] = pd.to_datetime(df['date'])

核心概念

数据类型

类型	示例	库
矢量	Shapefile、GeoJSON、GeoPackage	GeoPandas、Fiona、GDAL
栅格	GeoTIFF、NetCDF、COG	Rasterio、Xarray、GDAL
点云	LAS、LAZ	Laspy、PDAL、Open3D

坐标系

• EPSG:4326（WGS 84）- 地理坐标，经纬度，用于存储
• EPSG:3857（Web Mercator）- 仅用于网络地图（不要用于面积或距离计算！）
• EPSG:326xx/327xx（UTM）- 用于度量计算，每个分区的失真小于 1%
• 使用 gdf.estimate_utm_crs() 自动检测 UTM

# Always check CRS before operations
assert gdf1.crs == gdf2.crs, "CRS mismatch!"

# For area/distance calculations, use projected CRS
gdf_metric = gdf.to_crs(gdf.estimate_utm_crs())
area_sqm = gdf_metric.geometry.area

OGC 标准

• WMS：网络地图服务——栅格地图
• WFS：网络要素服务——矢量数据
• WCS：网络覆盖服务——栅格覆盖数据
• STAC：时空资产目录——现代元数据

常见操作

光谱指数

def calculate_indices(image_path, bands=(2, 3, 4, 8, 11)):
    """NDVI, EVI, SAVI, NDWI from Sentinel-2.

    `bands` maps (B02, B03, B04, B08, B11) to the 1-based band indices in
    your file. EVI's constants (6, 7.5, +1) assume surface reflectance in
    [0, 1]; raw L2A DN are scaled by 10000, so divide first or EVI is wrong.
    """
    with rasterio.open(image_path) as src:
        B02, B03, B04, B08, B11 = (src.read(b).astype(float) for b in bands)

    ndvi = (B08 - B04) / (B08 + B04 + 1e-8)
    evi = 2.5 * (B08 - B04) / (B08 + 6*B04 - 7.5*B02 + 1 + 1e-8)
    savi = ((B08 - B04) / (B08 + B04 + 0.5)) * 1.5
    ndwi = (B03 - B08) / (B03 + B08 + 1e-8)

    return {'NDVI': ndvi, 'EVI': evi, 'SAVI': savi, 'NDWI': ndwi}

矢量操作

# Buffer (use projected CRS! 1000 = metres in UTM).
# Keep the result in the projected frame; don't bolt projected geometry
# back onto a geographic gdf or you silently mix CRS in one column.
gdf_proj = gdf.to_crs(gdf.estimate_utm_crs())
gdf_proj['buffer_1km'] = gdf_proj.geometry.buffer(1000)

# Spatial relationships
intersects = gdf[gdf.geometry.intersects(other_geometry)]
contains = gdf[gdf.geometry.contains(point_geometry)]

# Geometric operations
gdf['centroid'] = gdf.geometry.centroid
gdf['simplified'] = gdf.geometry.simplify(tolerance=0.001)

# Overlay operations
intersection = gpd.overlay(gdf1, gdf2, how='intersection')
union = gpd.overlay(gdf1, gdf2, how='union')

地形分析

def terrain_metrics(dem_path, azimuth=315.0, altitude=45.0):
    """Slope (deg), aspect (deg clockwise from north, downslope), hillshade [0, 1].

    Assumes a projected, north-up DEM whose horizontal units match its elevation
    units (reproject geographic DEMs first, or the gradients are meaningless).
    """
    with rasterio.open(dem_path) as src:
        dem = src.read(1).astype(float)
        xres, yres = src.res                      # pixel size — gradients need it

    dz_drow, dz_dx = np.gradient(dem, yres, xres)  # rows run north -> south
    dz_dy = -dz_drow                               # northward gradient
    slope = np.degrees(np.arctan(np.hypot(dz_dx, dz_dy)))
    aspect = np.degrees(np.arctan2(-dz_dx, -dz_dy)) % 360

    # Hillshade: cos(zenith)cos(slope) + sin(zenith)sin(slope)cos(azimuth - aspect)
    zenith, az = np.radians(90.0 - altitude), np.radians(azimuth)
    s, a = np.radians(slope), np.radians(aspect)
    hillshade = np.clip(np.cos(zenith) * np.cos(s) +
                        np.sin(zenith) * np.sin(s) * np.cos(az - a), 0, 1)

    return slope, aspect, hillshade

网络分析

import osmnx as ox
import networkx as nx

# Download and analyze street network
G = ox.graph_from_place('San Francisco, CA', network_type='drive')
G = ox.add_edge_travel_times(ox.add_edge_speeds(G))  # both return the graph

# Shortest path
orig = ox.distance.nearest_nodes(G, -122.4, 37.7)
dest = ox.distance.nearest_nodes(G, -122.3, 37.8)
route = nx.shortest_path(G, orig, dest, weight='travel_time')

影像分类

from sklearn.ensemble import RandomForestClassifier
import rasterio
from rasterio.features import rasterize

def classify_imagery(raster_path, training_gdf, output_path):
    """Train RF and classify imagery."""
    with rasterio.open(raster_path) as src:
        image = src.read()
        profile = src.profile
        transform = src.transform

    # Extract training data
    X_train, y_train = [], []
    for _, row in training_gdf.iterrows():
        mask = rasterize([(row.geometry, 1)],
                        out_shape=(profile['height'], profile['width']),
                        transform=transform, fill=0, dtype=np.uint8)
        pixels = image[:, mask > 0].T
        X_train.extend(pixels)
        y_train.extend([row['class_id']] * len(pixels))

    # Train and predict
    rf = RandomForestClassifier(n_estimators=100, max_depth=20, n_jobs=-1)
    rf.fit(X_train, y_train)

    prediction = rf.predict(image.reshape(image.shape[0], -1).T)
    prediction = prediction.reshape(profile['height'], profile['width'])

    profile.update(dtype=rasterio.uint8, count=1)
    with rasterio.open(output_path, 'w', **profile) as dst:
        dst.write(prediction.astype(rasterio.uint8), 1)

    return rf

现代云原生工作流

STAC + Planetary Computer

import pystac_client
import planetary_computer
import odc.stac

# Search Sentinel-2 via STAC
catalog = pystac_client.Client.open(
    "https://planetarycomputer.microsoft.com/api/stac/v1",
    modifier=planetary_computer.sign_inplace,
)

search = catalog.search(
    collections=["sentinel-2-l2a"],
    bbox=[-122.5, 37.7, -122.3, 37.9],
    datetime="2023-01-01/2023-12-31",
    query={"eo:cloud_cover": {"lt": 20}},
)

# Load as xarray (cloud-native!)
# pystac-client >= 0.7: use search.items() (get_items() is deprecated).
items = list(search.items())[:5]
data = odc.stac.load(
    items,
    bands=["B02", "B03", "B04", "B08"],
    crs="EPSG:32610",
    resolution=10,
)

# Calculate NDVI on xarray
ndvi = (data.B08 - data.B04) / (data.B08 + data.B04)

云优化 GeoTIFF（COG）

import rasterio
from rasterio.session import AWSSession
from rasterio.windows import Window

# Read a window of a cloud-hosted COG (HTTP range requests fetch only that window).
# Credentials go on a rasterio.Env, not on rasterio.open; for public buckets use
# AWSSession(aws_unsigned=True).
with rasterio.Env(AWSSession(aws_access_key_id=..., aws_secret_access_key=...)):
    with rasterio.open('s3://bucket/path.tif') as src:
        subset = src.read(1, window=Window(col_off=1000, row_off=1000,
                                           width=1000, height=1000))

# Write a valid COG (tiled + internal overviews + COG layout) with rio-cogeo;
# a plain tiled GeoTIFF without overviews is not a COG
from rio_cogeo.cogeo import cog_translate, cog_validate
from rio_cogeo.profiles import cog_profiles

cog_translate('input.tif', 'output_cog.tif', cog_profiles.get('deflate'))
is_valid, errors, warnings = cog_validate('output_cog.tif')

性能提示

# 1. Spatial indexing (10-100x faster queries)
gdf.sindex  # Auto-created by GeoPandas

# 2. Chunk large rasters
with rasterio.open('large.tif') as src:
    for i, window in src.block_windows(1):
        block = src.read(1, window=window)

# 3. Dask for big data (lazy, chunked, dask-backed DataArray)
import rioxarray
da_raster = rioxarray.open_rasterio('large.tif', chunks=(1, 1024, 1024))
# .data is the underlying dask.array if you need the raw chunked array:
# dask_array = da_raster.data

# 4. Use Arrow for I/O
gdf.to_file('output.gpkg', use_arrow=True)

# 5. GDAL caching
from osgeo import gdal
gdal.SetCacheMax(2**30)  # 1GB cache

# 6. Parallel processing
rf = RandomForestClassifier(n_jobs=-1)  # All cores

最佳实践

1. 空间操作前务必检查 CRS
2. 面积或距离计算使用投影 CRS
3. 验证几何图形：gdf = gdf[gdf.is_valid]（使用 gdf.geometry.make_valid() 修复）
4. 删除缺失几何图形：gdf = gdf[gdf.geometry.notna() & ~gdf.geometry.is_empty]（fillna(None) 不适用于几何列）
5. 使用高效格式：GeoPackage 优于 Shapefile；大型数据使用 Parquet
6. 对光学影像应用云掩膜
7. 保留数据沿袭信息，以支持可复现研究
8. 根据分析尺度选择合适的分辨率

详细文档

• 坐标系 - CRS 基础知识、UTM、坐标转换
• 核心库 - GDAL、Rasterio、GeoPandas、Shapely
• 遥感 - 卫星任务、光谱指数、SAR
• 机器学习 - 用于 RS 的深度学习、CNN、GNN
• GIS 软件 - QGIS、ArcGIS、GRASS 集成
• 科学领域 - 海洋、水文、农业、林业
• 高级 GIS - 3D GIS、时空分析、拓扑
• 大数据 - 分布式处理、GPU 加速
• 行业应用 - 城市规划、灾害管理
• 编程语言 - Python、R、Julia、JS、C++、Java、Go、Rust
• 数据源 - 卫星目录、API
• 故障排查 - 常见问题、调试、错误参考
• 代码示例 - 500+ 个示例

---

GeoMaster 涵盖从基础 GIS 操作到高级遥感和机器学习的方方面面。

属于 AlterLab Academic Skills 套件。