Lab 4¶

Training a Custom Model for Wildfire Burn Scar Detection

⏱️ Estimated Duration: 60-90 minutes
📊 Difficulty Level: Intermediate

📥 Getting the Lab Materials

Getting the Lab Materials: Clone the repository:

git clone https://github.com/terrastackai/geospatial-studio.git
cd geospatial-studio/workshop/docs/notebooks
jupyter notebook lab4-burnscars-workflow.ipynb

⚠️ Note: This lab requires multiple JSON configuration files. Cloning the repository ensures you have all necessary files.

---

Overview¶

In this lab, you'll complete an end-to-end machine learning workflow for detecting wildfire burn scars from satellite imagery. This represents a real-world disaster response scenario where you need to:

1. Onboard training data - Upload and configure a labeled dataset
2. Register a foundation model - Set up the base model for fine-tuning
3. Submit a fine-tuning job - Train a custom model on your dataset
4. Monitor training progress - Track the training process
5. Run inference - Test your trained model on new data
6. Visualize results - View predictions in the UI

What You'll Learn¶

• How to onboard a custom training dataset
• How to register foundation models as base models
• How to configure and submit fine-tuning jobs
• How to monitor training progress
• How to run inference with your trained model
• Best practices for GPU resource management

Prerequisites¶

Required Resources¶

• GPU Access: This lab requires GPU resources for model training
• Training Data: Download from here
• Completed Labs: Labs 1-3 (for SDK familiarity)

About the Dataset¶

We'll use the HLS Burn Scars dataset from Hugging Face:

• 804 scenes of 512x512 pixels
• 6 spectral bands from Harmonized Landsat Sentinel-2 (HLS)
• Years: 2018-2021
• Location: Contiguous United States
• Purpose: Segmentation (pixel-level classification)

⚠️ Important: GPU Requirements¶

Training requires GPU resources:

• Fine-tuning requires NVIDIA GPUs for optimal performance
• Training on CPU is not recommended and has not been tested
• If you have a single GPU in your cluster, you may need to scale down the terratorch-inference deployment to release the GPU for training
• Training time varies based on GPU type (typically 30-60 minutes on modern GPUs)

For Mac users with Apple Silicon:

• Mac GPU support is currently in development
• You can train externally and upload the resulting checkpoint (see section 1.5.1 in the getting-started notebook)

---

1. Setup and Imports¶

First, let's import the required libraries and set up our connection to Geospatial Studio.

In [ ]:

# Import required packages
import json
import time
from IPython.display import display, Markdown

import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

from geostudio import Client

# Import required packages import json import time from IPython.display import display, Markdown import urllib3 urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) from geostudio import Client

2. Connect to Geospatial Studio¶

Use the same configuration file from Lab 1:

In [ ]:

# Initialize the client
client = Client(geostudio_config_file=".geostudio_config_file")

print("✅ Connected to Geospatial Studio!")

# Initialize the client client = Client(geostudio_config_file=".geostudio_config_file") print("✅ Connected to Geospatial Studio!")

3. Understanding the Workflow¶

Before we begin, let's understand the complete workflow:

graph LR
    A[Foundation Model] --> D[Fine-tuning Job]
    B[Training Dataset] --> D
    C[Task Template] --> D
    D --> E[Trained Model]
    E --> F[Inference]
    F --> G[Results]
    
    style A fill:#1f77b4,stroke:#333,stroke-width:2px,color:#fff
    style B fill:#1f77b4,stroke:#333,stroke-width:2px,color:#fff
    style C fill:#1f77b4,stroke:#333,stroke-width:2px,color:#fff
    style D fill:#ff7f0e,stroke:#333,stroke-width:2px,color:#fff
    style E fill:#2ca02c,stroke:#333,stroke-width:2px,color:#fff
    style F fill:#d62728,stroke:#333,stroke-width:2px,color:#fff
    style G fill:#9467bd,stroke:#333,stroke-width:2px,color:#fff

Key Components:

1. Foundation Model (Backbone): Pre-trained model with general geospatial understanding
2. Training Dataset: Labeled data for your specific task
3. Task Template: Configuration defining the learning task (from Lab 3)
4. Fine-tuning Job: Training process that customizes the model
5. Trained Model (Tune): Your custom model checkpoint
6. Inference: Running predictions on new data

---

4. Step 1: Register the Foundation Model¶

First, we need to register the Prithvi-EO-V2-300M foundation model as our base model. This is a 300-million parameter geospatial foundation model pre-trained on satellite imagery.

About Prithvi-EO-V2:

• Pre-trained on multi-spectral satellite data
• Understands spatial and temporal patterns in Earth observation data
• Serves as the starting point for fine-tuning

The backbone configuration includes:

• Model architecture parameters (embed_dim, num_heads, etc.)
• Checkpoint file location
• Model category and type

In [ ]:

# Load the backbone configuration
with open("../../../populate-studio/payloads/backbones/backbone-Prithvi_EO_V2_300M.json", "r") as f:
    backbone = json.load(f)

print("📋 Backbone Configuration:")
print(json.dumps(backbone, indent=2))

# Load the backbone configuration with open("../../../populate-studio/payloads/backbones/backbone-Prithvi_EO_V2_300M.json", "r") as f: backbone = json.load(f) print("📋 Backbone Configuration:") print(json.dumps(backbone, indent=2))

In [ ]:

# Register the backbone model
onboard_backbone_response = client.create_base_model(backbone)

print("\n✅ Foundation model registered successfully!")
print(json.dumps(onboard_backbone_response, indent=2))

# Save the base model ID for later use
base_model_id = onboard_backbone_response['id']
print(f"\n📌 Base Model ID: {base_model_id}")

# Register the backbone model onboard_backbone_response = client.create_base_model(backbone) print("\n✅ Foundation model registered successfully!") print(json.dumps(onboard_backbone_response, indent=2)) # Save the base model ID for later use base_model_id = onboard_backbone_response['id'] print(f"\n📌 Base Model ID: {base_model_id}")

5. Step 2: Onboard the Training Dataset¶

Now we'll onboard the burn scars training dataset. This process:

1. Downloads the dataset from a remote URL
2. Validates the data structure and labels
3. Indexes the dataset for training
4. Stores metadata in the Studio catalog

Dataset Configuration:

• 6 spectral bands: Blue, Green, Red, NIR, SWIR1, SWIR2
• 3 label categories: Fire Scar (1), No Data (0), Ignore (-1)
• File format: GeoTIFF with .mask.tif suffix for labels
• Purpose: Segmentation (pixel-level classification)

⚠️ Note: This process may take several minutes depending on network speed and cluster resources.

In [ ]:

# Load the dataset configuration
with open("../../../populate-studio/payloads/datasets/dataset-burn_scars.json", "r") as f:
    wild_fire_dataset = json.load(f)

print("📋 Dataset Configuration:")
print(json.dumps(wild_fire_dataset, indent=2))

# Load the dataset configuration with open("../../../populate-studio/payloads/datasets/dataset-burn_scars.json", "r") as f: wild_fire_dataset = json.load(f) print("📋 Dataset Configuration:") print(json.dumps(wild_fire_dataset, indent=2))

In [ ]:

# Onboard the dataset
print("🚀 Starting dataset onboarding...")
print("This may take several minutes to download and process the data.\n")

onboard_dataset_response = client.onboard_dataset(data=wild_fire_dataset)

print("✅ Dataset onboarding initiated!")
print(json.dumps(onboard_dataset_response, indent=2))

# Save the dataset ID for later use
dataset_id = onboard_dataset_response['dataset_id']
print(f"\n📌 Dataset ID: {dataset_id}")

# Onboard the dataset print("🚀 Starting dataset onboarding...") print("This may take several minutes to download and process the data.\n") onboard_dataset_response = client.onboard_dataset(data=wild_fire_dataset) print("✅ Dataset onboarding initiated!") print(json.dumps(onboard_dataset_response, indent=2)) # Save the dataset ID for later use dataset_id = onboard_dataset_response['dataset_id'] print(f"\n📌 Dataset ID: {dataset_id}")

Monitor Dataset Onboarding¶

The onboarding process runs as a background job. We'll poll the status until completion.

What's happening:

• Downloading dataset from remote storage
• Extracting and validating files
• Creating dataset index
• Generating metadata

Troubleshooting:

• If polling fails due to port-forwarding issues, wait a moment and re-establish port forwarding
• Monitor the onboarding job pod logs in your cluster: kubectl logs -f <onboarding-job-pod>
• On resource-constrained systems, other services may be temporarily affected

In [ ]:

# Poll until dataset onboarding is complete
print("⏳ Monitoring dataset onboarding progress...\n")

client.poll_onboard_dataset_until_finished(dataset_id)

print("\n✅ Dataset onboarding completed successfully!")
print("You can now view the dataset in the Studio UI under 'Data Catalog'.")

# Poll until dataset onboarding is complete print("⏳ Monitoring dataset onboarding progress...\n") client.poll_onboard_dataset_until_finished(dataset_id) print("\n✅ Dataset onboarding completed successfully!") print("You can now view the dataset in the Studio UI under 'Data Catalog'.")

6. Step 3: Submit Fine-Tuning Job¶

Now that we have:

• ✅ Foundation model registered
• ✅ Training dataset onboarded
• ✅ Task template created (from Lab 3)

We can submit a fine-tuning job to train our custom burn scar detection model.

Fine-Tuning Configuration:

• Base Model: Prithvi-EO-V2-300M (registered above)
• Dataset: Burn scars training data (onboarded above)
• Task Template: Segmentation template (from Lab 3)
• Name: burn-scars-demo

⚠️ GPU Resource Management¶

Before submitting the training job:

If you have a single GPU in your cluster:

# Scale down inference deployment to release GPU
kubectl scale deployment terratorch-inference --replicas=0

After training completes:

# Scale inference back up
kubectl scale deployment terratorch-inference --replicas=1

Why? The training job (geotune-xxx pod) needs GPU access. If the inference deployment is using the GPU, the training pod will remain in "Pending" state.

In [ ]:

# Get the task template ID from Lab 3
# If you haven't run Lab 3, you'll need to create the segmentation template first

# Load the template configuration
try:
    with open('../../../populate-studio/payloads/templates/template-seg.json', 'r') as f:
        segmentation_template = json.load(f)
    
    print("📋 Segmentation Task Template:")
    print(json.dumps(segmentation_template, indent=2))
except FileNotFoundError:
    print("⚠️ Template file not found. Please check populate-studio/payloads/templates/")
    segmentation_template = None

# Get the task template ID from Lab 3 # If you haven't run Lab 3, you'll need to create the segmentation template first # Load the template configuration try: with open('../../../populate-studio/payloads/templates/template-seg.json', 'r') as f: segmentation_template = json.load(f) print("📋 Segmentation Task Template:") print(json.dumps(segmentation_template, indent=2)) except FileNotFoundError: print("⚠️ Template file not found. Please check populate-studio/payloads/templates/") segmentation_template = None

In [ ]:

# Create the task template
if segmentation_template:
    template_response = client.create_task(segmentation_template)
    tune_template_id = template_response['id']
    print(f"✅ Task template created: {tune_template_id}")
else:
    tune_template_id = None
    print("⚠️ Cannot create template - template file not loaded")

# Create the task template if segmentation_template: template_response = client.create_task(segmentation_template) tune_template_id = template_response['id'] print(f"✅ Task template created: {tune_template_id}") else: tune_template_id = None print("⚠️ Cannot create template - template file not loaded")

In [ ]:

# Create the fine-tuning payload
payload = {
    "name": "burn-scars-demo",
    "description": "Segmentation model for wildfire burn scar detection",
    "dataset_id": dataset_id,
    "base_model_id": base_model_id,
    "tune_template_id": tune_template_id,
}

print("📋 Fine-Tuning Configuration:")
print(json.dumps(payload, indent=2))

# Create the fine-tuning payload payload = { "name": "burn-scars-demo", "description": "Segmentation model for wildfire burn scar detection", "dataset_id": dataset_id, "base_model_id": base_model_id, "tune_template_id": tune_template_id, } print("📋 Fine-Tuning Configuration:") print(json.dumps(payload, indent=2))

In [ ]:

# Submit the fine-tuning job
print("\n🚀 Submitting fine-tuning job...\n")

tune_submitted = client.submit_tune(payload, output='json')

print("✅ Fine-tuning job submitted successfully!")
print(json.dumps(tune_submitted, indent=2))

# Save the tune ID for later use
tune_id = tune_submitted['tune_id']
print(f"\n📌 Tune ID: {tune_id}")
print("\nYou can monitor progress in the Studio UI under 'Start fine-tuning' > 'Model & Tunes'")

# Submit the fine-tuning job print("\n🚀 Submitting fine-tuning job...\n") tune_submitted = client.submit_tune(payload, output='json') print("✅ Fine-tuning job submitted successfully!") print(json.dumps(tune_submitted, indent=2)) # Save the tune ID for later use tune_id = tune_submitted['tune_id'] print(f"\n📌 Tune ID: {tune_id}") print("\nYou can monitor progress in the Studio UI under 'Start fine-tuning' > 'Model & Tunes'")

7. Step 4: Monitor Training Progress¶

Training a geospatial foundation model takes time. We'll poll the status until completion.

What's happening during training:

1. Initialization: Loading base model and dataset
2. Training: Iterating through epochs, updating model weights
3. Validation: Evaluating performance on validation set
4. Checkpointing: Saving model state periodically
5. Completion: Finalizing and storing the trained model

Expected Duration:

• Modern GPU (V100/A100): 30-45 minutes
• Older GPU: 45-90 minutes
• CPU: Not recommended (untested)

Monitoring Options:

1. SDK Polling (below): Automated status checks
2. Studio UI: Visual progress tracking with metrics
3. Kubernetes Logs: kubectl logs -f <geotune-xxx-pod>

Troubleshooting:

• If polling fails, check port-forwarding and retry
• If pod is "Pending", check GPU availability
• Monitor cluster resources: kubectl top nodes

In [ ]:

# Poll until fine-tuning is complete
print("⏳ Monitoring fine-tuning progress...")
print("This will take 30-90 minutes depending on your GPU.\n")
print("💡 Tip: You can also monitor progress in the Studio UI\n")

client.poll_finetuning_until_finished(tune_id=tune_id)

print("\n✅ Fine-tuning completed successfully!")
print(f"Your trained model '{payload['name']}' is now available in the Model Catalog.")

# Poll until fine-tuning is complete print("⏳ Monitoring fine-tuning progress...") print("This will take 30-90 minutes depending on your GPU.\n") print("💡 Tip: You can also monitor progress in the Studio UI\n") client.poll_finetuning_until_finished(tune_id=tune_id) print("\n✅ Fine-tuning completed successfully!") print(f"Your trained model '{payload['name']}' is now available in the Model Catalog.")

8. Step 5: Run Inference with Trained Model¶

Now that training is complete, let's test our model on new data! We'll run inference on the Park Fire from August 2024 in California.

Inference Configuration:

1. Spatial Domain: Location and extent

   • URL to input imagery
   • Bounding box (optional)
   • Tiles or polygons (optional)

2. Temporal Domain: Date(s) of imagery

3. Model Settings: Band configuration and scaling

4. Post-Processing: Masking options

   • Cloud masking
   • Ocean masking
   • Snow/ice masking

5. Visualization: GeoServer layer configuration

   • Input RGB layer
   • Prediction layer with color mapping

The inference pipeline consists of:

1. URL Connector: Fetch input data
2. TerraTorch Inference: Run model prediction
3. Post-Processing: Apply masks and filters
4. GeoServer Push: Publish results for visualization

In [ ]:

# Define the inference payload
inference_payload = {
    "model_display_name": "burn-scars-demo",
    "location": "Red Bluff, California, United States",
    "description": "Park Fire Aug 2024",
    "spatial_domain": {
        "bbox": [],
        "urls": [
            "https://geospatial-studio-example-data.s3.us-east.cloud-object-storage.appdomain.cloud/examples-for-inference/park_fire_scaled.tif"
        ],
        "tiles": [],
        "polygons": []
    },
    "temporal_domain": [
        "2024-08-12"
    ],
    "pipeline_steps": [
        {
            "status": "READY",
            "process_id": "url-connector",
            "step_number": 0
        },
        {
            "status": "WAITING",
            "process_id": "terratorch-inference",
            "step_number": 1
        },
        {
            "status": "WAITING",
            "process_id": "postprocess-generic",
            "step_number": 2
        },
        {
            "status": "WAITING",
            "process_id": "push-to-geoserver",
            "step_number": 3
        }
    ],
    "post_processing": {
        "cloud_masking": "False",
        "ocean_masking": "False",
        "snow_ice_masking": None,
        "permanent_water_masking": "False"
    },
    "model_input_data_spec": [
        {
            "bands": [
                {
                    "index": "0",
                    "RGB_band": "B",
                    "band_name": "Blue",
                    "scaling_factor": "0.0001"
                },
                {
                    "index": "1",
                    "RGB_band": "G",
                    "band_name": "Green",
                    "scaling_factor": "0.0001"
                },
                {
                    "index": "2",
                    "RGB_band": "R",
                    "band_name": "Red",
                    "scaling_factor": "0.0001"
                },
                {
                    "index": "3",
                    "band_name": "NIR_Narrow",
                    "scaling_factor": "0.0001"
                },
                {
                    "index": "4",
                    "band_name": "SWIR1",
                    "scaling_factor": "0.0001"
                },
                {
                    "index": "5",
                    "band_name": "SWIR2",
                    "scaling_factor": "0.0001"
                }
            ],
            "connector": "sentinelhub",
            "collection": "hls_l30",
            "file_suffix": "_merged.tif",
            "modality_tag": "HLS_L30"
        }
    ],
    "geoserver_push": [
        {
            "z_index": 0,
            "workspace": "geofm",
            "layer_name": "input_rgb",
            "file_suffix": "",
            "display_name": "Input image (RGB)",
            "filepath_key": "model_input_original_image_rgb",
            "geoserver_style": {
                "rgb": [
                    {
                        "label": "RedChannel",
                        "channel": 1,
                        "maxValue": 255,
                        "minValue": 0
                    },
                    {
                        "label": "GreenChannel",
                        "channel": 2,
                        "maxValue": 255,
                        "minValue": 0
                    },
                    {
                        "label": "BlueChannel",
                        "channel": 3,
                        "maxValue": 255,
                        "minValue": 0
                    }
                ]
            },
            "visible_by_default": "True"
        },
        {
            "z_index": 1,
            "workspace": "geofm",
            "layer_name": "pred",
            "file_suffix": "",
            "display_name": "Model prediction",
            "filepath_key": "model_output_image",
            "geoserver_style": {
                "segmentation": [
                    {
                        "color": "#000000",
                        "label": "ignore",
                        "opacity": 0,
                        "quantity": "-1"
                    },
                    {
                        "color": "#000000",
                        "label": "no-data",
                        "opacity": 0,
                        "quantity": "0"
                    },
                    {
                        "color": "#ab4f4f",
                        "label": "fire-scar",
                        "opacity": 1,
                        "quantity": "1"
                    }
                ]
            },
            "visible_by_default": "True"
        }
    ]
}

print("📋 Inference Configuration:")
print(f"Location: {inference_payload['location']}")
print(f"Date: {inference_payload['temporal_domain'][0]}")
print(f"Model: {inference_payload['model_display_name']}")

# Define the inference payload inference_payload = { "model_display_name": "burn-scars-demo", "location": "Red Bluff, California, United States", "description": "Park Fire Aug 2024", "spatial_domain": { "bbox": [], "urls": [ "https://geospatial-studio-example-data.s3.us-east.cloud-object-storage.appdomain.cloud/examples-for-inference/park_fire_scaled.tif" ], "tiles": [], "polygons": [] }, "temporal_domain": [ "2024-08-12" ], "pipeline_steps": [ { "status": "READY", "process_id": "url-connector", "step_number": 0 }, { "status": "WAITING", "process_id": "terratorch-inference", "step_number": 1 }, { "status": "WAITING", "process_id": "postprocess-generic", "step_number": 2 }, { "status": "WAITING", "process_id": "push-to-geoserver", "step_number": 3 } ], "post_processing": { "cloud_masking": "False", "ocean_masking": "False", "snow_ice_masking": None, "permanent_water_masking": "False" }, "model_input_data_spec": [ { "bands": [ { "index": "0", "RGB_band": "B", "band_name": "Blue", "scaling_factor": "0.0001" }, { "index": "1", "RGB_band": "G", "band_name": "Green", "scaling_factor": "0.0001" }, { "index": "2", "RGB_band": "R", "band_name": "Red", "scaling_factor": "0.0001" }, { "index": "3", "band_name": "NIR_Narrow", "scaling_factor": "0.0001" }, { "index": "4", "band_name": "SWIR1", "scaling_factor": "0.0001" }, { "index": "5", "band_name": "SWIR2", "scaling_factor": "0.0001" } ], "connector": "sentinelhub", "collection": "hls_l30", "file_suffix": "_merged.tif", "modality_tag": "HLS_L30" } ], "geoserver_push": [ { "z_index": 0, "workspace": "geofm", "layer_name": "input_rgb", "file_suffix": "", "display_name": "Input image (RGB)", "filepath_key": "model_input_original_image_rgb", "geoserver_style": { "rgb": [ { "label": "RedChannel", "channel": 1, "maxValue": 255, "minValue": 0 }, { "label": "GreenChannel", "channel": 2, "maxValue": 255, "minValue": 0 }, { "label": "BlueChannel", "channel": 3, "maxValue": 255, "minValue": 0 } ] }, "visible_by_default": "True" }, { "z_index": 1, "workspace": "geofm", "layer_name": "pred", "file_suffix": "", "display_name": "Model prediction", "filepath_key": "model_output_image", "geoserver_style": { "segmentation": [ { "color": "#000000", "label": "ignore", "opacity": 0, "quantity": "-1" }, { "color": "#000000", "label": "no-data", "opacity": 0, "quantity": "0" }, { "color": "#ab4f4f", "label": "fire-scar", "opacity": 1, "quantity": "1" } ] }, "visible_by_default": "True" } ] } print("📋 Inference Configuration:") print(f"Location: {inference_payload['location']}") print(f"Date: {inference_payload['temporal_domain'][0]}") print(f"Model: {inference_payload['model_display_name']}")

In [ ]:

# Submit the inference request
print("\n🚀 Submitting inference request...\n")

inference_response = client.try_out_tune(tune_id=tune_id, data=inference_payload)

print("✅ Inference submitted successfully!")
print(json.dumps(inference_response, indent=2))

print("\n💡 Next Steps:")
print("1. Navigate to the Geospatial Studio UI")
print("2. Click on 'Start fine-tuning' card")
print("3. Under 'Model & Tunes', find 'burn-scars-demo'")
print("4. Under 'History', click on 'Park Fire Aug 2024'")
print("5. View the inference results on the map!")

# Submit the inference request print("\n🚀 Submitting inference request...\n") inference_response = client.try_out_tune(tune_id=tune_id, data=inference_payload) print("✅ Inference submitted successfully!") print(json.dumps(inference_response, indent=2)) print("\n💡 Next Steps:") print("1. Navigate to the Geospatial Studio UI") print("2. Click on 'Start fine-tuning' card") print("3. Under 'Model & Tunes', find 'burn-scars-demo'") print("4. Under 'History', click on 'Park Fire Aug 2024'") print("5. View the inference results on the map!")

9. Viewing Results in the UI¶

Your inference results are now available in the Geospatial Studio UI!

To view results:

1. Navigate to Model Catalog

   • Click "Start fine-tuning" on the home page
   • Or go directly to the Model Catalog

2. Find Your Model

   • Look for "burn-scars-demo" in the list
   • Click on the model card

3. View Inference History

   • Click on the "History" tab
   • Find "Park Fire Aug 2024"
   • Click to open the inference page

4. Explore the Map

   • Input RGB: Original satellite imagery
   • Model Prediction: Burn scar detection overlay (red areas)
   • Use layer controls to toggle visibility
   • Zoom and pan to explore details

What to Look For:

• Red areas indicate detected burn scars
• Compare with the input imagery to validate results
• Check for false positives/negatives
• Assess model performance on this test case

---

9. Advanced Topics: Customizing Your Workflow¶

Now that you've completed the end-to-end workflow, let's explore how to customize it for your own use cases.

---

9.1 Preparing Your Own Dataset¶

To train models on your own data, you need to prepare a dataset configuration that follows the Studio format.

Dataset Structure Requirements:

1. Data Organization:

   your-dataset/
   ├── train/
   │   ├── images/
   │   │   ├── scene_001_merged.tif
   │   │   ├── scene_002_merged.tif
   │   │   └── ...
   │   └── labels/
   │       ├── scene_001.mask.tif
   │       ├── scene_002.mask.tif
   │       └── ...
   ├── val/
   │   ├── images/
   │   └── labels/
   └── test/
       ├── images/
       └── labels/
   

2. Image Requirements:

   • Format: GeoTIFF (.tif)
   • Size: Typically 512x512 pixels (can vary)
   • Bands: Multi-spectral (e.g., RGB, NIR, SWIR)
   • Data type: Float32 or UInt16
   • Georeferencing: CRS and geotransform metadata

3. Label Requirements:

   • Format: GeoTIFF (.tif)
   • Same spatial dimensions as input images
   • Single band with integer class values
   • For segmentation: 0 = no data, -1 = ignore, 1+ = class labels
   • For regression: Float values representing the target variable

Dataset Configuration JSON:

Create a JSON file describing your dataset:

In [ ]:

# Example: Custom dataset configuration
custom_dataset_config = {
    "dataset_name": "My Custom Dataset",
    "dataset_url": "https://your-storage.com/your-dataset.zip",  # Or local path
    "label_suffix": ".mask.tif",  # How to identify label files
    "description": "Description of your dataset and its purpose",
    "purpose": "Segmentation",  # Or "Regression"
    "data_sources": [
        {
            "bands": [
                {
                    "index": "0",
                    "band_name": "Blue",
                    "scaling_factor": "0.0001",  # Scale raw values to 0-1 range
                    "RGB_band": "B"  # Optional: for visualization
                },
                {
                    "index": "1",
                    "band_name": "Green",
                    "scaling_factor": "0.0001",
                    "RGB_band": "G"
                },
                {
                    "index": "2",
                    "band_name": "Red",
                    "scaling_factor": "0.0001",
                    "RGB_band": "R"
                },
                {
                    "index": "3",
                    "band_name": "NIR",
                    "scaling_factor": "0.0001"
                }
            ],
            "connector": "sentinelhub",  # Data source type
            "collection": "your_collection",
            "file_suffix": "_merged.tif",  # How to identify image files
            "modality_tag": "YOUR_MODALITY"  # e.g., "SENTINEL2", "LANDSAT8"
        }
    ],
    "label_categories": [  # For segmentation only
        {
            "id": "1",
            "name": "Class 1",
            "color": "#ff0000",  # Hex color for visualization
            "opacity": 1
        },
        {
            "id": "2",
            "name": "Class 2",
            "color": "#00ff00",
            "opacity": 1
        },
        {
            "id": "0",
            "name": "No data",
            "color": "#000000",
            "opacity": 0
        },
        {
            "id": "-1",
            "name": "Ignore",
            "color": "#000000",
            "opacity": 0
        }
    ],
    "version": "v2"
}

# Save to file
import json
with open('my_custom_dataset.json', 'w') as f:
    json.dump(custom_dataset_config, f, indent=2)

print("✅ Custom dataset configuration created!")
print("\n💡 Tips:")
print("- Ensure your data is properly georeferenced")
print("- Use consistent file naming conventions")
print("- Validate label values match your categories")
print("- Test with a small subset first")

# Example: Custom dataset configuration custom_dataset_config = { "dataset_name": "My Custom Dataset", "dataset_url": "https://your-storage.com/your-dataset.zip", # Or local path "label_suffix": ".mask.tif", # How to identify label files "description": "Description of your dataset and its purpose", "purpose": "Segmentation", # Or "Regression" "data_sources": [ { "bands": [ { "index": "0", "band_name": "Blue", "scaling_factor": "0.0001", # Scale raw values to 0-1 range "RGB_band": "B" # Optional: for visualization }, { "index": "1", "band_name": "Green", "scaling_factor": "0.0001", "RGB_band": "G" }, { "index": "2", "band_name": "Red", "scaling_factor": "0.0001", "RGB_band": "R" }, { "index": "3", "band_name": "NIR", "scaling_factor": "0.0001" } ], "connector": "sentinelhub", # Data source type "collection": "your_collection", "file_suffix": "_merged.tif", # How to identify image files "modality_tag": "YOUR_MODALITY" # e.g., "SENTINEL2", "LANDSAT8" } ], "label_categories": [ # For segmentation only { "id": "1", "name": "Class 1", "color": "#ff0000", # Hex color for visualization "opacity": 1 }, { "id": "2", "name": "Class 2", "color": "#00ff00", "opacity": 1 }, { "id": "0", "name": "No data", "color": "#000000", "opacity": 0 }, { "id": "-1", "name": "Ignore", "color": "#000000", "opacity": 0 } ], "version": "v2" } # Save to file import json with open('my_custom_dataset.json', 'w') as f: json.dump(custom_dataset_config, f, indent=2) print("✅ Custom dataset configuration created!") print("\n💡 Tips:") print("- Ensure your data is properly georeferenced") print("- Use consistent file naming conventions") print("- Validate label values match your categories") print("- Test with a small subset first")

Onboard Your Custom Dataset:

In [ ]:

# Onboard your custom dataset
# with open('my_custom_dataset.json', 'r') as f:
#     my_dataset = json.load(f)
# 
# onboard_response = client.onboard_dataset(data=my_dataset)
# print(json.dumps(onboard_response, indent=2))
# 
# # Monitor onboarding
# client.poll_onboard_dataset_until_finished(onboard_response['dataset_id'])

print("📝 Uncomment the code above to onboard your custom dataset")

# Onboard your custom dataset # with open('my_custom_dataset.json', 'r') as f: # my_dataset = json.load(f) # # onboard_response = client.onboard_dataset(data=my_dataset) # print(json.dumps(onboard_response, indent=2)) # # # Monitor onboarding # client.poll_onboard_dataset_until_finished(onboard_response['dataset_id']) print("📝 Uncomment the code above to onboard your custom dataset")

9.2 Available Foundation Models¶

Geospatial Studio supports multiple foundation models. Each has different characteristics:

Model	Parameters	Best For	Strengths
Prithvi-EO-V1-100M	100M	General geospatial tasks	Balanced performance, well-tested
Prithvi-EO-V2-300M	300M	Complex patterns, large datasets	Higher capacity, better accuracy
Prithvi-EO-V2-600M	600M	State-of-the-art performance	Highest accuracy, requires more GPU
Clay-V1-Base	~300M	Multi-modal data	Handles diverse data types
TerraMind-V1-Base	~300M	Temporal analysis	Excellent for time series
TerraMind-V1-Large	~600M	Advanced temporal tasks	Best temporal understanding
ResNet-50	25M	Lightweight tasks	Fast training, lower GPU needs
ConvNeXt-Large	200M	Computer vision tasks	Strong baseline performance

Choosing a Model:

• For beginners: Start with Prithvi-EO-V1-100M or ResNet-50
• For production: Use Prithvi-EO-V2-300M or Clay-V1-Base
• For research: Try Prithvi-EO-V2-600M or TerraMind-V1-Large
• For limited GPU: Use ResNet-50 or Prithvi-EO-V1-100M
• For temporal data: Use TerraMind models

Example: Using Clay Foundation Model:

In [ ]:

# Example: Register Clay foundation model
clay_backbone = {
    "name": "clay_v1_base",
    "description": "Clay is a foundational model of Earth trained as a self-supervised Masked Autoencoder (MAE)",
    "checkpoint_filename": "clay_v1_base/clay_v1_base",
    "model_params": {
        "backbone": "clay_v1_base",
        "embed_dim": 768,
        "num_heads": 12,
        "tile_size": 1,
        "num_layers": 12,
        "patch_size": 16,
        "tubelet_size": 1,
        "model_category": "clay"
    }
}

# Register the model
# clay_response = client.create_base_model(clay_backbone)
# print(f"Clay model registered with ID: {clay_response['id']}")

print("📝 Uncomment to register Clay model")
print("\n💡 Note: Clay models work best with 512x512 images")

# Example: Register Clay foundation model clay_backbone = { "name": "clay_v1_base", "description": "Clay is a foundational model of Earth trained as a self-supervised Masked Autoencoder (MAE)", "checkpoint_filename": "clay_v1_base/clay_v1_base", "model_params": { "backbone": "clay_v1_base", "embed_dim": 768, "num_heads": 12, "tile_size": 1, "num_layers": 12, "patch_size": 16, "tubelet_size": 1, "model_category": "clay" } } # Register the model # clay_response = client.create_base_model(clay_backbone) # print(f"Clay model registered with ID: {clay_response['id']}") print("📝 Uncomment to register Clay model") print("\n💡 Note: Clay models work best with 512x512 images")

9.3 Task Templates and Hyperparameters¶

Task templates define the learning task type and training configuration. Studio supports:

1. Segmentation Templates:

• Generic Segmentation: Works with Prithvi models
• Clay Segmentation: Optimized for Clay models
• TerraMind Segmentation: For temporal segmentation
• ResNet/ConvNeXt Segmentation: For CNN-based models

2. Regression Templates:

• For continuous value prediction (e.g., biomass, temperature)
• Supports all foundation models

Key Hyperparameters to Tune:

# Training Configuration
{
    "runner": {
        "max_epochs": 10,              # Number of training epochs
        "early_stopping_patience": 20,  # Stop if no improvement
        "early_stopping_monitor": "val/loss"
    },
    "optimizer": {
        "type": "Adam",                # Adam, SGD, AdamW, RMSProp
        "lr": 0.0001,                  # Learning rate
        "weight_decay": 0.05           # L2 regularization
    },
    "data": {
        "batch_size": 4,               # Samples per batch
        "workers_per_gpu": 2           # Data loading workers
    },
    "model": {
        "frozen_backbone": false,      # Freeze pretrained weights?
        "decode_head": {
            "decoder": "UNetDecoder",   # UNetDecoder or UperNetDecoder
            "channels": 256,            # Decoder channels
            "num_convs": 4,             # Convolutional blocks
            "loss_decode": {
                "type": "CrossEntropyLoss"  # Loss function
            }
        }
    }
}

Tuning Tips:

1. Learning Rate: Start with 1e-4, adjust based on loss curves
2. Batch Size: Larger = faster but needs more GPU memory
3. Epochs: Start with 10-20, increase if underfitting
4. Frozen Backbone: Set to true for faster training with less data
5. Decoder: UNet is simpler, UperNet is more powerful

Example: Custom Training Configuration:

In [ ]:

# Example: Submit fine-tuning with custom hyperparameters
custom_tune_payload = {
    "name": "my-custom-model",
    "description": "Custom model with tuned hyperparameters",
    "dataset_id": "your_dataset_id",
    "base_model_id": "your_base_model_id",
    "tune_template_id": "your_template_id",
    # Override template defaults
    "runner": {
        "max_epochs": 20,  # Train longer
        "early_stopping_patience": 10
    },
    "optimizer": {
        "lr": 0.00005,  # Lower learning rate
        "type": "AdamW",  # Different optimizer
        "weight_decay": 0.01
    },
    "data": {
        "batch_size": 8  # Larger batch size
    }
}

# Submit the tune
# tune_response = client.submit_tune(custom_tune_payload, output='json')
# print(json.dumps(tune_response, indent=2))

print("📝 Uncomment to submit custom training job")
print("\n💡 Experiment with different hyperparameters to optimize performance")

# Example: Submit fine-tuning with custom hyperparameters custom_tune_payload = { "name": "my-custom-model", "description": "Custom model with tuned hyperparameters", "dataset_id": "your_dataset_id", "base_model_id": "your_base_model_id", "tune_template_id": "your_template_id", # Override template defaults "runner": { "max_epochs": 20, # Train longer "early_stopping_patience": 10 }, "optimizer": { "lr": 0.00005, # Lower learning rate "type": "AdamW", # Different optimizer "weight_decay": 0.01 }, "data": { "batch_size": 8 # Larger batch size } } # Submit the tune # tune_response = client.submit_tune(custom_tune_payload, output='json') # print(json.dumps(tune_response, indent=2)) print("📝 Uncomment to submit custom training job") print("\n💡 Experiment with different hyperparameters to optimize performance")

9.4 Comparing Models and Experiments¶

To find the best model for your task, train multiple variants and compare:

Experiment Matrix:

Experiment	Base Model	Template	Learning Rate	Epochs	Notes
baseline	Prithvi-V1-100M	Segmentation	1e-4	10	Quick baseline
exp-1	Prithvi-V2-300M	Segmentation	1e-4	10	Larger model
exp-2	Prithvi-V2-300M	Segmentation	5e-5	20	Lower LR, more epochs
exp-3	Clay-V1-Base	Clay Seg	6e-5	15	Different architecture
exp-4	Prithvi-V2-300M	Segmentation	1e-4	10	Frozen backbone

Tracking Experiments:

In [ ]:

# Example: Run multiple experiments
experiments = [
    {
        "name": "baseline-prithvi-v1",
        "base_model_id": "prithvi_v1_100m_id",
        "optimizer": {"lr": 0.0001}
    },
    {
        "name": "exp1-prithvi-v2-300m",
        "base_model_id": "prithvi_v2_300m_id",
        "optimizer": {"lr": 0.0001}
    },
    {
        "name": "exp2-lower-lr",
        "base_model_id": "prithvi_v2_300m_id",
        "optimizer": {"lr": 0.00005},
        "runner": {"max_epochs": 20}
    }
]

# Submit all experiments
# tune_ids = []
# for exp in experiments:
#     payload = {
#         "name": exp["name"],
#         "dataset_id": dataset_id,
#         "base_model_id": exp["base_model_id"],
#         "tune_template_id": tune_template_id,
#         **{k: v for k, v in exp.items() if k not in ["name", "base_model_id"]}
#     }
#     response = client.submit_tune(payload, output='json')
#     tune_ids.append(response['tune_id'])
#     print(f"✅ Submitted: {exp['name']} (ID: {response['tune_id']})")

print("📝 Uncomment to run experiment matrix")
print("\n💡 Compare results in MLflow UI to find the best model")

# Example: Run multiple experiments experiments = [ { "name": "baseline-prithvi-v1", "base_model_id": "prithvi_v1_100m_id", "optimizer": {"lr": 0.0001} }, { "name": "exp1-prithvi-v2-300m", "base_model_id": "prithvi_v2_300m_id", "optimizer": {"lr": 0.0001} }, { "name": "exp2-lower-lr", "base_model_id": "prithvi_v2_300m_id", "optimizer": {"lr": 0.00005}, "runner": {"max_epochs": 20} } ] # Submit all experiments # tune_ids = [] # for exp in experiments: # payload = { # "name": exp["name"], # "dataset_id": dataset_id, # "base_model_id": exp["base_model_id"], # "tune_template_id": tune_template_id, # **{k: v for k, v in exp.items() if k not in ["name", "base_model_id"]} # } # response = client.submit_tune(payload, output='json') # tune_ids.append(response['tune_id']) # print(f"✅ Submitted: {exp['name']} (ID: {response['tune_id']})") print("📝 Uncomment to run experiment matrix") print("\n💡 Compare results in MLflow UI to find the best model")

9.5 Running Inference on Custom Data¶

Once you have a trained model, you can run inference on new data:

Inference Options:

1. URL-based: Point to remote GeoTIFF files
2. Tile-based: Use map tile coordinates (z/x/y)
3. Polygon-based: Define area of interest with GeoJSON
4. Bbox-based: Specify bounding box coordinates

Example: Inference on Multiple Locations:

In [ ]:

# Example: Run inference on multiple locations
locations = [
    {
        "name": "Location 1",
        "url": "https://your-storage.com/location1.tif",
        "date": "2024-08-15"
    },
    {
        "name": "Location 2",
        "url": "https://your-storage.com/location2.tif",
        "date": "2024-08-20"
    }
]

# Submit inference for each location
# for loc in locations:
#     payload = {
#         "model_display_name": "my-custom-model",
#         "location": loc["name"],
#         "description": f"Inference on {loc['name']}",
#         "spatial_domain": {
#             "urls": [loc["url"]],
#             "bbox": [],
#             "tiles": [],
#             "polygons": []
#         },
#         "temporal_domain": [loc["date"]],
#         # ... rest of inference configuration
#     }
#     response = client.try_out_tune(tune_id=your_tune_id, data=payload)
#     print(f"✅ Inference submitted for {loc['name']}")

print("📝 Uncomment to run batch inference")
print("\n💡 Use this for processing multiple areas or time periods")

# Example: Run inference on multiple locations locations = [ { "name": "Location 1", "url": "https://your-storage.com/location1.tif", "date": "2024-08-15" }, { "name": "Location 2", "url": "https://your-storage.com/location2.tif", "date": "2024-08-20" } ] # Submit inference for each location # for loc in locations: # payload = { # "model_display_name": "my-custom-model", # "location": loc["name"], # "description": f"Inference on {loc['name']}", # "spatial_domain": { # "urls": [loc["url"]], # "bbox": [], # "tiles": [], # "polygons": [] # }, # "temporal_domain": [loc["date"]], # # ... rest of inference configuration # } # response = client.try_out_tune(tune_id=your_tune_id, data=payload) # print(f"✅ Inference submitted for {loc['name']}") print("📝 Uncomment to run batch inference") print("\n💡 Use this for processing multiple areas or time periods")

9.6 Best Practices Summary¶

Data Preparation:

• ✅ Ensure consistent image sizes and formats
• ✅ Validate label values and categories
• ✅ Split data into train/val/test sets (70/15/15)
• ✅ Check for data quality issues (clouds, artifacts)
• ✅ Document band configurations and scaling factors

Model Selection:

• ✅ Start with smaller models for prototyping
• ✅ Use larger models for production
• ✅ Consider GPU memory constraints
• ✅ Match model to data characteristics (temporal, multi-modal)

Training:

• ✅ Monitor training curves in MLflow
• ✅ Use early stopping to prevent overfitting
• ✅ Experiment with learning rates
• ✅ Try frozen backbone for small datasets
• ✅ Save best checkpoints

Evaluation:

• ✅ Test on held-out data
• ✅ Visual inspection of predictions
• ✅ Compare multiple model variants
• ✅ Validate on diverse geographic regions
• ✅ Check for edge cases and failure modes

Deployment:

• ✅ Document model version and configuration
• ✅ Test inference pipeline end-to-end
• ✅ Monitor inference performance
• ✅ Plan for model updates and retraining

---

10. Summary and Key Takeaways¶

Congratulations! You've completed an end-to-end machine learning workflow for geospatial AI. 🎉

What You Accomplished¶

✅ Registered a foundation model (Prithvi-EO-V2-300M)

✅ Onboarded a training dataset (804 labeled burn scar scenes)

✅ Submitted a fine-tuning job (customized model for burn scar detection)

✅ Monitored training progress (tracked job status and completion)

✅ Ran inference (tested model on new wildfire data)

✅ Visualized results (viewed predictions in the UI)

Key Concepts Learned¶

1. Foundation Models: Pre-trained models provide a strong starting point
2. Fine-Tuning: Customizing models for specific tasks and domains
3. Dataset Onboarding: Proper data preparation is crucial for training
4. GPU Management: Resource allocation affects training efficiency
5. Inference Pipelines: Multi-step process from data to visualization
6. Model Evaluation: Visual inspection helps assess performance

Real-World Applications¶

This workflow can be adapted for:

• Disaster Response: Rapid damage assessment
• Environmental Monitoring: Deforestation, land use change
• Agriculture: Crop health, yield prediction
• Urban Planning: Infrastructure mapping, growth tracking
• Climate Science: Ice sheet monitoring, flood mapping

Next Steps¶

1. Experiment with Different Models

   • Try other foundation models
   • Compare performance across models

2. Customize Training

   • Adjust hyperparameters in task templates
   • Experiment with different architectures

3. Use Your Own Data

   • Prepare custom datasets
   • Train models for your specific use case

4. Integrate with Applications

   • Use the API for automated workflows
   • Build custom applications on top of Studio

5. Explore Advanced Features

   • Multi-temporal analysis
   • Ensemble models
   • Active learning workflows

Resources¶

• Documentation: Geospatial Studio Docs
• SDK Reference: Geospatial Studio SDK
• Dataset: HLS Burn Scars on Hugging Face
• Foundation Models: IBM Geospatial Models

---

🎓 Workshop Complete!¶

You've now completed all four labs of the Geospatial Studio workshop:

• Lab 1: Getting Started with the SDK
• Lab 2: Onboarding Pre-computed Examples
• Lab 3: Uploading Model Checkpoints and Running Inference
• Lab 4: End-to-End Burn Scars Workflow ✨

Thank you for participating! We hope you found this workshop valuable and are excited to build geospatial AI applications with IBM Geospatial Studio.

Questions or Feedback?

• Open an issue on GitHub
• Join our community discussions
• Contact the development team

Happy mapping! 🌍🛰️