--- name: cellxgene-census description: Query the CZ CELLxGENE Census programmatically for versioned public single-cell and spatial transcriptomics data. Use when you need population-scale cell metadata, gene expression slices, Census summary counts, source H5AD URIs/downloads, embeddings, spatial Census data, or reference atlas comparisons across organisms, tissues, diseases, assays, and cell types. For analyzing your own local single-cell data use scanpy, anndata, or scvi-tools. allowed-tools: Read Write Edit Bash license: MIT compatibility: Requires Python >=3.10,<3.13. Examples target cellxgene-census 1.17.x and the 2025-11-08 stable LTS Census; spatial workflows need the spatial extra and TileDB-SOMA >=1.15.5. No authentication is required for public Census data. metadata: version: "1.2" skill-author: K-Dense Inc. ---

# CZ CELLxGENE Census

## Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.

The Census includes: - **217+ million total cells** and **125+ million unique cells** in the 2025-11-08 stable LTS release - **1,845 datasets** in the 2025-11-08 stable LTS release - **Human, mouse, marmoset, rhesus macaque, and chimpanzee** data in the current schema - **Standardized metadata** (cell types, tissues, diseases, donors) - **Raw gene expression** matrices and source H5AD lookup/download helpers - **Pre-calculated summary counts, embeddings, and spatial data** - **Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools**

## When to Use This Skill

This skill should be used when: - Querying single-cell expression data by cell type, tissue, or disease - Exploring available single-cell datasets and metadata - Training machine learning models on single-cell data - Performing large-scale cross-dataset analyses - Integrating Census data with scanpy or other analysis frameworks - Computing statistics across millions of cells - Accessing pre-calculated embeddings or model predictions

## Installation and Setup

Install the Census API: ```bash uv pip install "cellxgene-census==1.17.*" ```

For spatial workflows: ```bash uv pip install "cellxgene-census[spatial]==1.17.*" "spatialdata[extra]>=0.2.5" ```

For PyTorch model training, use TileDB-SOMA-ML. The old `cellxgene_census.experimental.ml` loaders are deprecated:

```bash uv pip install "cellxgene-census==1.17.*" tiledbsoma-ml ```

## Core Workflow Patterns

Eight patterns, each with code, are in [references/core_workflow_patterns.md](references/core_workflow_patterns.md):

1. **Opening the Census** — always pin `census_version` so an analysis stays reproducible. 2. **Exploring Census information** — available datasets, cell counts, and summary tables. 3. **Querying expression data** — small to medium scale into an `AnnData`. 4. **Large-scale queries** — out-of-core processing when the slice will not fit in memory. 5. **Machine learning with PyTorch** — the Census data loaders. 6. **Spatial Census data** — accessing spatial assays. 7. **Integration with Scanpy** — handing a Census slice to a standard Scanpy workflow. 8. **Multi-dataset integration** — combining datasets and handling batch effects.

## Key Concepts and Best Practices

### Always Filter for Primary Data Unless analyzing duplicates, always include `is_primary_data == True` in queries to avoid counting cells multiple times: ```python obs_value_filter="cell_type == 'B cell' and is_primary_data == True" ```

### Specify Census Version for Reproducibility Always specify the Census version in production analyses: ```python census = cellxgene_census.open_soma(census_version="2025-11-08") ```

### Estimate Query Size Before Loading For large queries, first check the number of cells to avoid memory issues: ```python # Get cell count metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'brain' and is_primary_data == True", column_names=["soma_joinid"] ) n_cells = len(metadata) print(f"Query will return {n_cells:,} cells")

# If too large (>100k), use out-of-core processing ```

### Use tissue_general for Broader Groupings The `tissue_general` field provides coarser categories than `tissue`, useful for cross-tissue analyses: ```python # Broader grouping obs_value_filter="tissue_general == 'immune system'"

# Specific tissue obs_value_filter="tissue == 'peripheral blood mononuclear cell'" ```

### Select Only Needed Columns Minimize data transfer by specifying only required metadata columns: ```python obs_column_names=["cell_type", "tissue_general", "disease"] # Not all columns ```

### Check Dataset Presence for Gene-Specific Queries When analyzing specific genes, verify which datasets measured them: ```python presence = cellxgene_census.get_presence_matrix( census, "homo_sapiens", var_value_filter="feature_name in ['CD4', 'CD8A']" ) ```

### Two-Step Workflow: Explore Then Query First explore metadata to understand available data, then query expression: ```python # Step 1: Explore what's available metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="disease == 'COVID-19' and is_primary_data == True", column_names=["cell_type", "tissue_general"] ) print(metadata.value_counts())

# Step 2: Query based on findings adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True", ) ```

## Available Metadata Fields

### Cell Metadata (obs) Key fields for filtering: - `cell_type`, `cell_type_ontology_term_id` - `tissue`, `tissue_general`, `tissue_ontology_term_id` - `disease`, `disease_ontology_term_id` - `assay`, `assay_ontology_term_id` - `donor_id`, `sex`, `self_reported_ethnicity` - `development_stage`, `development_stage_ontology_term_id` - `dataset_id` - `is_primary_data` (Boolean: True = unique cell)

The current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with `list(census["census_data"].keys())`.

### Gene Metadata (var) - `feature_id` (Ensembl gene ID, e.g., "ENSG00000161798") - `feature_name` (Gene symbol, e.g., "FOXP2") - `feature_type` - `feature_length` (Gene length in base pairs) - `nnz`, `n_measured_obs` (availability summaries useful for checking sparsity and coverage)

## Reference Documentation

This skill includes detailed reference documentation:

### references/census_schema.md Comprehensive documentation of: - Census data structure and organization - All available metadata fields - Value filter syntax and operators - SOMA object types - Data inclusion criteria

**When to read:** When you need detailed schema information, full list of metadata fields, or complex filter syntax.

### references/common_patterns.md Examples and patterns for: - Exploratory queries (metadata only) - Small-to-medium queries (AnnData) - Large queries (out-of-core processing) - PyTorch integration - Spatial Census access patterns - Scanpy integration workflows - Multi-dataset integration - Best practices and common pitfalls

**When to read:** When implementing specific query patterns, looking for code examples, or troubleshooting common issues.

## Common Use Cases

### Use Case 1: Explore Cell Types in a Tissue ```python with cellxgene_census.open_soma() as census: cells = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'lung' and is_primary_data == True", column_names=["cell_type"] ) print(cells["cell_type"].value_counts()) ```

### Use Case 2: Query Marker Gene Expression ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']", obs_value_filter="cell_type in ['T cell', 'B cell'] and is_primary_data == True", ) ```

### Use Case 3: Train Cell Type Classifier ```python import tiledbsoma as soma from tiledbsoma_ml import ExperimentDataset, experiment_dataloader

with cellxgene_census.open_soma() as census: experiment = census["census_data"]["homo_sapiens"] with experiment.axis_query( measurement_name="RNA", obs_query=soma.AxisQuery(value_filter="is_primary_data == True"), ) as query: dataset = ExperimentDataset( query=query, layer_name="raw", obs_column_names=["cell_type"], batch_size=128, shuffle=True, ) dataloader = experiment_dataloader(dataset)

for X, obs in dataloader: labels = obs["cell_type"] # Training logic pass ```

### Use Case 4: Cross-Tissue Analysis ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True", )

# Analyze macrophage differences across tissues sc.tl.rank_genes_groups(adata, groupby="tissue_general") ```

## Troubleshooting

### Query Returns Too Many Cells - Add more specific filters to reduce scope - Use `tissue` instead of `tissue_general` for finer granularity - Filter by specific `dataset_id` if known - Switch to out-of-core processing for large queries

### Memory Errors - Reduce query scope with more restrictive filters - Select fewer genes with `var_value_filter` - Use out-of-core processing with `axis_query()` - Process data in batches

### Duplicate Cells in Results - Always include `is_primary_data == True` in filters - Check if intentionally querying across multiple datasets

### Gene Not Found - Verify gene name spelling (case-sensitive) - Try Ensembl ID with `feature_id` instead of `feature_name` - Check dataset presence matrix to see if gene was measured - Some genes may have been filtered during Census construction

### Version Inconsistencies - Always specify `census_version` explicitly - Use same version across all analyses - Check release notes for version-specific changes

--- name: cellxgene-census description: Query the CZ CELLxGENE Census programmatically for versioned public single-cell and spatial transcriptomics data. Use when you need population-scale cell metadata, gene expression slices, Census summary counts, source H5AD URIs/downloads, embeddings, spatial Census data, or reference atlas comparisons across organisms, tissues, diseases, assays, and cell types. For analyzing your own local single-cell data use scanpy, anndata, or scvi-tools. allowed-tools: Read Write Edit Bash license: MIT compatibility: Requires Python >=3.10,<3.13. Examples target cellxgene-census 1.17.x and the 2025-11-08 stable LTS Census; spatial workflows need the spatial extra and TileDB-SOMA >=1.15.5. No authentication is required for public Census data. metadata: version: "1.2" skill-author: K-Dense Inc. ---

# CZ CELLxGENE Census

## Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.

The Census includes: - **217+ million total cells** and **125+ million unique cells** in the 2025-11-08 stable LTS release - **1,845 datasets** in the 2025-11-08 stable LTS release - **Human, mouse, marmoset, rhesus macaque, and chimpanzee** data in the current schema - **Standardized metadata** (cell types, tissues, diseases, donors) - **Raw gene expression** matrices and source H5AD lookup/download helpers - **Pre-calculated summary counts, embeddings, and spatial data** - **Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools**

## When to Use This Skill

This skill should be used when: - Querying single-cell expression data by cell type, tissue, or disease - Exploring available single-cell datasets and metadata - Training machine learning models on single-cell data - Performing large-scale cross-dataset analyses - Integrating Census data with scanpy or other analysis frameworks - Computing statistics across millions of cells - Accessing pre-calculated embeddings or model predictions

## Installation and Setup

Install the Census API: ```bash uv pip install "cellxgene-census==1.17.*" ```

For spatial workflows: ```bash uv pip install "cellxgene-census[spatial]==1.17.*" "spatialdata[extra]>=0.2.5" ```

For PyTorch model training, use TileDB-SOMA-ML. The old `cellxgene_census.experimental.ml` loaders are deprecated:

```bash uv pip install "cellxgene-census==1.17.*" tiledbsoma-ml ```

## Core Workflow Patterns

Eight patterns, each with code, are in [references/core_workflow_patterns.md](references/core_workflow_patterns.md):

1. **Opening the Census** — always pin `census_version` so an analysis stays reproducible. 2. **Exploring Census information** — available datasets, cell counts, and summary tables. 3. **Querying expression data** — small to medium scale into an `AnnData`. 4. **Large-scale queries** — out-of-core processing when the slice will not fit in memory. 5. **Machine learning with PyTorch** — the Census data loaders. 6. **Spatial Census data** — accessing spatial assays. 7. **Integration with Scanpy** — handing a Census slice to a standard Scanpy workflow. 8. **Multi-dataset integration** — combining datasets and handling batch effects.

## Key Concepts and Best Practices

### Always Filter for Primary Data Unless analyzing duplicates, always include `is_primary_data == True` in queries to avoid counting cells multiple times: ```python obs_value_filter="cell_type == 'B cell' and is_primary_data == True" ```

### Specify Census Version for Reproducibility Always specify the Census version in production analyses: ```python census = cellxgene_census.open_soma(census_version="2025-11-08") ```

### Estimate Query Size Before Loading For large queries, first check the number of cells to avoid memory issues: ```python # Get cell count metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'brain' and is_primary_data == True", column_names=["soma_joinid"] ) n_cells = len(metadata) print(f"Query will return {n_cells:,} cells")

# If too large (>100k), use out-of-core processing ```

### Use tissue_general for Broader Groupings The `tissue_general` field provides coarser categories than `tissue`, useful for cross-tissue analyses: ```python # Broader grouping obs_value_filter="tissue_general == 'immune system'"

# Specific tissue obs_value_filter="tissue == 'peripheral blood mononuclear cell'" ```

### Select Only Needed Columns Minimize data transfer by specifying only required metadata columns: ```python obs_column_names=["cell_type", "tissue_general", "disease"] # Not all columns ```

### Check Dataset Presence for Gene-Specific Queries When analyzing specific genes, verify which datasets measured them: ```python presence = cellxgene_census.get_presence_matrix( census, "homo_sapiens", var_value_filter="feature_name in ['CD4', 'CD8A']" ) ```

### Two-Step Workflow: Explore Then Query First explore metadata to understand available data, then query expression: ```python # Step 1: Explore what's available metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="disease == 'COVID-19' and is_primary_data == True", column_names=["cell_type", "tissue_general"] ) print(metadata.value_counts())

# Step 2: Query based on findings adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True", ) ```

## Available Metadata Fields

### Cell Metadata (obs) Key fields for filtering: - `cell_type`, `cell_type_ontology_term_id` - `tissue`, `tissue_general`, `tissue_ontology_term_id` - `disease`, `disease_ontology_term_id` - `assay`, `assay_ontology_term_id` - `donor_id`, `sex`, `self_reported_ethnicity` - `development_stage`, `development_stage_ontology_term_id` - `dataset_id` - `is_primary_data` (Boolean: True = unique cell)

The current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with `list(census["census_data"].keys())`.

### Gene Metadata (var) - `feature_id` (Ensembl gene ID, e.g., "ENSG00000161798") - `feature_name` (Gene symbol, e.g., "FOXP2") - `feature_type` - `feature_length` (Gene length in base pairs) - `nnz`, `n_measured_obs` (availability summaries useful for checking sparsity and coverage)

## Reference Documentation

This skill includes detailed reference documentation:

### references/census_schema.md Comprehensive documentation of: - Census data structure and organization - All available metadata fields - Value filter syntax and operators - SOMA object types - Data inclusion criteria

**When to read:** When you need detailed schema information, full list of metadata fields, or complex filter syntax.

### references/common_patterns.md Examples and patterns for: - Exploratory queries (metadata only) - Small-to-medium queries (AnnData) - Large queries (out-of-core processing) - PyTorch integration - Spatial Census access patterns - Scanpy integration workflows - Multi-dataset integration - Best practices and common pitfalls

**When to read:** When implementing specific query patterns, looking for code examples, or troubleshooting common issues.

## Common Use Cases

### Use Case 1: Explore Cell Types in a Tissue ```python with cellxgene_census.open_soma() as census: cells = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'lung' and is_primary_data == True", column_names=["cell_type"] ) print(cells["cell_type"].value_counts()) ```

### Use Case 2: Query Marker Gene Expression ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']", obs_value_filter="cell_type in ['T cell', 'B cell'] and is_primary_data == True", ) ```

### Use Case 3: Train Cell Type Classifier ```python import tiledbsoma as soma from tiledbsoma_ml import ExperimentDataset, experiment_dataloader

with cellxgene_census.open_soma() as census: experiment = census["census_data"]["homo_sapiens"] with experiment.axis_query( measurement_name="RNA", obs_query=soma.AxisQuery(value_filter="is_primary_data == True"), ) as query: dataset = ExperimentDataset( query=query, layer_name="raw", obs_column_names=["cell_type"], batch_size=128, shuffle=True, ) dataloader = experiment_dataloader(dataset)

for X, obs in dataloader: labels = obs["cell_type"] # Training logic pass ```

### Use Case 4: Cross-Tissue Analysis ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True", )

# Analyze macrophage differences across tissues sc.tl.rank_genes_groups(adata, groupby="tissue_general") ```

## Troubleshooting

### Query Returns Too Many Cells - Add more specific filters to reduce scope - Use `tissue` instead of `tissue_general` for finer granularity - Filter by specific `dataset_id` if known - Switch to out-of-core processing for large queries

### Memory Errors - Reduce query scope with more restrictive filters - Select fewer genes with `var_value_filter` - Use out-of-core processing with `axis_query()` - Process data in batches

### Duplicate Cells in Results - Always include `is_primary_data == True` in filters - Check if intentionally querying across multiple datasets

### Gene Not Found - Verify gene name spelling (case-sensitive) - Try Ensembl ID with `feature_id` instead of `feature_name` - Check dataset presence matrix to see if gene was measured - Some genes may have been filtered during Census construction

### Version Inconsistencies - Always specify `census_version` explicitly - Use same version across all analyses - Check release notes for version-specific changes

--- name: cellxgene-census description: Query the CZ CELLxGENE Census programmatically for versioned public single-cell and spatial transcriptomics data. Use when you need population-scale cell metadata, gene expression slices, Census summary counts, source H5AD URIs/downloads, embeddings, spatial Census data, or reference atlas comparisons across organisms, tissues, diseases, assays, and cell types. For analyzing your own local single-cell data use scanpy, anndata, or scvi-tools. allowed-tools: Read Write Edit Bash license: MIT compatibility: Requires Python >=3.10,<3.13. Examples target cellxgene-census 1.17.x and the 2025-11-08 stable LTS Census; spatial workflows need the spatial extra and TileDB-SOMA >=1.15.5. No authentication is required for public Census data. metadata: version: "1.2" skill-author: K-Dense Inc. ---

# CZ CELLxGENE Census

## Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.

The Census includes: - **217+ million total cells** and **125+ million unique cells** in the 2025-11-08 stable LTS release - **1,845 datasets** in the 2025-11-08 stable LTS release - **Human, mouse, marmoset, rhesus macaque, and chimpanzee** data in the current schema - **Standardized metadata** (cell types, tissues, diseases, donors) - **Raw gene expression** matrices and source H5AD lookup/download helpers - **Pre-calculated summary counts, embeddings, and spatial data** - **Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools**

## When to Use This Skill

This skill should be used when: - Querying single-cell expression data by cell type, tissue, or disease - Exploring available single-cell datasets and metadata - Training machine learning models on single-cell data - Performing large-scale cross-dataset analyses - Integrating Census data with scanpy or other analysis frameworks - Computing statistics across millions of cells - Accessing pre-calculated embeddings or model predictions

## Installation and Setup

Install the Census API: ```bash uv pip install "cellxgene-census==1.17.*" ```

For spatial workflows: ```bash uv pip install "cellxgene-census[spatial]==1.17.*" "spatialdata[extra]>=0.2.5" ```

For PyTorch model training, use TileDB-SOMA-ML. The old `cellxgene_census.experimental.ml` loaders are deprecated:

```bash uv pip install "cellxgene-census==1.17.*" tiledbsoma-ml ```

## Core Workflow Patterns

Eight patterns, each with code, are in [references/core_workflow_patterns.md](references/core_workflow_patterns.md):

1. **Opening the Census** — always pin `census_version` so an analysis stays reproducible. 2. **Exploring Census information** — available datasets, cell counts, and summary tables. 3. **Querying expression data** — small to medium scale into an `AnnData`. 4. **Large-scale queries** — out-of-core processing when the slice will not fit in memory. 5. **Machine learning with PyTorch** — the Census data loaders. 6. **Spatial Census data** — accessing spatial assays. 7. **Integration with Scanpy** — handing a Census slice to a standard Scanpy workflow. 8. **Multi-dataset integration** — combining datasets and handling batch effects.

## Key Concepts and Best Practices

### Always Filter for Primary Data Unless analyzing duplicates, always include `is_primary_data == True` in queries to avoid counting cells multiple times: ```python obs_value_filter="cell_type == 'B cell' and is_primary_data == True" ```

### Specify Census Version for Reproducibility Always specify the Census version in production analyses: ```python census = cellxgene_census.open_soma(census_version="2025-11-08") ```

### Estimate Query Size Before Loading For large queries, first check the number of cells to avoid memory issues: ```python # Get cell count metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'brain' and is_primary_data == True", column_names=["soma_joinid"] ) n_cells = len(metadata) print(f"Query will return {n_cells:,} cells")

# If too large (>100k), use out-of-core processing ```

### Use tissue_general for Broader Groupings The `tissue_general` field provides coarser categories than `tissue`, useful for cross-tissue analyses: ```python # Broader grouping obs_value_filter="tissue_general == 'immune system'"

# Specific tissue obs_value_filter="tissue == 'peripheral blood mononuclear cell'" ```

### Select Only Needed Columns Minimize data transfer by specifying only required metadata columns: ```python obs_column_names=["cell_type", "tissue_general", "disease"] # Not all columns ```

### Check Dataset Presence for Gene-Specific Queries When analyzing specific genes, verify which datasets measured them: ```python presence = cellxgene_census.get_presence_matrix( census, "homo_sapiens", var_value_filter="feature_name in ['CD4', 'CD8A']" ) ```

### Two-Step Workflow: Explore Then Query First explore metadata to understand available data, then query expression: ```python # Step 1: Explore what's available metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="disease == 'COVID-19' and is_primary_data == True", column_names=["cell_type", "tissue_general"] ) print(metadata.value_counts())

# Step 2: Query based on findings adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True", ) ```

## Available Metadata Fields

### Cell Metadata (obs) Key fields for filtering: - `cell_type`, `cell_type_ontology_term_id` - `tissue`, `tissue_general`, `tissue_ontology_term_id` - `disease`, `disease_ontology_term_id` - `assay`, `assay_ontology_term_id` - `donor_id`, `sex`, `self_reported_ethnicity` - `development_stage`, `development_stage_ontology_term_id` - `dataset_id` - `is_primary_data` (Boolean: True = unique cell)

The current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with `list(census["census_data"].keys())`.

### Gene Metadata (var) - `feature_id` (Ensembl gene ID, e.g., "ENSG00000161798") - `feature_name` (Gene symbol, e.g., "FOXP2") - `feature_type` - `feature_length` (Gene length in base pairs) - `nnz`, `n_measured_obs` (availability summaries useful for checking sparsity and coverage)

## Reference Documentation

This skill includes detailed reference documentation:

### references/census_schema.md Comprehensive documentation of: - Census data structure and organization - All available metadata fields - Value filter syntax and operators - SOMA object types - Data inclusion criteria

**When to read:** When you need detailed schema information, full list of metadata fields, or complex filter syntax.

### references/common_patterns.md Examples and patterns for: - Exploratory queries (metadata only) - Small-to-medium queries (AnnData) - Large queries (out-of-core processing) - PyTorch integration - Spatial Census access patterns - Scanpy integration workflows - Multi-dataset integration - Best practices and common pitfalls

**When to read:** When implementing specific query patterns, looking for code examples, or troubleshooting common issues.

## Common Use Cases

### Use Case 1: Explore Cell Types in a Tissue ```python with cellxgene_census.open_soma() as census: cells = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'lung' and is_primary_data == True", column_names=["cell_type"] ) print(cells["cell_type"].value_counts()) ```

### Use Case 2: Query Marker Gene Expression ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']", obs_value_filter="cell_type in ['T cell', 'B cell'] and is_primary_data == True", ) ```

### Use Case 3: Train Cell Type Classifier ```python import tiledbsoma as soma from tiledbsoma_ml import ExperimentDataset, experiment_dataloader

with cellxgene_census.open_soma() as census: experiment = census["census_data"]["homo_sapiens"] with experiment.axis_query( measurement_name="RNA", obs_query=soma.AxisQuery(value_filter="is_primary_data == True"), ) as query: dataset = ExperimentDataset( query=query, layer_name="raw", obs_column_names=["cell_type"], batch_size=128, shuffle=True, ) dataloader = experiment_dataloader(dataset)

for X, obs in dataloader: labels = obs["cell_type"] # Training logic pass ```

### Use Case 4: Cross-Tissue Analysis ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True", )

# Analyze macrophage differences across tissues sc.tl.rank_genes_groups(adata, groupby="tissue_general") ```

## Troubleshooting

### Query Returns Too Many Cells - Add more specific filters to reduce scope - Use `tissue` instead of `tissue_general` for finer granularity - Filter by specific `dataset_id` if known - Switch to out-of-core processing for large queries

### Memory Errors - Reduce query scope with more restrictive filters - Select fewer genes with `var_value_filter` - Use out-of-core processing with `axis_query()` - Process data in batches

### Duplicate Cells in Results - Always include `is_primary_data == True` in filters - Check if intentionally querying across multiple datasets

### Gene Not Found - Verify gene name spelling (case-sensitive) - Try Ensembl ID with `feature_id` instead of `feature_name` - Check dataset presence matrix to see if gene was measured - Some genes may have been filtered during Census construction

### Version Inconsistencies - Always specify `census_version` explicitly - Use same version across all analyses - Check release notes for version-specific changes

--- name: cellxgene-census description: Query the CZ CELLxGENE Census programmatically for versioned public single-cell and spatial transcriptomics data. Use when you need population-scale cell metadata, gene expression slices, Census summary counts, source H5AD URIs/downloads, embeddings, spatial Census data, or reference atlas comparisons across organisms, tissues, diseases, assays, and cell types. For analyzing your own local single-cell data use scanpy, anndata, or scvi-tools. allowed-tools: Read Write Edit Bash license: MIT compatibility: Requires Python >=3.10,<3.13. Examples target cellxgene-census 1.17.x and the 2025-11-08 stable LTS Census; spatial workflows need the spatial extra and TileDB-SOMA >=1.15.5. No authentication is required for public Census data. metadata: version: "1.2" skill-author: K-Dense Inc. ---

# CZ CELLxGENE Census

## Overview

The CZ CELLxGENE Census provides programmatic access to a comprehensive, versioned collection of standardized single-cell and spatial transcriptomics data from CZ CELLxGENE Discover. This skill enables efficient querying and analysis of public Census releases without downloading whole datasets first.

The Census includes: - **217+ million total cells** and **125+ million unique cells** in the 2025-11-08 stable LTS release - **1,845 datasets** in the 2025-11-08 stable LTS release - **Human, mouse, marmoset, rhesus macaque, and chimpanzee** data in the current schema - **Standardized metadata** (cell types, tissues, diseases, donors) - **Raw gene expression** matrices and source H5AD lookup/download helpers - **Pre-calculated summary counts, embeddings, and spatial data** - **Integration with AnnData, Scanpy, TileDB-SOMA, TileDB-SOMA-ML, and other analysis tools**

## When to Use This Skill

This skill should be used when: - Querying single-cell expression data by cell type, tissue, or disease - Exploring available single-cell datasets and metadata - Training machine learning models on single-cell data - Performing large-scale cross-dataset analyses - Integrating Census data with scanpy or other analysis frameworks - Computing statistics across millions of cells - Accessing pre-calculated embeddings or model predictions

## Installation and Setup

Install the Census API: ```bash uv pip install "cellxgene-census==1.17.*" ```

For spatial workflows: ```bash uv pip install "cellxgene-census[spatial]==1.17.*" "spatialdata[extra]>=0.2.5" ```

For PyTorch model training, use TileDB-SOMA-ML. The old `cellxgene_census.experimental.ml` loaders are deprecated:

```bash uv pip install "cellxgene-census==1.17.*" tiledbsoma-ml ```

## Core Workflow Patterns

Eight patterns, each with code, are in [references/core_workflow_patterns.md](references/core_workflow_patterns.md):

1. **Opening the Census** — always pin `census_version` so an analysis stays reproducible. 2. **Exploring Census information** — available datasets, cell counts, and summary tables. 3. **Querying expression data** — small to medium scale into an `AnnData`. 4. **Large-scale queries** — out-of-core processing when the slice will not fit in memory. 5. **Machine learning with PyTorch** — the Census data loaders. 6. **Spatial Census data** — accessing spatial assays. 7. **Integration with Scanpy** — handing a Census slice to a standard Scanpy workflow. 8. **Multi-dataset integration** — combining datasets and handling batch effects.

## Key Concepts and Best Practices

### Always Filter for Primary Data Unless analyzing duplicates, always include `is_primary_data == True` in queries to avoid counting cells multiple times: ```python obs_value_filter="cell_type == 'B cell' and is_primary_data == True" ```

### Specify Census Version for Reproducibility Always specify the Census version in production analyses: ```python census = cellxgene_census.open_soma(census_version="2025-11-08") ```

### Estimate Query Size Before Loading For large queries, first check the number of cells to avoid memory issues: ```python # Get cell count metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'brain' and is_primary_data == True", column_names=["soma_joinid"] ) n_cells = len(metadata) print(f"Query will return {n_cells:,} cells")

# If too large (>100k), use out-of-core processing ```

### Use tissue_general for Broader Groupings The `tissue_general` field provides coarser categories than `tissue`, useful for cross-tissue analyses: ```python # Broader grouping obs_value_filter="tissue_general == 'immune system'"

# Specific tissue obs_value_filter="tissue == 'peripheral blood mononuclear cell'" ```

### Select Only Needed Columns Minimize data transfer by specifying only required metadata columns: ```python obs_column_names=["cell_type", "tissue_general", "disease"] # Not all columns ```

### Check Dataset Presence for Gene-Specific Queries When analyzing specific genes, verify which datasets measured them: ```python presence = cellxgene_census.get_presence_matrix( census, "homo_sapiens", var_value_filter="feature_name in ['CD4', 'CD8A']" ) ```

### Two-Step Workflow: Explore Then Query First explore metadata to understand available data, then query expression: ```python # Step 1: Explore what's available metadata = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="disease == 'COVID-19' and is_primary_data == True", column_names=["cell_type", "tissue_general"] ) print(metadata.value_counts())

# Step 2: Query based on findings adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="disease == 'COVID-19' and cell_type == 'T cell' and is_primary_data == True", ) ```

## Available Metadata Fields

### Cell Metadata (obs) Key fields for filtering: - `cell_type`, `cell_type_ontology_term_id` - `tissue`, `tissue_general`, `tissue_ontology_term_id` - `disease`, `disease_ontology_term_id` - `assay`, `assay_ontology_term_id` - `donor_id`, `sex`, `self_reported_ethnicity` - `development_stage`, `development_stage_ontology_term_id` - `dataset_id` - `is_primary_data` (Boolean: True = unique cell)

The current schema includes organism collections beyond human and mouse. Confirm available organisms for the selected release with `list(census["census_data"].keys())`.

### Gene Metadata (var) - `feature_id` (Ensembl gene ID, e.g., "ENSG00000161798") - `feature_name` (Gene symbol, e.g., "FOXP2") - `feature_type` - `feature_length` (Gene length in base pairs) - `nnz`, `n_measured_obs` (availability summaries useful for checking sparsity and coverage)

## Reference Documentation

This skill includes detailed reference documentation:

### references/census_schema.md Comprehensive documentation of: - Census data structure and organization - All available metadata fields - Value filter syntax and operators - SOMA object types - Data inclusion criteria

**When to read:** When you need detailed schema information, full list of metadata fields, or complex filter syntax.

### references/common_patterns.md Examples and patterns for: - Exploratory queries (metadata only) - Small-to-medium queries (AnnData) - Large queries (out-of-core processing) - PyTorch integration - Spatial Census access patterns - Scanpy integration workflows - Multi-dataset integration - Best practices and common pitfalls

**When to read:** When implementing specific query patterns, looking for code examples, or troubleshooting common issues.

## Common Use Cases

### Use Case 1: Explore Cell Types in a Tissue ```python with cellxgene_census.open_soma() as census: cells = cellxgene_census.get_obs( census, "homo_sapiens", value_filter="tissue_general == 'lung' and is_primary_data == True", column_names=["cell_type"] ) print(cells["cell_type"].value_counts()) ```

### Use Case 2: Query Marker Gene Expression ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", var_value_filter="feature_name in ['CD4', 'CD8A', 'CD19']", obs_value_filter="cell_type in ['T cell', 'B cell'] and is_primary_data == True", ) ```

### Use Case 3: Train Cell Type Classifier ```python import tiledbsoma as soma from tiledbsoma_ml import ExperimentDataset, experiment_dataloader

with cellxgene_census.open_soma() as census: experiment = census["census_data"]["homo_sapiens"] with experiment.axis_query( measurement_name="RNA", obs_query=soma.AxisQuery(value_filter="is_primary_data == True"), ) as query: dataset = ExperimentDataset( query=query, layer_name="raw", obs_column_names=["cell_type"], batch_size=128, shuffle=True, ) dataloader = experiment_dataloader(dataset)

for X, obs in dataloader: labels = obs["cell_type"] # Training logic pass ```

### Use Case 4: Cross-Tissue Analysis ```python with cellxgene_census.open_soma() as census: adata = cellxgene_census.get_anndata( census=census, organism="Homo sapiens", obs_value_filter="cell_type == 'macrophage' and tissue_general in ['lung', 'liver', 'brain'] and is_primary_data == True", )

# Analyze macrophage differences across tissues sc.tl.rank_genes_groups(adata, groupby="tissue_general") ```

## Troubleshooting

### Query Returns Too Many Cells - Add more specific filters to reduce scope - Use `tissue` instead of `tissue_general` for finer granularity - Filter by specific `dataset_id` if known - Switch to out-of-core processing for large queries

### Memory Errors - Reduce query scope with more restrictive filters - Select fewer genes with `var_value_filter` - Use out-of-core processing with `axis_query()` - Process data in batches

### Duplicate Cells in Results - Always include `is_primary_data == True` in filters - Check if intentionally querying across multiple datasets

### Gene Not Found - Verify gene name spelling (case-sensitive) - Try Ensembl ID with `feature_id` instead of `feature_name` - Check dataset presence matrix to see if gene was measured - Some genes may have been filtered during Census construction

### Version Inconsistencies - Always specify `census_version` explicitly - Use same version across all analyses - Check release notes for version-specific changes