Federated Analytics

This page describes the architecture of the Federated Analytics (FA) feature, the responsibility of each component, and which files to touch when making changes.

How It Works

Researcher                         Node
──────────                         ────
FederatedAnalytics.fetch_stats()
  └─ FARequestJob.execute()  ──── FARequest ──►  node.py
                                                   └─ FAJob.run()
                                                        ├─ dataset.compute_stats()
                                                        │    └─ AnalyticsOrchestrator
                                                        │         └─ Accumulators
                                                        └─ FAReply 
  └─ FAResult (aggregate)  ◄────────────────────────────────┘

1. The researcher calls a stat method (e.g. exp.analytics.mean()).
2. FederatedAnalytics checks its cache; missing stats are sent as an FARequest to each node via FARequestJob.
3. On the node, FAJob validates the request and calls dataset.compute_stats().
4. AnalyticsOrchestrator reads analytics_schema() from the dataset, builds an accumulator tree, iterates samples, and returns partial statistics.
5. The node sends back an FAReply; the researcher side merges node replies into an FAResult and aggregates globally via AGGREGATORS_MAP.

Statistics, API and Aggregation

Available Statistics

Stats (in fedbiomed/common/constants.py) is the single source of truth for stat names. Every node accumulator, registry entry, and aggregator function must reference one of these values. Adding a stat starts here.

FederatedAnalytics is the entry point for all analytics requests. It handles caching and delegates network I/O to FARequestJob. The API is split into two methods depending on whether the requested statistic needs extra parameters.

Use fetch_stats when no computation arguments are needed — statistics are identified by name, and an optional dataset_schema restricts which columns or modalities are included:

fetch_stats(
    stats: str | list[str] | None = None,     # one or more Stats string values; None → ["count", "mean", "variance"]
    dataset_schema: str | list | None = None, # column/modality filter; None → whole schema
) -> FAResult

Use fetch_stats_with_args when the statistic requires parameters (e.g. histogram bin edges). Because the parameters are specific to individual columns or modalities, schema selection is encoded directly inside stats_args — there is no separate dataset_schema:

fetch_stats_with_args(
    stats_args: dict,   # encodes both schema selection and computation parameters
) -> FAResult

Both methods cache their results. fetch_stats keys the cache by (node_ids, dataset_schema); fetch_stats_with_args keys it by (node_ids, stats_args). In both cases, an identical call is served from the cache without a network round-trip.

Convenience methods are thin wrappers around fetch_stats + global_stats for the most common stats:

exp.analytics.mean(dataset_schema=None)
# → {'year': 2016.96, 'price': 16235.20, 'mileage': 23908.94, 'mpg': 55.25, ...}

exp.analytics.mean(dataset_schema=["price", "mileage"])
# → {'price': 16235.20, 'mileage': 23908.94}

Stat Dependencies

The orchestrator resolves dependencies automatically before building the accumulator tree. This means requesting variance will also compute mean and count on the node even if they are not listed explicitly in stats. Dependencies and required arguments for each stat are declared in fedbiomed/common/analytics/accumulators/_registry.py.

Cross-Node Aggregation

Once all nodes have replied, FAResult calls AGGREGATORS_MAP to combine per-node partial results into a single global value per modality or column. Each function is registered via the @aggregator(stat) decorator; its parameter names match Stats string values.

Component Responsibilities

Common layer

Node layer

Researcher layer

Adding a New Statistic

1. Add the name to Stats in fedbiomed/common/constants.py.
2. Implement the accumulator in fedbiomed/common/analytics/accumulators/_operations.py.
3. Register it in fedbiomed/common/analytics/accumulators/_registry.py (element type, dependency, required args).
4. Add the cross-node aggregator to AGGREGATORS_MAP in fedbiomed/common/analytics/_aggregators.py.
5. (Optional) Add a convenience method in FederatedAnalytics (fedbiomed/researcher/federated_workflows/_federated_analytics.py).
6. Add tests in tests/test_analytics/.

Test Coverage

Related Documentation

• User guide: Federated Analytics
• Notebook tutorial: notebooks/federated_analytics.ipynb
• API reference: Common Analytics · Node Jobs · Researcher Federated Workflows