Meta FAIR Releases NeuralSet: A Python Package for Neuro-AI That Supports fMRI, M/EEG, Spikes, and HuggingFace Embeddings

Researchers at Meta's FAIR lab have released NeuralSet, a Python framework designed to eliminate one of the most persistent bottlenecks in Neuro-AI research: the painful, fragmented process of getting brain data into a deep learning pipeline. Neuroscience already has excellent, battle-tested software. Tools like MNE-Python, EEGLAB, FieldTrip, Brainstorm, Nilearn, and fMRIPrep are the gold standard for signal processing across electrophysiology and neuroimaging.

However, these tools were designed for a pre-deep-learning world, relying on eager loading, assuming entire datasets fit into RAM, and lacking native abstractions to temporally align neural time series with high-dimensional embeddings from modern AI frameworks like HuggingFace Transformers. The result is that researchers spend enormous effort building ad-hoc pipelines that require manual data wrangling, manual caching, and complex backend configurations — just to get brain signals paired with, say, GPT-2 text embeddings for a single experiment. As public datasets on platforms like OpenNeuro now reach the terabyte scale, and experimental protocols increasingly incorporate continuous speech and video stimuli, this infrastructure gap is no longer just inconvenient — it is a scientific bottleneck.

NeuralSet's core design principle is structure–data decoupling. Instead of loading raw signals upfront, NeuralSet represents the logical structure of any experiment as lightweight, event-driven metadata — completely separate from the memory- and compute-intensive extraction of actual signals. The framework is organized around five core abstractions: Events, Extractors, Segments, Batch Data, and a Backend layer.

In practice, everything in an experiment — an fMRI run, a word spoken during a task, a video stimulus — is modeled as an Event: a lightweight Python dictionary defined by a type, a start time, a duration, and a timeline (a unique identifier for a continuous recording session). A Study object assembles all events in an entire dataset into a single pandas DataFrame. Importantly, NeuralSet supports BIDS-compliant datasets, though it is not restricted to them.

Because the DataFrame contains only lightweight metadata — not the raw signals themselves — engineers can filter, explore, and recombine massive datasets using standard pandas operations without loading a single byte of raw data into memory. NeuralSet is built on the exca package, which handles deterministic, hash-based caching, full computational provenance, and hardware-agnostic execution. The research team presents a detailed comparison of NeuralSet against 18 existing neuroscience software packages across neural devices, experimental task types, and infrastructure features.

NeuralSet is the only package in the comparison that achieves full support across all categories. Check out the Paper and GitHub Page.