Quickstart¶

Get started with the podcast benchmark framework in minutes.

Setup¶

To download data and set up your local virtual environment:

./setup.sh

This will: - Create a Python virtual environment (conda or venv) - Install all required dependencies - Download the necessary podcast listening data

Setup options:

./setup.sh --gpu           # Install GPU dependencies (CUDA packages)
./setup.sh --dev           # Install dev dependencies (testing), skip data download
./setup.sh --env-name NAME # Custom environment name (default: decoding_env)

Training Your First Model¶

The framework comes with several pre-configured models you can train immediately.

1. Neural Convolutional Decoder¶

This recreates the decoder from Tang et al. 2022, which decodes word embeddings directly from neural data:

make neural-conv

2. Foundation Model Decoder¶

This trains a decoder from a foundation model's latent representations to word embeddings:

make foundation-model

3. POPT Foundation Model¶

Evaluate the POPT foundation model on word embedding decoding:

python main.py --config configs/foundation_models/popt_word_embedding.yml

Results¶

Training results will be saved to: - results/ - Performance metrics and CSV files - checkpoints/ - Saved model checkpoints - event_logs/ - TensorBoard logs

See Baseline Results for performance benchmarks across all tasks.

Configuration¶

To modify data, behavior, or hyperparameters:

Edit the relevant configuration file in configs/: - configs/neural_conv_decoder/ - Neural convolutional decoder settings - configs/example_foundation_model/ - Foundation model decoder settings - configs/foundation_models/ - POPT and other foundation model configs - configs/controls/ - Control experiments (e.g., no brain data baselines)

Model implementations can be found in the models/ directory.

See Onboarding a New Model for details on configuration options.