This repository contains the PyTorch implementation of im_net, a neuron/layer type (IM_Layer) that optimizes local objective functions built from Partial Information Decomposition (PID) instead of end-to-end gradient descent. It hosts the code for two papers, each in its own subfolder with its own README:
paper_trivariate/— "What Should a Neuron Aim For? Designing Local Objective Functions Based on Information Theory", feedforward classifiers trained with trivariate PID goals.paper_hopfield/— Redundancy Maximization as a Principle of Associative Memory Learning in Hopfield Networks Infomorphic Hopfield Networks, associative-memory networks whose recurrent update rule is trained with local PID goals.
This repository was developed by Valentin Neuhaus, Andreas C. Schneider, David A. Ehrlich and Mark Blümel, and is actively maintained by the authors.
Developed and tested mostly on Linux. No Windows nor macOS-specific testing has been performed.
All Python dependencies are listed in env.yml and installed via conda. Key dependencies include PyTorch, Hydra (1.2), Optuna (2.10), and scikit-learn — see env.yml for the complete list.
No non-standard hardware is required. An NVIDIA GPU is optional and only used to accelerate training; training also runs on CPU via params.pref_gpu=False (or exp_params.pref_gpu=False in paper_trivariate).
git clone <repository-url>
cd infomorph_networksim_net can optionally compute PID atoms with two additional third-party estimators (dit, BROJA_2PID) for comparison, vendored as git submodules under external/. They are not required for training or reproducing the papers' main results. To add them:
git submodule update --init --recursiveconda env create -f env.yml
conda activate infomorphic_envAround 10 minutes on a standard desktop (excluding submodule/package download time, which depends on network speed).
All commands must be run from the repository root, since im_net loads moebius.pkl (provided at the repository root) using a path relative to the current working directory.
Both papers configure and override runs the same way, via Hydra: parameters live in each paper's conf/ folder as YAML, and any value can be overridden on the command line, e.g. to change the number of training epochs. See each paper's own README for its entry points, available models, and example commands:
All results (performance, PID atom sizes, and the parameters used) are saved to an experiments folder, in a subfolder named after the run's timestamp.
This project is licensed under the BSD 3-Clause License — see LICENSE for details.
The corresponding author for the code is valentin.neuhaus@ds.mpg.de.
For questions, also reach out to us at mark.bluemel@ds.mpg.de, andreas.schneider@ds.mpg.de.