bio_transformations package

Overview

The bio_transformations package implements biologically inspired modifications to artificial neural networks. It enhances the learning capabilities of neural networks by mimicking the plasticity and stability characteristics observed in biological synapses.

Key Components

  1. BioConverter

    The main interface for converting standard PyTorch modules to bio-inspired versions.

  2. BioModule

    The core class implementing biologically inspired modifications.

Submodules

bio_transformations.bio_converter module

The BioConverter class is responsible for converting standard PyTorch modules to BioNet modules with bio-inspired modifications. It provides methods for:

  • Converting entire model architectures

  • Configuring bio-inspired parameters

  • Applying weight splitting techniques

bio_transformations.bio_module module

The BioModule class implements the core biologically inspired modifications. It includes the following key functions:

Synaptic Plasticity Functions

  • volume_dependent_lr(): Implements learning rates based on weight magnitude.

  • fuzzy_learning_rates(): Applies diverse learning rates to individual synapses.

  • crystallize(): Simulates synaptic stabilization over time.

Structural Plasticity Functions

  • rejuvenate_weights(): Mimics spine turnover in biological networks.

Homeostatic Plasticity Functions

  • scale_grad(): Implements synaptic scaling for overall network stability.

Additional Functions

  • l1_reg(): Computes L1 regularization of module parameters.

  • enforce_dales_principle(): Enforces Dale’s principle on module weights.

  • dalian_network_initialization(): Initializes network weights according to Dale’s principle.

For detailed information on each function, please refer to the individual function documentation.

Extending BioModule

To add a new biologically motivated function:

  1. Implement the new function in the BioModule class.

  2. Add the function name to the exposed_functions list.

  3. Update __init__ methods if new parameters are required.

  4. Create a test case in test_biomodule.py.

  5. Update this documentation to include details about the new function.

Module contents

For a practical guide on using these components, please refer to the Tutorials section.