Background

Living animals are one of the primary sources of data for the development of artificial neural sys- tems. In general, animals are able to perform all of their behaviors and actions under the guidance of a single neural network and its underlying algorithms. One of the end goals for artificial intelligence systems is to emulate these complex neural networks in a virtual manner, to allow our creations to act in a way that replicates how a real animal would behave and learn. Embodied control in neuro- science holds the potential to deepen our understanding of how the real brain works. If a sufficient model of the brain can be created in conjunction with the behaviors that a biorealistically simulated agent generates, then we can quickly generate new hypotheses about real world mappings between the brain and behavior. To achieve this researchers within this field are working to decode all the parts of animals’ neural networks through inferential and replicatory techniques. One of the leading method to do so is to build models to learn how to behave like a real animal, mainly through deep learning models that emulate imitation learning or reinforcement learning. Prior research within this intersection of neuroscience and machine learning have explored methods to solve tasks given input data from a sensory channel, such as vision or sound. The particular approach taken in “Deep Neuroethology of a Virtual Rodent”, along with this replicatory work, focuses on understanding the underlying mechanisms of “embodied control” (Merel et al., 2019), which is how animals utilize a ”combination of their body, sensory input, and behavior” to accomplish tasks, with a particular focus on motor control. The implications of this study include furthering developments in understanding the underlying mechanisms of animal behavior within the fields of motor neuroscience and artificial intelligence.

Walker Agent Visualization

Online Model:

LHS is the walker walking, RHS is the walker's states and rewards

The online model seems to have more unstableness compared

Behavioral Cloning (BC) Model

LHS is the walker walking, RHS is the walker's states and rewards

Side-by-side Comparison

BC Agents is more robust as the kinematic data contain less noise.

TSNE Clustering of Kinematics

The TSNE Cluster is loose, indicating inconsistancy of the kinematics

The TSNE Cluster is dense, indicating the smoothness of the kinematics

CKA Analysis

The CKA analysis suggests the representations between the online reinforcement and behavior cloning (BC) models are dissimilar. The addition of noise helps the BC model learn slightly more similar representations.