About webmatr0n

This author has not yet filled in any details.
So far webmatr0n has created 81 blog entries.

Dr. Wan Chen Lin

Wan Chen Lin received her Ph.D. in behavioral and systems neuroscience. Congratulations Dr. Lin!

Dr. Wan Chen Lin2021-06-02T16:26:54+00:00

Learning Rates Are Not All the Same: The Interpretation of Computational Model Parameters Depends on the Context

Reinforcement Learning (RL) has revolutionized the cognitive and brain sciences, explaining behavior from simple conditioning to problem solving, across the life span, and anchored in brain function. However, discrepancies in results are increasingly apparent between studies, particularly in the developmental literature. To better understand these, we investigated to which extent parameters generalize between tasks and models, and capture specific and uniquely interpretable (neuro)cognitive processes. 291 participants aged 8-30 years completed three learning tasks in a single session, and were fitted using state-of-the-art RL models. RL decision noise/exploration parameters generalized well between tasks, decreasing between ages 8-17. Learning rates for negative feedback did not generalize, and learning rates for positive feedback showed intermediate generalizability, dependent on task similarity. These findings can explain discrepancies in the existing literature. Future research therefore needs to carefully consider task characteristics when relating findings across studies, and develop strategies to computationally model how context impacts behavior.

Maria K Eckstein, Sarah L Master, Liyu Xia, Ronald E Dahl, Linda Wilbrecht, Anne Gabrielle Eva Collins, Learning Rates Are Not All the Same: The Interpretation of Computational Model Parameters Depends on the Context (May 2021), https://www.biorxiv.org/content/10.1101/2021.05.28.446162v1
doi: https://doi.org/10.1101/2021.05.28.446162

Learning Rates Are Not All the Same: The Interpretation of Computational Model Parameters Depends on the Context2022-06-18T20:51:56+00:00

What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience

Reinforcement learning (RL) is a concept that has been invaluable to research fields including machine learning, neuroscience, and cognitive science. However, what RL entails partly differs between fields, leading to difficulties when interpreting and translating findings.

This paper lays out these differences and zooms in on cognitive (neuro)science, revealing that we often overinterpret RL modeling results, with severe consequences for future research. Specifically, researchers often assume—implicitly—that model parameters generalize between tasks, models, and participant populations, despite overwhelming negative empirical evidence for this assumption. We also often assume that parameters measure specific, unique, and meaningful (neuro)cognitive processes, a concept we call interpretability, for which empirical evidence is also lacking.

We conclude that future computational research needs to pay increased attention to these implicit assumptions when using RL models, and suggest an alternative framework that resolves these issues and allows us to unleash the potential of RL in cognitive (neuro)science.

Maria Eckstein, Linda Wilbrecht, Anne Collins, What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience (May 2021), https://psyarxiv.com/e7kwx/

 

What do Reinforcement Learning Models Measure? Interpreting Model Parameters in Cognition and Neuroscience2022-06-18T20:52:15+00:00

The Unique Advantage of Adolescents in Probabilistic Reversal: Reinforcement Learning and Bayesian Inference Provide Adequate and Complementary Models

During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated whether adolescents are uniquely adapted to this transition, compared to younger children and adults. In a stochastic, volatile reversal learning task with a sample of 291 participants aged 8-30, we found that adolescents 13-15 years old outperformed both younger and older participants. We developed two independent cognitive models, one based on Reinforcement learning (RL) and the other Bayesian inference (BI), and used hierarchical Bayesian model fitting to assess developmental changes in underlying cognitive mechanisms. Choice parameters in both models improved monotonously. By contrast, RL update parameters and BI mental-model parameters peaked closest to optimal values in 13-to-15-year-olds. Combining both models using principal component analysis yielded new insights, revealing that three readily-interpretable components contributed to the early-to mid-adolescent performance peak. This research highlights early-to mid-adolescence as a neurodevelopmental window that may be more optimal for behavioral adjustment in volatile and uncertain environments. It also shows how increasingly detailed insights can be gleaned by invoking different cognitive models.

Maria K. Eckstein, Sarah L. Master, Ronald E. Dahl, Linda Wilbrecht, Anne G.E. Collins, The Unique Advantage of Adolescents in Probabilistic Reversal: Reinforcement Learning and Bayesian Inference Provide Adequate and Complementary Models (Mar. 2021) https://www.biorxiv.org/content/10.1101/2020.07.04.187971v2.full
doi: https://doi.org/10.1101/2020.07.04.187971

The Unique Advantage of Adolescents in Probabilistic Reversal: Reinforcement Learning and Bayesian Inference Provide Adequate and Complementary Models2022-06-18T20:52:28+00:00

A role for adaptive developmental plasticity in learning and decision making

From both a medical and educational perspective, there is enormous value to understanding the environmental factors that sculpt learning and decision making. These questions are often approached from proximate levels of analysis, but may be further informed by the adaptive developmental plasticity framework used in evolutionary biology. The basic adaptive developmental plasticity framework posits that biological sensitive periods evolved to use information from the environment to sculpt emerging phenotypes. Here, we lay out how we can apply this framework to learning and decision making in the mammalian brain and propose a working model in which dopamine neurons and their activity may serve to inform downstream circuits about environmental statistics. More widespread use of this evolutionary framework and its associated models can help inform and guide basic research and intervention science.

Wan Chen Lin, Kristen Delevich, Linda Wilbrecht, A role for adaptive developmental plasticity in learning and decision making, 36 Current Opinion in Behavioral Sciences pp 48–54 (2020), https://doi.org/10.1016/j.cobeha.2020.07.010, http://www.sciencedirect.com/science/article/pii/S2352154620301121

A role for adaptive developmental plasticity in learning and decision making2021-06-02T16:28:40+00:00

Choice suppression is achieved through opponent but not independent function of the striatal indirect pathway in mice

The dorsomedial striatum (DMS) plays a key role in action selection, but little is known about how direct and indirect pathway spiny projection neurons (dSPNs and iSPNs) contribute to choice suppression in freely moving animals. Here, we used pathway-specific chemogenetic manipulation during a serial choice foraging task to test opposing predictions for iSPN function generated by two theories: 1) the ‘select/suppress’ heuristic which suggests iSPN activity is required to suppress alternate choices and 2) the network-inspired Opponent Actor Learning model (OpAL) which proposes that the weighted difference of dSPN and iSPN activity determines choice. We found that chemogenetic activation, but not inhibition, of iSPNs disrupted learned suppression of nonrewarded choices, consistent with the predictions of the OpAL model. Our findings suggest that iSPNs’ role in stopping and freezing does not extend in a simple fashion to choice suppression. These data may provide insights critical for the successful design of interventions for addiction or other conditions in which suppression of behavior is desirable.

Kristen Delevich, Benjamin Hoshal, Anne GE Collins, Linda Wilbrecht, Choice suppression is achieved through opponent but not independent function of the striatal indirect pathway in mice, BioRxiv, https://www.biorxiv.org/content/10.1101/675850v3
doi: https://doi.org/10.1101/675850

 

Choice suppression is achieved through opponent but not independent function of the striatal indirect pathway in mice2022-06-18T20:52:41+00:00

Sex and Pubertal Status Influence Dendritic Spine Density on Frontal Corticostriatal Projection Neurons in Mice

In humans, nonhuman primates, and rodents, the frontal cortices exhibit grey matter thinning and dendritic spine pruning that extends into adolescence. This maturation is believed to support higher cognition but may also confer psychiatric vulnerability during adolescence. Currently, little is known about how specific cell types in the frontal cortex mature or whether puberty plays a role in the maturation of some cell types but not others. Here, we used mice to characterize the spatial topography and adolescent development of cross-corticostriatal (cSTR) neurons that project through the corpus collosum to the dorsomedial striatum. We found that apical spine density on cSTR neurons in the medial prefrontal cortex decreased significantly between late juvenile (P29) and young adult time points (P60), with females exhibiting higher spine density than males at both ages. Adult males castrated prior to puberty onset had higher spine density compared to sham controls. Adult females ovariectomized before puberty onset showed greater variance in spine density measures on cSTR cells compared to controls, but their mean spine density did not significantly differ from sham controls. Our findings reveal that these cSTR neurons, a subtype of the broader class of intratelencephalic-type neurons, exhibit significant sex differences and suggest that spine pruning on cSTR neurons is regulated by puberty in male mice.

Kristen Delevich, Nana J Okada, Ameet Rahane, Zicheng Zhang, Christopher D Hall, Linda Wilbrecht, Sex and Pubertal Status Influence Dendritic Spine Density on Frontal Corticostriatal Projection Neurons in MiceCerebral Cortex, , bhz325, https://doi.org/10.1093/cercor/bhz325 (preprint available at https://www.biorxiv.org/content/biorxiv/early/2019/09/30/787408.full.pdf)

Sex and Pubertal Status Influence Dendritic Spine Density on Frontal Corticostriatal Projection Neurons in Mice2020-02-13T03:35:21+00:00

Distentangling the systems contributing to changes in learning during adolescence

Multiple neurocognitive systems contribute simultaneously to learning. For example, dopamine and basal ganglia (BG) systems are thought to support reinforcement learning (RL) by incrementally updating the value of choices, while the prefrontal cortex (PFC) contributes different computations, such as actively maintaining precise information in working memory (WM). It is commonly thought that WM and PFC show more protracted development than RL and BG systems, yet their contributions are rarely assessed in tandem. Here, we used a simple learning task to test how RL and WM contribute to changes in learning across adolescence. We tested 187 subjects ages 8 to 17 and 53 adults (25-30). Participants learned stimulus-action associations from feedback; the learning load was varied to be within or exceed WM capacity. Participants age 8-12 learned slower than participants age 13-17, and were more sensitive to load. We used computational modeling to estimate subjects’ use of WM and RL processes. Surprisingly, we found more protracted changes in RL than WM during development. RL learning rate increased with age until age 18 and WM parameters showed more subtle, gender- and puberty-dependent changes early in adolescence. These results can inform education and intervention strategies based on the developmental science of learning.

Sarah L. Master, Maria K. Eckstein, Neta Gotlieb, Ronald Dahl, Linda Wilbrecht, Anne G.E. Collins, Distentangling the systems contributing to changes in learning during adolescence, Developmental Cognitive Neuroscience (2019),
https://doi.org/10.1016/j.dcn.2019.100732, http://www.sciencedirect.com/science/article/pii/S1878929319303196

Distentangling the systems contributing to changes in learning during adolescence2019-11-20T23:08:54+00:00

Variation in early life maternal care predicts later long range frontal cortex synapse development in mice

Empirical and theoretical work suggests that early postnatal experience may inform later developing synaptic connectivity to adapt the brain to its environment. We hypothesized that early maternal experience may program the development of synaptic density on long range frontal cortex projections. To test this idea, we used maternal separation (MS) to generate environmental variability and examined how MS affected 1) maternal care and 2) synapse density on virally-labeled long range axons of offspring reared in MS or control conditions. We found that MS and variation in maternal care predicted bouton density on dorsal frontal cortex axons that terminated in the basolateral amygdala (BLA) and dorsomedial striatum (DMS) with more, fragmented care associated with higher density. The effects of maternal care on these distinct axonal projections of the frontal cortex were manifest at different ages. Maternal care measures were correlated with frontal cortex → BLA bouton density at mid-adolescence postnatal (P) day 35 and frontal cortex → DMS bouton density in adulthood (P85). Meanwhile, we found no evidence that MS or maternal care affected bouton density on ascending orbitofrontal cortex (OFC) or BLA axons that terminated in the dorsal frontal cortices. Our data show that variation in early experience can alter development in a circuit-specific and age-dependent manner that may be relevant to early life adversity.

A. Wren Thomas, Kristen Delevich, Irene Chang, Linda Wilbrecht, Variation in early life maternal care predicts later long range frontal cortex synapse development in mice, Developmental Cognitive Neuroscience (2009)
https://doi.org/10.1016/j.dcn.2019.100737, http://www.sciencedirect.com/science/article/pii/S187892931930324X

Variation in early life maternal care predicts later long range frontal cortex synapse development in mice2019-11-20T23:02:33+00:00