Hidden state inference requires abstract contextual representations in ventral hippocampus

The ability to form and utilize subjective, latent contextual representations to influence decision making is a crucial determinant of everyday life. The hippocampus is widely hypothesized to bind together otherwise abstract combinations of stimuli to represent such latent contexts, and to allow their use to support the process of hidden state inference. Yet, direct evidence for this remains limited. Here we show that the CA1 area of the ventral hippocampus is necessary for mice to perform hidden state inference during a 2-armed bandit task. vCA1 neurons robustly differentiate between the two abstract contexts required for this strategy in a manner similar to the differentiation of spatial locations, despite the contexts being formed only from past probabilistic outcomes. These findings offer insight into how latent contextual information is used to optimize decision-making processes, and emphasize a key role of the hippocampus in hidden state inference.

Karyna Mishchanchuk, Gabrielle Gregoriou, Albert Jiaxu Qu, Alizee Kastler, Quentin Huys, Linda Wilbrecht, Andrew F MacAskill, Hidden state inference requires abstract contextual representations in ventral hippocampus, bioRxiv 2024.05.17.594673; doi: https://doi.org/10.1101/2024.05.17.594673 
Hidden state inference requires abstract contextual representations in ventral hippocampus2024-05-20T09:13:59+00:00

Adolescent and adult mice use both incremental reinforcement learning and short term memory when learning concurrent stimulus-action associations

Computational modeling has revealed that human research participants use both rapid working memory (WM) and incremental reinforcement learning (RL) (RL+WM) to solve a simple instrumental learning task, relying on WM when the number of stimuli is small and supplementing with RL when the number of stimuli exceeds WM capacity. Inspired by this work, we examined which learning systems and strategies are used by adolescent and adult mice when they first acquire a conditional associative learning task. In a version of the human RL+WM task translated for rodents, mice were required to associate odor stimuli (from a set of 2 or 4 odors) with a left or right port to receive reward. Using logistic regression and computational models to analyze the first 200 trials per odor, we determined that mice used both incremental RL and stimulus-insensitive, one-back strategies to solve the task. While these one-back strategies may be a simple form of short-term or working memory, they did not approximate the boost to learning performance that has been observed in human participants using WM in a comparable task. Adolescent and adult mice also showed comparable performance, with no change in learning rate or softmax beta parameters with adolescent development and task experience. However, reliance on a one-back perseverative, win-stay strategy increased with development in males in both odor set sizes. Our findings advance a simple conditional associative learning task and new models to enable the isolation and quantification of reinforcement learning alongside other strategies mice use while learning to associate stimuli with rewards within a single behavioral session. These data and methods can inform and aid comparative study of reinforcement learning across species.

Juliana B. Chase, Liyu Xia, Lung-Hao Tai, Wan Chen Lin, Anne G.E. Collins, Linda Wilbrecht, Adolescent and adult mice use both incremental reinforcement learning and short term memory when learning concurrent stimulus-action associations, bioRxiv 2024.04.29.591768; doi: https://doi.org/10.1101/2024.04.29.591768

Adolescent and adult mice use both incremental reinforcement learning and short term memory when learning concurrent stimulus-action associations2024-05-20T09:10:20+00:00

Goal-directed learning in adolescence: neurocognitive development and contextual influences

Adolescence is a time during which we transition to independence, explore new activities and begin pursuit of major life goals. Goal-directed learning, in which we learn to perform actions that enable us to obtain desired outcomes, is central to many of these processes. Currently, our understanding of goal-directed learning in adolescence is itself in a state of transition, with the scientific community grappling with inconsistent results. When we examine metrics of goal-directed learning through the second decade of life, we find that many studies agree there are steady gains in performance in the teenage years, but others report that adolescent goal-directed learning is already adult-like, and some find adolescents can outperform adults. To explain the current variability in results, sophisticated experimental designs are being applied to test learning in different contexts. There is also increasing recognition that individuals of different ages and in different states will draw on different neurocognitive systems to support goal-directed learning. Through adoption of more nuanced approaches, we can be better prepared to recognize and harness adolescent strengths and to decipher the purpose (or goals) of adolescence itself.

Wilbrecht, L., Davidow, J.Y. Goal-directed learning in adolescence: neurocognitive development and contextual influences. Nat. Rev. Neurosci. (2024). https://doi.org/10.1038/s41583-023-00783-w

Goal-directed learning in adolescence: neurocognitive development and contextual influences2024-01-25T12:46:38+00:00

The interpretation of computational model parameters depends on the context

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning to complex problem solving, to shed light on developmental and individual differences, and to anchor cognitive processes in specific brain mechanisms. However, the RL literature increasingly reveals contradictory results, which might cast doubt on these claims. We hypothesized that many contradictions arise from two commonly-held assumptions about computational model parameters that are actually often invalid: That parameters generalize between contexts (e.g. tasks, models) and that they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8–30 years to complete three learning tasks in one experimental session, and fitted RL models to each. We found that some parameters (exploration / decision noise) showed significant generalization: they followed similar developmental trajectories, and were reciprocally predictive between tasks. Still, generalization was significantly below the methodological ceiling. Furthermore, other parameters (learning rates, forgetting) did not show evidence of generalization, and sometimes even opposite developmental trajectories. Interpretability was low for all parameters. We conclude that the systematic study of context factors (e.g. reward stochasticity; task volatility) will be necessary to enhance the generalizability and interpretability of computational cognitive models.

Maria Katharina Eckstein, Sarah L Master, Liyu Xia, Ronald E Dahl, Linda Wilbrecht, Anne GE Collins, The interpretation of computational model parameters depends on the context, eLife (2022) 11:e75474. https://doi.org/10.7554/eLife.75474

The interpretation of computational model parameters depends on the context2022-11-07T12:53:57+00:00

The maturation of exploratory behavior in adolescent Mus spicilegus on two photoperiods

Dispersal from the natal site or familial group is a core milestone of adolescent development in many species. A wild species of mouse, Mus spicilegus, presents an exciting model in which to study adolescent development and dispersal because it shows different life history trajectory depending on season of birth. M. spicilegus born in spring and summer on long days (LD) disperse in the first 3 months of life, while M. spicilegus born on shorter autumnal days (SD) delay dispersal through the wintertime. We were interested in using these mice in a laboratory context to compare age-matched mice with differential motivation to disperse. To first test if we could find a proxy for dispersal related behavior in the laboratory environment, we measured open field and novel object investigation across development in M. spicilegus raised on a LD 12 h:12 h light:dark cycle. We found that between the first and second month of life, distance traveled and time in center of the open field increased significantly with age in M. spicilegus. Robust novel object investigation was observed in all age groups and decreased between the 2nd and 3rd month of life in LD males. Compared to male C57BL/6 mice, male M. spicilegus traveled significantly longer distances in the open field but spent less time in the center of the field. However, when a novel object was placed in the center of the open field, Male M. spicilegus, were significantly more willing to contact and mount it. To test if autumnal photoperiod affects exploratory behavior in M. spicilegus in a laboratory environment, we reared a cohort of M. spicilegus on a SD 10 h:14 h photoperiod and tested their exploratory behavior at P60-70. At this timepoint, we found SD rearing had no effect on open field metrics, but led to reduced novel object investigation. We also observed that in P60-70 males, SD reared M. spicilegus weighed less than LD reared M. spicilegus. These observations establish that SD photoperiod can delay weight gain and blunt some, but not all forms of exploratory behavior in adolescent M. spicilegus.

Noah G. Cryns, Wan Chen Lin, Niloofar Motahari, Oliver J. Krentzman, Weihang Chen, George Prounis, Linda Wilbrecht, The maturation of exploratory behavior in adolescent Mus spicilegus on two photoperiods, Front. Behav. Neurosci., Nov. 4 2022, Sec. Individual and Social Behaviors
https://doi.org/10.3389/fnbeh.2022.988033

The maturation of exploratory behavior in adolescent Mus spicilegus on two photoperiods2022-11-07T12:47:22+00:00

Activation, but not inhibition, of the indirect pathway disrupts choice rejection in a freely moving, multiple-choice foraging task

The dorsomedial striatum (DMS) plays a key role in action selection, but less is known about how direct and indirect pathway spiny projection neurons (dSPNs and iSPNs, respectively) contribute to choice rejection in freely moving animals. Here, we use pathway-specific chemogenetic manipulation during a serial choice foraging task to test the role of dSPNs and iSPNs in learned choice rejection. We find that chemogenetic activation, but not inhibition, of iSPNs disrupts rejection of nonrewarded choices, contrary to predictions of a simple “select/suppress” heuristic. Our findings suggest that iSPNs’ role in stopping and freezing does not extend in a simple fashion to choice rejection in an ethological, freely moving context. These data may provide insights critical for the successful design of interventions for addiction or other conditions in which it is desirable to strengthen choice rejection.

Kristen Delevich,  Benjamin Hoshal, Lexi Z. Zhou, Yuting Zhang, Satya Vedula, Wan Chen Lin, Juliana Chase, Anne G.E. Collins, Linda Wilbrecht, Activation, but not inhibition, of the indirect pathway disrupts choice rejection in a freely moving, multiple-choice foraging task, 40(4) Cell Reports 111129 (July 26, 2022). DOI: https://doi.org/10.1016/j.celrep.2022.111129

Activation, but not inhibition, of the indirect pathway disrupts choice rejection in a freely moving, multiple-choice foraging task2022-07-29T03:19:24+00:00

Transient food insecurity during the juvenile-adolescent period affects adult weight, cognitive flexibility, and dopamine neurobiology

A major challenge for neuroscience, public health, and evolutionary biology is to understand the effects of scarcity and uncertainty on the developing brain. Currently, a significant fraction of children and adolescents worldwide experience insecure access to food. The goal of our work was to test in mice whether the transient experience of insecure versus secure access to food during the juvenile-adolescent period produced lasting differences in learning, decision-making, and the dopamine system in adulthood. We manipulated feeding schedules in mice from postnatal day (P)21 to P40 as food insecure or ad libitum and found that when tested in adulthood (after P60), males with different developmental feeding history showed significant differences in multiple metrics of cognitive flexibility in learning and decision-making. Adult females with different developmental feeding history showed no differences in cognitive flexibility but did show significant differences in adult weight. We next applied reinforcement learning models to these behavioral data. The best fit models suggested that in males, developmental feeding history altered how mice updated their behavior after negative outcomes. This effect was sensitive to task context and reward contingencies. Consistent with these results, in males, we found that the two feeding history groups showed significant differences in the AMPAR/NMDAR ratio of excitatory synapses on nucleus-accumbens-projecting midbrain dopamine neurons and evoked dopamine release in dorsal striatal targets. Together, these data show in a rodent model that transient differences in feeding history in the juvenile-adolescent period can have significant impacts on adult weight, learning, decision-making, and dopamine neurobiology.

Wan Chen Lin, Christine Liu, Polina Kosillo, Lung-Hao Tai, Ezequiel Galarce, Helen S. Bateup, Stephan Lammel, Linda Wilbrecht,
Transient food insecurity during the juvenile-adolescent period affects adult weight, cognitive flexibility, and dopamine neurobiology,
Current Biology, ISSN 0960-9822, (2022) https://doi.org/10.1016/j.cub.2022.06.089 https://www.sciencedirect.com/science/article/pii/S0960982222010946

Transient food insecurity during the juvenile-adolescent period affects adult weight, cognitive flexibility, and dopamine neurobiology2022-07-21T23:12:30+00:00

Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal

During adolescence, youth venture out, explore the wider world, and are challenged to learn how to navigate novel and uncertain environments. We investigated how performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes the balance of persistence and flexibility. In a sample of 291 participants aged 8–30, we found that in the mid-teen years, adolescents outperformed both younger and older participants. We developed two independent cognitive models, based on Reinforcement learning (RL) and Bayesian inference (BI). The RL parameter for learning from negative outcomes and the BI parameters specifying participants’ mental models were closest to optimal in mid-teen adolescents, suggesting a central role in adolescent cognitive processing. By contrast, persistence and noise parameters improved monotonically with age. We distilled the insights of RL and BI using principal component analysis and found that three shared components interacted to form the adolescent performance peak: adult-like behavioral quality, child-like time scales, and developmentally-unique processing of positive feedback. This research highlights adolescence as a neurodevelopmental window that can create performance advantages in volatile and uncertain environments. It also shows how detailed insights can be gleaned by using cognitive models in new ways.

Maria K. Eckstein, Sarah L. Master, Ronald E. Dahl, Linda Wilbrecht, Anne G.E. Collins, Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal, Developmental Cognitive Neuroscience, Volume 55, 2022, 101106, ISSN 1878-9293, https://doi.org/10.1016/j.dcn.2022.101106, https://www.sciencedirect.com/science/article/pii/S1878929322000494

Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal2022-06-18T20:50:11+00:00

Making sense of strengths and weaknesses observed in adolescent laboratory rodents

During adolescence, rodents disperse from their natal site, find a new home, and navigate social relationships and threats. Although rats and mice in the laboratory cannot fully express these natural behaviors, they show striking changes in their affective and cognitive behavior across the adolescent period. In some laboratory-based behavior metrics, adolescent rodents fail to show the same behaviors expressed by adults, but in other metrics, adolescent behavioral performance is more robust or more flexible than at other ages. These data are often interpreted in light of proximate level analysis of development of neural circuits. It is also informative to attempt ultimate-level explanations and consider how sex and species-specific adolescent behavioral changes support dispersal, foraging, and social interactions in the wild.

Wan Chen Lin, Linda Wilbrecht, Making sense of strengths and weaknesses observed in adolescent laboratory rodents, Current Opinion in Psychology, Volume 45, 2022, 101297, ISSN 2352-250X, https://doi.org/10.1016/j.copsyc.2021.12.009, https://www.sciencedirect.com/science/article/pii/S2352250X21002499

Making sense of strengths and weaknesses observed in adolescent laboratory rodents2022-06-18T20:42:23+00:00

Sex Differences in Pubertal Circadian and Ultradian Rhythmic Development Under Semi-Naturalistic Conditions

Biological rhythms in core body temperature (CBT) provide informative markers of adolescent development under controlled laboratory conditions. However, it is unknown whether these markers are preserved under more variable, semi-naturalistic conditions, and whether CBT may therefore prove useful in a real-world setting. To evaluate this possibility, we examined fecal steroid concentrations and CBT rhythms from pre-adolescence (p26) through early adulthood (p76) in intact male and female Wistar rats under natural light and climate at the Stephen Glickman Field Station for the Study of Behavior, Ecology and Reproduction. Despite greater environmental variability, CBT markers of pubertal onset and its rhythmic progression were comparable with those previously reported in laboratory conditions in female rats and extend actigraphy-based findings in males. Specifically, sex differences emerged in CBT circadian rhythm (CR) power and amplitude prior to pubertal onset and persisted into early adulthood, with females exhibiting elevated CBT and decreased CR power compared with males. Within-day (ultradian rhythm [UR]) patterns also exhibited a pronounced sex difference associated with estrous cyclicity. Pubertal onset, defined by vaginal opening, preputial separation, and sex steroid concentrations, occurred later than previously reported under lab conditions for both sexes. Vaginal opening and increased fecal estradiol concentrations were closely tied to the commencement of 4-day oscillations in CBT and UR power. By contrast, preputial separation and the first rise in testosterone concentration were not associated with adolescent changes to CBT rhythms in male rats. Together, males and females exhibited unique temporal patterning of CBT and sex steroids across pubertal development, with tractable associations between hormonal concentrations, external development, and temporal structure in females. The preservation of these features outside the laboratory supports CBT as a strong candidate for translational pubertal monitoring under semi-naturalistic conditions in females.

Grant, Azure D., Linda Wilbrecht, and Lance J. Kriegsfeld,  Sex Differences in Pubertal Circadian and Ultradian Rhythmic Development Under Semi-Naturalistic Conditions, Journal of Biological Rhythms, (May 2022). https://doi.org/10.1177/07487304221092715

Sex Differences in Pubertal Circadian and Ultradian Rhythmic Development Under Semi-Naturalistic Conditions2022-06-18T20:34:48+00:00

This site does not reflect the views of the University of California and should not be attributed to the University.