Introduction

Several authors have pointed out that operant learning plays an important role in the establishment of problematic behaviors such as drug abuse, self-injury, among others (e.g., Podlesnik & Kelley, 2015). Therefore, it has been suggested that research on operant extinction (the decrease in behavior due to the omission of the reinforcer), may provide useful insights for behavioral analysts dealing with the reduction of problem behavior (e.g., Bouton, Winterbauer, & Todd, 2012; Wathen & Podlesnik, 2018). For instance, contemporary studies indicate that the response reduction observed during extinction is not permanent, rather, it reappears under different circumstances (Bouton, 2019).

One of those circumstances is reinstatement, which is the reappearance of an extinguished behavior, caused by delivering the previously associated reinforcer independently of whether the subject performs the response (e.g., Rescorla & Skucy, 1969). For example, Baker, Steinwald, and Bouton (1991), trained hungry rats to press a lever for food for five days. Then, all rats received five extinction sessions. One day after the last extinction session, all rats were placed in the experimental chambers where the experimental group received a session without the lever and with food presentations, while the control group received nothing in the experimental chamber. The next day, the level of lever-pressing was registered in both groups. Baker et al. (1991) observed reinstatement only in the experimental group (see also, Delamater, 1997).

Since it has been recognized that the reinstatement of operant behaviors can be used as a laboratory model that contributes to our understanding of the factors involved in the relapse of problem behavior after a behavioral intervention (e.g., a person who has successfully completed their smoking cessation therapy may relapse if exposed to someone else's cigarette smoke), several authors have proposed that the evaluation of behavioral techniques which prevent reoccurrence might help therapists in the development of clinical strategies focused on thwarting relapse (e.g., Bernal-Gamboa, Gámez, & Nieto, 2017; Gámez & Bernal-Gamboa, 2018).

In recent years, the retrieval-extinction paradigm has been proposed as a behavioral technique to reduce the reinstatement of operant responding. In 2012, Xue et al. trained rats to self-administer heroin intravenously for 10 days. Then, one of the groups received a typical extinction procedure for 14 sessions (i.e., responding was no longer followed by a dose of heroin). The other group was exposed to the retrieval-extinction paradigm (i.e., a brief extinction session was conducted, then after a delay, a longer extinction session was carried out). After an acute non-contingent exposure to priming injections of heroin, rats were tested for reinstatement of (nonreinforced) operant responding. Xue et al. (2012) reported that the group that had received the retrieval-extinction paradigm showed lower levels of reinstatement.

Although this retrieval-extinction paradigm seems promising, it is important to note that, to the best of the author’s knowledge, those findings have not been replicated by other researchers. Thus, the main goal of the present experiment was to test the generalizability of the retrieval-extinction paradigm’s effectiveness in reducing the reinstatement of operant behaviors. Thereby, the differences between our experiment and the one reported by Xue et al. (2012) (Experiment 5) were the following: 1) Training, while Xue et al. (2012) used a Fixed Ratio 1 (FR1) schedule, we used a Variable Interval (VI) 30s schedule. 2) Reinforcers, in the original experiment heroin was employed, whereas food pellets were used in the present experiment. 3) Parameters of the extinction-retrieval paradigm, Xue et al. (2012) used a brief 15-minute extinction session, then, after a 10-minute delay, they conducted a longer 180-minute extinction session. In the present experiment, the brief extinction session lasted for 3 minutes, followed by a 20-minute delay, and afterwards a longer 30-minute extinction session was used.

The experimental design is presented in Table 1. All rats were trained to press a lever for food. Then, during Phase 2, one of the groups (Extinction) received the typical extinction procedure, while the other group of rats (Retrieval) was exposed to the retrieval-extinction paradigm (i.e., a brief extinction session was carried out, then, after a delay, a longer extinction session took place). Finally, all rats received two tests. The first test was carried out after the last session of Phase 2, whereas the second test was conducted after all rats had received free presentations of the food.

Method

Subjects

Sixteen female Wistar rats (8 per group) weighing in average 280.4 g were used. The rats were approximately three months old and experimentally naïve at the beginning of the experiment. They were housed individually in methracrylate cages (21 x 24 x 46 cm, height x width x depth) inside a room maintained on a 12-12 hr light-dark cycle (07:00 onset and 19:00 offset of lights). The temperature of the colony room ranged between 20–25 °C, while the humidity value was 45–60%. All subjects were maintained with ad libitum access to water but were food-deprived to 83% of their initial body weights throughout the experiment.

Ethical Aspects

The care and handling of the rats was carried out under the ethical standard 8.09 and its subsections a, b, c and d, as established in the "Ethical Principles of Psychologists and Codes of Conduct" of the American Psychological Association (2010); as well as in accordance with articles 76, 77, 78, 79 and 80 of the "Ethical Code of the Mexican Society of Psychology" (Sociedad Mexicana de Psicología, 2009) and with the Official Mexican Standard NOM-062-ZOO-1999 "Technical specifications for the production, care and use of laboratory animals” (Diario Oficial de la Federación, 2001). The present experimental protocol was also conducted under strict agreement with the guidelines established by the Ethical Committee of the Faculty of Psychology of the National University of Mexico.

Apparatus

Eight identical chambers manufactured by MED Associates (model ENV-008) measuring 29 cm x 22 cm x 24 cm (H x W x D) were used. Each chamber was enclosed in a sound-attenuating chamber equipped with an exhaust fan that produced a background noise of 60 dB. The side walls and ceiling were made of clear acrylic plastic, while the front and rear walls were made of stainless steel. The chamber floor consisted of sixteen 0.5-cm diameter stainless steel rods spaced 1.5 cm apart. A recessed 5 cm x 5 cm food magazine in which 45 mg Noyes A/I pellets could be delivered was centered on the front wall. Each chamber had one retractable lever, which was positioned to the left of the food tray. The levers were 4.8 cm long and positioned 6.8 cm above the floor. The chambers were connected to a PC that controlled and recorded the events.

Procedure

Since the parameters used by Xue et al. (2012) are not commonly used in the studies focused on extinction and the recovery of operant responses with rats (e.g., Bernal-Gamboa, Gámez, & Nieto, 2018), we explored whether the findings of Xue et al. (2012) could be extended to a more frequently used procedure. Thus, as we stated above-mentioned, we varied 1) the schedule of reinforcement used during training, 2) the type of reinforcer and 3) the length of the extinction-retrieval paradigm. Sessions were conducted on successive days, at the same time each day. During the first day, all rats received acclimation to the contexts. During these sessions, food pellets were delivered approximately 30 times on a variable time (VT) 30 s schedule. The levers were retracted. Sessions lasted for 15 min.

Phase 1. During the next five days, rats were trained to press the lever for food on a VI 30s schedule. Each session lasted for 30 minutes.

Phase 2. On each of the following days, rats in the Extinction Group received a continuous 33 minutes of the typical extinction sessions (i.e., pressing the lever did not produce food anymore). Throughout each day of this phase, the Retrieval Group received two extinction sessions of different durations, separated by a delay. The first extinction session lasted for 3 min (brief extinction session). Rats were then returned to their home cages, where they were placed for 20 minutes. After that, rats were moved to the experimental chambers, and received a longer extinction session, which lasted for 30 minutes. No pellets were presented during this phase.

Test 1. Immediately after the last session of Phase 2, all rats received a 10-minute test session in extinction. Rats could press the lever, but no pellets were presented during this phase.

Re-exposure. On the following day, rats were placed in the experimental chambers for 15 minutes, without the levers. Rats received approximately 30 food pellets that were delivered independently of their behavior. Pellets were delivered on a 30 s VT.

Test 2. The next day, all rats received a 10-minute test session identical to the one conducted during Test 1.

Dependent Variable and Statistical Analysis

Mean responses per minute were compared using analyses of variance (ANOVA). The rejection criteria was set at p < .05, and effect sizes were reported using partial eta-squared (ƞ_p²). Additionally, 90% confidence intervals for the effect sizes were calculated and reported for each analysis.

Results

Figure 1 shows the mean responses per minute during each session of Phase 1 (left panel).The figure indicates that both groups acquired the lever-pressing response similarly and that response rate increased as Phase 1 progressed. A 2 (Group) x 5 (Session) ANOVA conducted on the Phase 1 data showed only a significant main effect of Session, F(4, 56) = 22.80, p = .0001, ƞ_p²= .62, CI 90% [0.45, 0.69]. The main effect of Group, as well as all interactions related to this factor were not significant F’s < 1, confirming that both groups acquired the behavior in a similar manner.

The right panel of Figure 1 shows the mean responses per minute during each session of Phase 2, for both groups. Both groups showed a similar decrease in performance. A 2 (Group) x 3 (Session) ANOVA conducted on the Phase 2 data only found a significant main effect of Session, F(2, 28) = 23.19, p = .0001, ƞ_p² = .62, CI 90% [0.39, 0.72]. Neither the main effect of the Group, F(1, 14) = 1.02, p = .33, nor the Group x Session interaction, F’s < 1, were significant, showing no difference in performance.

Figure 2 shows the mean responses per minute for the Extinction and Retrieval groups in both test sessions. A 2 (Group) x 2 (Test) ANOVA showed a significant main effect of Group F(1, 14) = 13.24, p = .002, ƞ_p² = .49, CI 90% [0.14, 0.66], a main effect of Test F(1, 14) = 58.51, p = .0001, ƞ_p² = .80, CI 90% [0.58, 0.87], as well as the Group x Test interaction, F(1 , 14) = 6.50, p = .02, ƞ_p² = .32, CI 90% [0.03, 0.54]. Follow-up comparisons conducted to explore this interaction indicated that both the Extinction Group, F(1, 14) = 52.01, p = .0001, ƞ_p² = .77, CI 90% [0.54, 0.86] and the Retrieval Group, F(1, 14) = 13.00, p = .002, ƞ_p² = .37, CI 90% [0.13, 0.66] performed higher levels of lever-pressing during Test 2 (i.e., reinstatement). Furthermore, these analyses indicated that rats in the Retrieval Group showed lower levels of reinstatement in comparison with the Extinction Group, F(1, 14) = 9.72, p = .007, ƞ_p² = .26 , CI 90% [0.08, 0.61], thereby showing that the retrieval-extinction paradigm reduces reinstatement in a free operant task.

Discussion

In one experiment we evaluated the impact of the retrieval-extinction paradigm on the reinstatement of operant behaviors in rats. Our data indicated that all rats showed higher levels of responding in the second test as a consequence of re-exposure to food, which means that the reinstatement effect was reported in both groups. However, the data also showed that rats which had been exposed to the retrieval-extinction paradigm presented lower levels of reinstatement, suggesting that the retrieval-extinction paradigm is an effective behavioral technique to attenuate the reappearance of extinguished behaviors, caused by delivering the previously associated reinforcers independently of the response.

The present results are consistent with the conclusions of Xue et al. (2012), extending those findings to a situation that involved parametric modifications (i.e., training schedule, sessions’ lengths, reinforcer type). Likewise, our data is also consistent with research that has reported the reduction of other response recovery effects in operant procedures, such as renewal (e.g., Millan, Milligan-Saville, & McNally, 2013) and spontaneous recovery (Xue et al., 2012), thus supporting the generalizability of the retrieval-extinction paradigm.

Therefore, the retrieval-extinction paradigm can be combined with other behavioral techniques that reduce the reinstatement of operant behaviors, such as the presentation of cues associated with extinction during testing (Bernal-Gamboa et al., 2017; Gámez & Bernal-Gamboa, 2018), or using long retention intervals between extinction sessions (Bernal-Gamboa et al., 2018; Experiment 3).

Our findings show the efficacy of the retrieval-extinction paradigm in reducing reinstatement. However, it is important to note that the present experiment did not elucidate the mechanism that underlies the retrieval-extinction paradigm. Some authors have proposed that the underlying mechanism of the retrieval-extinction paradigm is based on memory reconsolidation, or memory updating (e.g., Monfils, Cowansage, Klann, & LeDoux, 2009; Schiller et al., 2010). According to this perspective, if the memory of the operant response initially predicted the delivery of the reinforcer (i.e., response-reinforcer), then the exposure to the brief extinction session would allow an update (or reconsolidation) of that memory. This implies changing the original memory to something like response-no reinforcer, which would permanently modify the behavior (i.e., the subject would show a performance similar to extinction since the lever-pressing would no longer produce the reinforcer). Some studies are in agreement with this perspective (e.g., Lee, Nader, & Schiller, 2017; Monfils & Holmes, 2018), nevertheless, our present results are problematic for the memory reconsolidation account. Although our results showed that the reinstatement was reduced, we still found a reinstatement effect nonetheless, which, according to the memory reconsolidation view, should not have occurred. Following that view, if the retrieval-extinction paradigm had indeed produced an update (or reconsolidation) of the memory generated during acquisition, then rats should have behaved as if training for lever-pressing had never taken place, thereby, no reinstatement should have been observed (see Millan et al., 2013; Xue et al., 2012, for similar results in renewal and spontaneous recovery).

A different account that could deal with the present data has been proposed by Millan et al. (2013). They suggest that the brief extinction session in the retrieval-extinction paradigm does not change the memories, but serves as a signal for further extinction (i.e., the longer extinction session) which helps to discriminate better between the training and the extinction memories instead. Thus, following Millan’s proposal, since rats were tested without reinforcement, the retrieval-extinction training should facilitate recalling extinction better, thereby producing an attenuation in reinstatement. Although our findings support Millan’s proposal, future experiments could continue to evaluate the mechanisms underlying the retrieval-extinction paradigm, not only for their theoretical value, but also for their possible implications in the applied field. A full understanding of the paradigm might facilitate an appropriate knowledge transfer for the development of a possible therapeutic strategy that may controls the relapse of problematic or unhealthy operant behaviors, such as pathological gambling or excessive intake of sweetened beverages.

Although we suggested that our data is consistent with the view of Millan et al. (2013), it is important to note that additional groups should be included in future studies to evaluated with more detail the predictions of that proposal. Moreover, another potential limitation of the present experiment that should be considered for subsequent research is conducting an experiment that directly compares the group with the parameters of Xue et al. (2012) and the group with different parameters. This kind of contrast may provide additional data to understand the effect and the underlying mechanisms.

In sum, we found that the retrieval-extinction paradigm reduces the reinstatement of food-seeking, using a free operant procedure with rats. In addition, our data suggests that the underlying mechanism does not depend on memory reconsolidation (because according to that view, the Retrieval Group should not have shown any reinstatement at all; see Millan et al., 2013 for a similar discussion), rather, it relies on an easy discrimination between training and extinction memories, as we stated in the above mentioned paragraph, Millan et al. (2013) proposed that the retrieval-extinction paradigm procedure helps to better discriminate when the response will not be reinforced (because this procedure is used during both extinction and test, it might act as an extinction reminder, see Bernal-Gamboa et al., 2017). In addition, given that this proposal does not assume a memory updating, a reduction but not elimination of reinstatement is expected.

Acknowledgments

This research is part of the first author's doctoral research. This research was funded by UNAM/DGAPA through grant project PAPIIT IN309720.

References

American Psychological Association (2010). Publication manual of the American Psychological Association (6th ed.). Washington D.C.: American Psychological Association.

Baker, A. G., Steinwald, H., & Bouton, M. E. (1991). Contextual conditioning and reinstatement of extinguished instrumental responding. Quarterly Journal of Experimental Psychology, 43(2B), 199–218. doi: 10.1080/14640749108401267

Bernal-Gamboa, R., Gámez, A. M., & Nieto, J. (2017). Reducing spontaneous recovery and reinstatement of operant performance through extinction-cues. Behavioural Processes, 135, 1–7. doi: 10.1016/j.beproc.2016.11.010

Bernal-Gamboa, R., Gámez, A. M., & Nieto, J. (2018). Spacing extinction sessions as a behavioral technique for preventing relapse in an animal model of voluntary actions. Behavioural Processes, 151, 54-61. doi: 10.1016/j.beproc.2018.01.021

Bouton, M. E. (2019). Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control. Psychopharmacology, 236(1), 7-19. doi: 10.1007/s00213-018-5076-4

Bouton, M. E., Winterbauer, N. E., & Todd, T. P. (2012). Relapse processes after the extinction of instrumental learning: Renewal, resurgence, and reacquisition. Behavioural Processes, 90(1), 130–141. doi: 10.1016/j.beproc.2012.03.004

Delamater, A. R. (1997). Selective reinstatement of stimulus-outcome associations. Animal Learning & Behavior, 25(4), 400-412.

Diario Oficial de la Federación (2001). Norma Oficial Mexicana NOM-062-ZOO-1999: Especificaciones técnicas para la Producción, Cuidado y Uso de los Animales de Laboratorio. Recuperado de: http://www.ibt.unam.mx/computo/pdfs/bioterio.NOM-062.pdf

Gámez, A. M., & Bernal-Gamboa, R. (2018). Reinstatement of instrumental actions in humans: Possible mechanisms and their implications to prevent it. Acta Psychologica, 183, 29–36. doi: 10.1016/j.actpsy.2017.12.012

Lee, J. L. C., Nader, K., & Schiller, D. (2017). An update on memory reconsolidation updating. Trends in Cognitive Sciences, 21(7), 531-545. doi: 10.1016/j.tics.2017.04.006

Millan, E. Z., Milligan-Saville, J., & McNally, G. P. (2013). Memory retrieval, extinction, and reinstatement of alcohol seeking. Neurobiology of Learning and Memory, 101, 26-32. doi: 10.1016/j.nlm.2012.12.010

Monfils, M. H., Cowansage, K. K., Klann, E., & LeDoux, J. E. (2009). Extinction–reconsolidation boundaries: Key to persistent attenuation of fear memories. Science, 324(5929), 951–955. doi: 10.1126/science.1167975

Monfils, M. H., & Holmes, E. A. (2018). Memory boundaries: opening a window inspired by reconsolidation to treat anxiety, trauma-related, and addiction disorders. Lancet Psychiatry, 5(12), 1032-1042. doi: 10.1016/S2215-0366(18)30270-0

Podlesnik, C. A., & Kelley, M. E. (2015). Translational research on relapse of operant behavior. Mexican Journal of Behavior Analysis, 41(2), 226–251.

Rescorla, R. A., & Skucy, J. C. (1969). Effect of response-independent reinforcers during extinction. Journal of Comparative and Physiological Psychology, 67(3), 381–389. doi: 10.1037/h0026793

Schiller, D., Monfils, M. H., Raio, C. M., Johnson, D. C., Ledoux, J. E., & Phelps, E. A. (2010). Preventing the return of fear in humans using reconsolidation update mechanisms. Nature, 463(7277), 49–53. doi: 10.1038/nature08637

Sociedad Mexicana de Psicología (2009). Código ético del psicólogo. México D.F.: Editorial Trillas.

Wathen, S. N., & Podlesnik, C. A. (2018). Laboratory models of treatment relapse and mitigation techniques. Behavior Analysis: Research and Practice, 18(4), 362-387. doi: 10.1037/bar0000119

Xue, Y. X., Luo, Y. X., Wu, P., Shi, H. S., Xue, L. F., Chen, C. … Lu, L. (2012). A memory retrieval–extinction procedure to prevent drug craving and relapse. Science, 336(6078), 241–245. doi: 10.1126/science.1215070

Notas de autor

Enviar correspondencia a: Mason, T. A. E-mail: terealmason@hotmail.com

Información adicional

Citar este artículo como: Mason, T.A., Bernal-Gamboa, R., & Nieto, J. (2021). The reinstatement of operant behavior is reduced by the retrieval-extinction paradigm. Revista Argentina de Ciencias del Comportamiento, 13(2), 70-76

Enlace alternativo

https://revistas.unc.edu.ar/index.php/racc/article/view/28557/34945 (pdf)