Introduction

Magnitude, quality, rate, and delay are reinforcement parameters that affect the rate at which subjects respond (Kimble, 1961). Delay of reinforcement has been assessed by increasing the temporal interval between a response and the reinforcer. The common result of such a manipulation is a decrease in the rate of response as the temporal interval between the response and reinforcer increases. This effect has been identified as delay of reinforcement gradient ( for recent reviews see Lattal, 2010; Pulido, Lanzagorta, Morán, Reyes, & Rubí, 2004).

Assessing delay of reinforcement is relevant because even when the response-reinforcer contingency is maintained, a disruption in temporal contiguity between both elements affects the rate of responding (Avila & Bruner, 1997; Dews, 1960; Ferster, 1953; Lattal, 1987; Ruiz, Bruner, & Balderrama, 2007; Skinner, 1938; Sosa & Pulido, 2011). When assessing delay of reinforcement, different procedures have been used. For example, most experiments arrange a two-link procedure in which the first link determines when the response will produce the food delivery, and the second link determines the interval between such response and the food delivery (e.g., 0, 5, 10 s). This latter interval has been identified as the nominal delay; therefore, manipulations have focused on that interval. For instance, certain experiments have involved the use of a resetting delay, where responding resets the timer for the delay interval until no responding occurs (Gleeson & Lattal, 1987; Skinner, 1938; van Haaren, 1992), while a nonresetting delay has been studied in others (Avila & Bruner, 1999; Bell, 1999; Lattal, 1984). In the latter, no consequences for responding are programmed, since the latter allows the subject to respond during the delay, thus decreasing the obtained delay. Nevertheless, it has been argued that when using a resetting delay, the decrease in responding due to the delay could be cofounded with a reduction in the rate of reinforcement resulting from continued responding during the delay period (Lattal, 2010). Likewise, in any of the above-mentioned procedures, a stimulus change may or may not be programmed during the delay. Such procedures have been identified as signaled and unsignaled, respectively (Chung, 1965; Ferster, 1953; Pierce, Hanford, & Zimmerman, 1972), where tandem or chained schedules of reinforcement arrange food delivery.

By signaling the delay, in contrast to an unsignaled delay, a higher rate of responding typically is observed. This result has been explained by means of the conditioned reinforcing function of the signal, where its introduction increases response rates (Azzi, Fix, Keller, & Rocha e Silva, 1964; Lattal, 2010; Pulido et al., 2004). However, Ferster (1953) and Dews (1960) have also explained these results by appealing to mediating behavior occurring during the delay. Thus, a stimulus during delay can develop a two-way function, either as conditioned reinforcer of responding prior to its onset, and as discriminative stimulus of other behavior during its presentation (see Dinsmoor, 1950; Myers, 1958).

Delay of reinforcement has been assessed using different procedures. In one, the delay is systematically changed across different phases (Bruner, Avila, & Gallardo, 1994; Dews, 1960; Lattal, 1984). Another involves using multiple schedules of reinforcement, in which one component is correlated with immediate reinforcement and the other components are correlated with different delay values (Richards, 1973; Ruiz et al., 2007). Other manipulations have assessed the effects of signaled and unsignaled delays across different phases of the experiment (Azzi et al., 1964; Richards, 1981; Schaal & Branch, 1988) or between different components of multiple schedules of reinforcement (Bell, 1999; Podlesnik, Jimenez-Gomez, Ward, & Shahan, 2006; Schaal & Branch, 1990). In one example of the former, Azzi et al. (1964) established lever-pressing in rats using a continuous reinforcement schedule (CRF) where each lever press delivered the programmed reinforcer. Once responding was established, they introduced different delay values in successive phases (i.e. 1, 3, 5, 7.5, 10, 15 and 20 s). Responding decreased as the delay was increased from 0 s in the CRF condition to 20 s in the last phase. In the second part of the experiment, they introduced a stimulus change (blackout) during the 20-s delay. During this condition responding increased in contrast to the condition when no stimulus change was in effect during the delay. Richards (1981) also compared key peck responding of pigeons under different signaled and unsignaled delay values. Like Azzi et al., Richards reported that pigeons responded at higher rates when delays were signaled, and such rates were similar or only slightly below those observed during the immediate reinforcement condition.

Another procedure that has been used to assess the effects of signaled and unsignaled delay of reinforcement is using multiple schedules of reinforcement. For instance, using a two-component multiple schedule, Schaal and Branch (1990) examined the effects of different signal durations imposed during a 27-s delay. In a similar experiment, Bell (1999) assessed the effects of immediate, unsignaled and signaled delay of reinforcement, using a three-component multiple schedule. In both experiments (Bell, 1999; Schaal & Branch, 1990), higher response rates occurred during the signaled delay than during the unsignaled delay component. Nevertheless, opposite to what has been reported in between-phases procedures (Azzi et al., 1964; Richards, 1981), or even to what Schaal and Branch reported using a multiple schedule, Bell found differences between signaled delayed and immediate reinforcement, observing a higher response rate during the former.

Even though in certain experiments (Bell, 1999; Podlesnik et al., 2006; Schaal & Branch, 1990) the effect of delay and reinforcement rate are not confounded, other experiments have reported conflicting findings. For instance, Schaal and Branch’s (1990) results are similar to what is commonly reported, that is, higher response rates with immediate reinforcement in contrast to rates during any of the delay conditions; and higher rates under signaled than unsignaled delay (Azzi et al., 1964; Richards, 1981; Schaal & Branch, 1988). In contrast, subjects in Bell’s (1999) experiment responded at higher rates during the fully signaled delay condition than during the immediate-reinforcement condition. Such conflicting results (Bell, 1999) seem difficult to explain since they differ from procedures used for assessing delay. Nevertheless, provided that similar contingencies of reinforcement are arranged, procedural differences should not be accounted for the result, but mostly the fact that when using a multiple schedule other experiments (Podlesnik et al., 2006; Schaal & Branch, 1990) have yielded data that are consistent with most of the delay of reinforcement results when the different delay durations are assessed across phases (Azzi et al., 1964; Bruner et al., 1994; Dews, 1960; Lattal, 1984; Richards, 1981).

The experiments described above suggest that there are two reliable procedures for assessing delay of reinforcement, and both seem to generate the same results. Nevertheless, using a between-phases procedure requires a larger number of sessions to complete; whereas, training subjects using multiple schedules would require less time, since different variables are assessed in each component and thus can contrasted with one another. Such convenience could allow better use of the financial and animal resources available.

Given such considerations, the present experiment was conducted in an attempt to replicate the effects of delays of reinforcement by comparing the effects of immediate, signaled and unsignaled delay of reinforcement across different phases (Azzi et al., 1964; Ferster, 1953; Lattal, 1984), to the effects obtained when those same conditions were imposed as components of a three-component multiple schedule of reinforcement (Bell, 1999; Podlesnik et al., 2006), while keeping the interreinforcer intervals equal regardless of the delay in effect (see Bruner, Pulido, & Escobar, 1999; Flores & Mateos, 2009; Weil, 1984).

Method

Subjects

Eight five-month old, experimentally naïve female Wistar rats were used. Each rat was maintained at approximately 80% of its free-feeding weight throughout the experiment. All subjects were individually housed in a temperature-controlled room under a 12:12 hr. Light/dark cycle with free water-access. Subjects care and housing complied with the animal care guidelines established by the Universidad de Guadalajara, and according to NOM-062-ZOO-1999 (Norma Oficial Mexicana).

Apparatus

Four Med Associates Inc. (ENV-008) operant conditioning chambers were used, all equipped with the following Med Associates Inc®. Apparatus. A food dispenser (ENV-203M), which delivered 45-mg grain-based pellets for rodents, was located to the center of the work panel; a stainless-steel, 4.5cm long retractable lever (ENV-112CM) was placed to the left side of the food dispenser at a height of 7cm from the chamber’s floor, lever activation required a 0.25 N; a 2-cm diameter translucent light (28-V) was placed above the lever; on the wall opposite the work panel, a white noise amplifier (ENV-225SM) and a speaker producing 80db of 2000Hz were placed. Each chamber was enclosed in a sound-attenuating box (ENV-022MD) equipped with ventilation fans that provided external-sound masking. A Windows computer coupled to a Med Associates Inc interface (SG6080D) and the MED-PC IV® software controlled the experimental events and collected data.

Procedure

All rats were exposed to one session daily. Each session started with insertion of the lever 2-minutes after rats were placed in the chamber. Given that two different procedure were used, session durations varied between experimental groups.

Training. All rats were trained to enter the food magazine and to lever press by the differential-reinforcement-of-successive-approximations procedure during three sessions. Each of these sessions terminated after 60 min or once the rat produced 40 reinforcers under a fixed- ratio 1 schedule, whichever occurred first. Once 40 reinforcers were delivered in two consecutive sessions, lever pressing was maintained using a variable interval (VI) schedule, the mean interreinforcer interval of which increased gradually before the final baseline VI schedule value was used. The criterion for ending each session was the same as the previously described. Once all rats reached the final baseline VI value, 2 four-rats groups were formed (a 3-Phase Group and a Multiple-Schedule Group). Rats were assigned to each group by matching final training response rates to the greatest extent.

Between-Phases procedure. During the first phase (Immediate), reinforcers were delivered according to a VI 30-s schedule and no delay was programmed. For the second, Unsignaled Delay, phase, reinforcers were programmed according to a tandem VI 25- s fixed- time (FT) 5-s schedule, where the second link corresponded to the 5-s delay, during which no stimulus change was programmed. The third phase (Signaled Delay) was the same as the second, except that a 5 s white light above the lever during the delay, converting the schedule to a chained VI 25-s FT 5-s schedule. Each phase was in effect for thirty 30-min sessions.

Multiple-Schedule procedure. Rats were exposed to the same conditions as those in the Between-Phase procedure (Immediate, Unsignaled Delay, and Signaled Delay), except that each of these conditions were correlated with different components of a 3-component multiple schedule. Each 2-min component started with the insertion of the lever and was presented 8 times so that during a single session a total of 24 component presentations were arranged. Components occurred quasirandomly, with the restriction that the same component could not be presented more than two consecutive times. No events were programmed during a 30-s intercomponent interval, during which the lever was retracted. Component 1 was correlated with the immediate reinforcer, Components 2 and 3 were correlated with the unsignaled and signaled delays, respectively. During each component, the absence of an added auditory stimulus was correlated with the component arranging immediate reinforcement; whereas for the unsignaled- and signaled-delay components, a fixed or intermittent tone (0.5 s on/off) was used, respectively. Rats were maintained in this condition for 30 sessions.

Results

Rats’ mean responses per minute during the last five sessions of each condition are presented in Figure 1. Responses per minute for the Multiple-Schedule and the 3-Phase procedures are shown in the left and right columns, respectively. Responding during the 5-s delay was excluded, thus only mean responses per min during the VI 25-s component are shown.

For all rats, response rates were higher during the Immediate condition in contrast to either of the Delay conditions. Likewise, with the exception of Rat 5, mean responses per minute during the Signaled-Delay condition were higher than those during the Unsignaled-Delay condition, regardless of the procedure.

Figure 1
Mean responses per minute

Mean responses per minute of each rat based on the last five sessions of each condition. Rats exposed to the Multiple-Schedule and Between-Phases procedure are shown in the left and right columns, respectively. Error bars represent the standard error of the mean (SEM).

Figure 2 shows the mean number of responses during the Unsignaled and Signaled Delay conditions calculated from the last five sessions in each condition. Differences in responding during the delays were observed between rats. For rats trained under the multiple schedule, response rates were lower during the signaled delay for Rats 2 and 3; whereas for the other two rats, no differences in responding occurred between unsignaled and signaled delays. Rats trained under the Between-Phases procedure had higher response rates during the Unsignaled Delay condition.

Figure 2
Mean number of responses

Mean number of responses during the last five sessions of each condition for rats trained using the Multiple-Schedule and Between-Phases procedure are shown in the left and right columns, respectively. Error bars represent the standard error of the mean (SEM).

Mean reinforcers per minute (S. / min), across the different conditions are shown in Table 1. Reinforcers per minute during the last five sessions of each condition were similar for rats trained under the Multiple-Schedule procedure; in contrast to rats trained under the Between- Phases procedure, where for three of the four rats the S./ min varied between conditions.

Table 1

Mean reinforcers per minute SR min for each rat calculated from the last five sessions of each condition

Note. Reinforcers per minute for rats trained under the Multiple-Schedule (Rats 1, 2, 3, 4) and the Between-Phases procedure (Rats 5, 6, 7, 8) appear in the upper and lower portion of the table, respectively.

Discussion

In the present experiment the effects of the immediate, signaled and unsignaled delay of reinforcement on lever-pressing response rates (responses per minute) were compared using two procedures: a three-component Multiple-Schedule and a Between-Phases procedure. As previous experiments (Azzi et al., 1964; Bell, 1999; Lattal, 1984; Mateos, 2012; Schaal & Branch, 1988), response rates using either procedure were higher when food was delivered immediately when compared to either of the delay conditions. Likewise, with the exception of Rat 5, response rates were higher under signaled than unsignaled delay condition. Thus, overall, consistent results were observed regardless of the procedure used.

The latter finding is consistent with those of several previous experiments (Azzi et al., 1964; Pulido, Rubí, & Backer, 2008; Richards, 1981; Ruiz et al., 2007). In some cases, this result has been explained in terms of a conditioned reinforcement function developed by the signal. Also, as was mentioned, response-dependent signals can develop a dual function; as a conditioned reinforcer when its onset depends upon responding, and as a discriminative stimulus when its presentation controls a different type of responding (Dinsmoor, 1950; Myers, 1958). Some evidence of the latter was observed in the current experiment, where responding during the VI component and during the delay differed between rats. That is, most of the rats responded more when no signal was added during the delay; whereas for others, responding was equal or greater when a stimulus was added. This difference in responding could be accounted for by the discriminative function developed by the signal during its presentation, provided that it controlled a different response topography other than lever-pressing once the latter decreased. An increased operant responding frequency during the delay when a stimulus is added is not at odds with other experiments. For instance, Azzi et al., (1964) reported that most of their rats responded at a higher rate when delay was signaled than when no signal was present; likewise, Lattal (1984) reported a similar result in most of his pigeons. Even though Azzi et al. Observed such result in most of their rats, it’s quite interesting how some days one of theirs rats responded less during the signaled delay interval, which in fact is what we observed for most of our rats (see also Chung, 1965; Chung & Herrnstein, 1967; Mateos, 2012). The present results during the signaled delay periods are consistent with the variable effects just noted. Likewise, it must be noted that the type of stimulus change during delay has differed across different experiments, which could explain the reported differences in responding during the delay interval (Lattal, 1987). Leaving this variability within and across experiments aside, our results are similar with respect to other comparisons of signaled and unsignaled delays on operant responding maintained by positive reinforcement (Lattal, 1984; Mateos, 2012; Schaal & Branch, 1988, 1990).

Regardless of which procedure was used, the present results were consistent in yielding similar differences between signaled and unsignaled reinforcement delays. These findings could pave the way to future research aimed at the assessment of multiple variables in a delay of reinforcement experiment, where even when multiple sessions could be required, a broader array of results can be provided by a single phase instead of multiple phases. Most importantly, it provides some support to previous findings using multiple schedules of reinforcement (Bell, 1999; Podlesnik et al., 2006; Ruiz et al., 2007; Schaal & Branch, 1988, 1990) where no direct comparison with between-phases procedures was done.

Acknowledgments

The authors acknowledge the thoughtful comments made to a preliminary version presented at the XXVII Annual Meeting of the Mexican Society for Behavior Analysis in Aguascalientes, Aguascalientes.

References

Avila, R., & Bruner, C. A. (1997). Efectos de historia en gradientes obtenidos alargando y acortando la demora de reforzamiento. Revista Mexicana de Análisis de la Conducta, 23(1), 85–96.

Avila, R., & Bruner, C. A. (1999). Adquisición del palanqueo en ratas con reforzamiento demorado en un procedimiento de ensayo discreto. Revista Latinoamericana de Psicología, 31(1), 49–64.

Azzi, R., Fix, D., Keller, F., & Rocha e Silva, M. (1964). Exteroceptive control of response under delayed reinforcement. Journal of the Experimental Analysis of Behavior, 7(2), 159–162.

Bell, M. (1999). Pavlovian contingencies and resistance to change in a multiple schedule. Journal of the Experimental Analysis of Behavior, 72(1), 81–96. Doi: 10.1901/jeab.1999.72-81

Bruner, C. A., Avila, R., & Gallardo, L. M. (1994). La adquisición del palanqueo en ratas bajo un programa intermitente de reforzamiento demorado. Revista Mexicana de Análisis de la Conducta, 20(2), 119–129.

Bruner, C. A., Pulido, M. A., & Escobar, R. (1999). Response acquisition and maintenance with a temporally defined schedule of delayed reinforcement. Revista Mexicana de Análisis de la Conducta, 25(3), 379–391.

Chung, S. H. (1965). Effects of delayed reinforcement in a concurrent situation. Journal of the Experimental Analysis of Behavior, 8(6), 439–444. Doi: 10.1901/jeab.1965.8-439

Chung, S. H., & Herrnstein, R. J. (1967). Choice and delay of reinforcement. Journal of the Experimental Analysis of Behavior, 10(1), 67–74. Doi: 10.1901/jeab.1967.10-67

Dews, P. B. (1960). Free-operant behavior under conditions of delayed reinforcement. I. CRF-Type schedules. Journal of the Experimental Analysis of Behavior, 3(3), 221–234. Doi: 10.1901/jeab.1960.3-221

Dinsmoor, J. A. (1950). A quantitative comparison of the discriminative and reinforcing functions of a stimulus. Journal of Experimental Psychology, 40(4), 458– 472. Doi: 10.1037/h0056266

Ferster, C. B. (1953). Sustained behavior under delayed reinforcement. Journal of Experimental Psychology, 45(4), 218–224.

Flores, C., & Mateos, L. R. (2009). Adquisición del responder bajo condiciones de reforzamiento demorado en programas temporales señalados. Universitas Psychologica, 8(1), 237–246.

Gleeson, S., & Lattal, K. A. (1987). Response-Reinforcer Relations and the Maintenance of Behavior. Journal of the Experimental Analysis of Behavior, 48, 383–393. Doi: 10.1901/jeab.1987.48-383

Lattal, K. A. (1984). Signal functions in delayed reinforcement. Journal of the Experimental Analysis of Behavior, 42(2), 239–253. Doi: 10.1901/jeab.1984.42-239

Lattal, K. A. (1987). The effect of delay and intervening events on reinforcement value. In M. L. Commons, J. A. Nevin, & H. Rachlin (Eds.), Quantitative Analysis of Behavior (Vol. 5). New Jersey: Lawrence Erlbaum Associates Publisher.

Lattal, K. A. (2010). Delayed reinforcement of operant behavior. Journal of the Experimental Analysis of Behavior, 93(1), 129–139. Doi: 10.1901/jeab.2010.93-129

Mateos, L. R. (2012). Análisis experimental de los efectos de la demora de reforzamiento señalada vs. No señalada: funciones de estímulo. Universidad de Guadalajara.

Myers, J. L. (1958). Secondary reinforcement: a review of recent experimentation. Psychological Bulletin, 55(5), 284–301. Doi: 10.1037/h0046125

Pierce, C. H., Hanford, P. V, & Zimmerman, J. (1972). Effects of different delay of reinforcement procedures on variable-interval responding. Journal of the Experimental Analysis of Behavior, 18(1), 141–146.

Podlesnik, C. A., Jimenez-Gomez, C., Ward, R. D., & Shahan, T. A. (2006). Resistance to Change of Responding Maintained by Unsignaled Delays to Reinforcement: A Response-Bout Analysis. Journal of the Experimental Analysis of Behavior, 85(3), 329–347. Doi: 10.1901/jeab.2006.47-05

Pulido, M. A., Lanzagorta, N., Morán, E., Reyes, A., & Rubí, M. (2004). El efecto de las señales en programas de reforzamiento demorado: una revisión contemporánea. Enseñanza e Investigación En Psicología, 9(2), 321–339.

Richards, R. W. (1973). Stimulus generalization and delay of reinforcement during one component of a multiple schedule. Journal of the Experimental Analysis of Behavior, 19(2), 303–309. Doi: 10.1901/jeab.1973.19-303

Richards, R. W. (1981). A comparison of signaled and unsignaled delay of reinforcement. Journal of the Experimental Analysis of Behavior, 35(2), 145–152. Doi: 10.1901/jeab.1981.35-145

Ruiz, J. A., Bruner, C. A., & Balderrama, D. M. (2007). Efecto de tiempo relativo en demoras de reforzamiento señaladas y no señaladas. Revista Mexicana de Análisis de la Conducta, 33(2), 119–138.

Schaal, D. W., & Branch, M. N. (1988). Responding of pigeons under variable-interval schedules of unsignaled, briefly signaled, and completely signaled delays to reinforcement. Journal of the Experimental Analysis of Behavior, 50(1), 33–54. Doi: 10.1901/jeab.1988.50-33

Schaal, D. W., & Branch, M. N. (1990). Responding of pigeons under variable-interval schedules of signaled-delayed reinforcement: effects of delay-signal duration. Journal of the Experimental Analysis of Behavior, 53(1), 103–121.

Skinner, B. F. (1938). The Behavior of Organisms. New York: Appleton-Century- Crofts.

Sosa, R., & Pulido, M. A. (2011). Response acquisition with delayed conditioned reinforcement. Revista Mexicana de Análisis de la Conducta, 37(2), 83–98. Doi: 10.5514/rmac.v37.i2.26141

Van Haaren, F. (1992). Response acquisition with fixed and variable resetting delays of reinforcement in male and female wistar rats. Physiology and Behavior, 52(4), 767–772. Doi: 10.1016/0031-9384(92)90412-U

Weil, J. L. (1984). The effects of delayed reinforcement on free-operant responding. Journal of the Experimental Analysis of Behavior, 2(2), 143–155.