Introduction

Zebrafish (Danio rerio) widely serve as an animal model in biological research including developmental biology, neuroscience, pharmacology, toxicology, and genetics for having such features as high fecundity, transparent embryos, rapid development, and low maintenance cost (e.g., Gerlai, 2015; Gerlai, Lahav, Guo, & Rosenthal, 2000; Gould, 2011; Stewart, Braubach, Spitsbergen, Gerlai, & Kalueff, 2014). Their genome has been fully sequenced (Howe, Clark, Torroja, Torrance, Berthelot, Muffato, et al., 2013) and many different types of genetically-modified zebrafish are available for research (e.g., National Bioresource Project of Japan). Zebrafish are a relatively new animal model in behavioral research, but general principles of behavior observed in other species (e.g., rats and pigeons) also appear in zebrafish. For example, zebrafish acquire a conditioned avoidance response to a stimulus associated with an aversive stimulus such as electric shock (e.g., Aoki, Tsuboi, & Okamoto, 2014; Morin, de Souza Silva, Müller, Hardigan, & Spieler, 2013). Other operant processes also have been investigated with zebrafish, including reinforcement (Kuroda & Mizutani, 2018; Manabe, Dooling, & Takaku, 2013a, b), punishment (Kuroda, Mizutani, Cançado, & Podlesnik, 2019), stimulus control (Colwill, Raymond, Ferreira, & Escudero, 2005), and the relapse of previously reinforced and extinguished behavior (Kuroda, Mizutani, Cançado, & Podlesnik, 2017a,b). Results of these experiments generally are similar to those obtained with other species. Zebrafish thus have high potential for use in examining relations between biological and behavioral processes.

Social behavior is another notable feature of zebrafish – they naturally form a shoal (i.e., being close in space to each other) both in the wild (Suriyampola, Shelton, Shukla, Roy, Bhat, & Martins, 2016) and in laboratory settings (e.g., Buske & Gerlai, 2011a,b, 2012; Kalueff, Stewart, & Gerlai, 2015; Mahabir, Chatterjee, & Gerlai, 2013). When given a choice between compartments with and without other zebrafish, they prefer the former compartment (Stewart, Nguyen, Wong, Poudel, & Kalueff, 2014; Qin, Wong, Seguin, & Gerlai, 2014). The approach to other fish seems specific to conspecifics, as zebrafish show “fear responses” (e.g., jumping and erratic movement) in the presence of natural predators, such as the Indian leaf fish (Nandus nandus;Bass & Gerlai, 2008; Gerlai, Fernandes, & Pereira, 2009). Such aversive responses to predators occur even for those zebrafish raised in a laboratory where the predator has never previously been encountered (Kuroda et al., 2019).

These findings suggest that genes play a direct role in at least some aspects of zebrafish social behavior. Stronger evidence for genetic control over zebrafish social behavior comes from genetic research. For example, samdori 2 (a member of a chemokine-like gene family) knockout zebrafish show tighter social cohesion than wild-type zebrafish when placed in a novel environment (Choi et al., 2018). Silencing subregions of the dorsal habenula using a transgenic line of zebrafish results in more aggressive behavior toward conspecifics, which affects social hierarchy (Chou et al., 2016). Although this research indicates genetic influence on zebrafish social behavior, the role of learning processes in the development of their social behavior has received little investigation.

Establishment of social behavior with reinforcement has been reported with other species. Pigeons playing “ping-pong” is a classic example (Skinner, 1962). A pigeon was placed on either side of a small table. Each could receive food on the opponent pigeon failing to peck a ping-pong ball moving toward it, perhaps rudimentarily illustrating “competition.” Skinner also provided an example of “cooperation” in which a pair of pigeons, separated apart with a glass partition, was given a set of three vertically-aligned keys for each. The pigeons could receive food upon pecking a corresponding pair of keys (e.g., top keys) at the same time. Likewise, Tan and Hackenberg (2016) arranged a reinforcement contingency in which a pair of rats could receive food by pressing levers contiguously in time (i.e., within 0.5 s). The coordinated lever-pressing was higher in the presence of the contingency than in a control condition in which food was delivered independently of responding (see also Chalmeau & Gallo, 1995; Crawford, 1937; de Carvalho et al., 2018; Łopuch & Popik, 2011). Epstein, Lanza, and Skinner (1980) trained a more complex form of social behavior in a pair of pigeons named “Jack” and “Jill.” Jack pecked a key labeled “What color?” and then its partner Jill reported a color illuminated behind a curtain by pecking a corresponding key (e.g., “Y” for yellow) visible to Jack. Next Jack pecked a key labeled “Thank you,” which operated a feeder for Jill. Finally, Jack could receive food by pecking a key illuminating the correct color (i.e., yellow in this case).

A survey of the previous studies reveals that there is a difference in the way social behavior has been defined in biological and behavioral research. In biological research, it has been a common practice to define social behavior by its structure. For example, terms describing a structure or form of a group such as shoal (Buske & Gerlai, 2011a,b, 2012; Kalueff et al., 2015; Mahabir et al., 2013; Suriyampola et al., 2016), cohesion (Choi et al., 2018), and hierarchy (Chou et al., 2016) often are used when describing zebrafish social behavior. Accordingly, physical measures such as the number of fish in a shoal, the time spent with conspecifics (Stewart, Nguyen et al., 2014) and the distance between fish (Qin et al., 2014) often serve as indices of social behavior. In contrast, behavior analysts generally rely on functional definitions for the behavior they study. In some studies mentioned above, social behavior is referred to as the behavior of a pair of subjects that together function to produce food (“cooperation” described in Skinner, 1962; Tan & Hackenberg, 2016). In other case, social behavior is referred to as a subject’s responding that changes as a function of its opponent’s responding (“competition,” described in Skinner, 1962). In yet another study, social behavior was referred to as the mutual exchange of reinforcers between a pair of subjects (Epstein et al., 1980). Given these functions, it is common in behavior analysis to use a rather arbitrary response (e.g., responding on a discrete mechanical operandum such as a pigeon response key) as an index of social behavior.

Zebrafish serve as an animal model in both biological and behavioral research and so a balance between structural and functional definitions for social behavior appears more fruitful; strictly relying on either may limit the communication between these two fields. A potentially useful tool for measuring different aspects of social behavior is the 3D tracking system developed by Kuroda (2018). This system can track the motion of animal in real time. Moreover, a “virtual” operandum can be set anywhere in a 3D space for studying operant behavior. Thus, it is possible, for example, to set a virtual operandum on a moving zebrafish so that this fish serves as the operandum for another fish. The tracking system also has a capacity to detect the locations of multiple subjects at a given moment in a space, thereby capturing the distance between them as well as an overall structure of a shoal.

The goal of the present experiment was to examine whether reinforcement contingencies increase the approach to conspecifics in zebrafish while tracking their motions with the method described by Kuroda (2018). A reinforcement contingency was arranged in a way that a pair of zebrafish could receive food by approaching one another. Approaching conspecifics was selected as a target response here as an initial step toward the study of social behavior because it likely is a precursor to many forms of social behavior (e.g., cooperation).

Method

Subjects

Four pairs of experimentally naïve male and female zebrafish (Danio rerio) served as subjects. The fish were wild-type (not genetically modified), obtained from the National Bioresource Project of Japan, Riken Center for Brain Science, and each was 8 months old at the start of the experiment. Each pair was housed in a 15 cm × 15 cm × 15 cm aquarium made with 0.5-cm thick glass. A 0.3-cm thick glass partition (12 cm × 14 cm) was placed on the midline of front and back walls, creating two equal-sized compartments inside the aquarium (see Figure 1). Each member of a pair (one male and one female) was placed in the left and right compartments, respectively. The presence of the glass partition was to aid a technical limitation of the tracking system – the tracking system had the capacity to detect multiple zebrafish but not to track their motions individually in the absence of their identification. The presence of the partition restricted the possible location of each fish (e.g., the left compartment for a male fish), thereby allowing for their identification.

Water was filled to a level of 10 cm in each compartment of the aquarium. The water was maintained at 25 degrees Celsius with a thermostat heater except during sessions. Half of the water was replaced with fresh, aerated dechlorinated water weekly. The aquarium was maintained in a room with a 14hr:10hr light-dark cycle (lights on at 6:00 a. m.). Postsession feedings of supplement food (Kyorin, Hikari Labo 270) occurred 30 min after sessions when the fish obtained fewer than 30 food deliveries in a session.

Apparatus

The home/experimental aquarium was elevated by placing it on the edges of two empty aquariums (see Figure 1). A 3D camera (Intel. RealSense^TM D435 model) was placed underneath, 20 cm away from, and parallel to, the bottom of the aquarium. An infrared (IR) projector built in the camera was turned on in an attempt to increase the precision in depth measurement. The camera generated approximately 30 sets of color and depth frames per second. These frames went through a series of image processing to track the center of their body (see Kuroda, 2018, for details).

On the top of the aquarium was a Plexiglas lid with a feeder (Manabe et al., 2013a) placed above each compartment of the aquarium. Each feeder delivered decapsulated brine shrimp eggs as food reinforcers through an aperture on the Plexiglas lid. A test with 100 operations indicated that the feeder for the left and right compartments delivered a mean of 22.42 eggs (SD = 5.17) and 18.6 eggs (SD = 10.31) per reinforcer delivery, respectively. The same two feeders were used for all four pairs of fish. A C++ program, compiled with Visual Studio^TM 2015 on a Dell^TM laptop computer (Latitude. E5530 model), controlled all devices and recorded all experimental events.

Procedure

General features. The onset of sessions was signaled by covering the top of the aquarium with the Plexiglas lid described above and also by covering each side of aquarium walls with a white plastic partition that minimized visual distraction of fish. The motion of each fish was tracked throughout each session. The target response was a pair of fish’s approach to one another across the glass partition within the Euclidean distance of 1.5 cm (including 0.3 cm for the partition). The distance between the two fish was calculated with the following equation derived from the Pythagorean theorem:

Where .1 and .2 stand for the height, .1 and .2 for the width, and .1 and .2 for the depth of each fish’s location in the aquarium, respectively. In addition to the distance criterion, two additional criteria limited the registration of an approach response. First, the obtained distance was considered valid only when both fish were detected within a 0.5-s window (cf. Tan & Hackenberg, 2016). This criterion was in effect because the detection of fish sometimes was not continuous (see Kuroda, 2018, for details). The 0.5-s window thus ensured that the two fish were close in space to each other at the moment of response registration. Second, once an approach response was registered, the next response was not registered until the distance between the two fish was greater than 1.5 cm. The latter criterion precluded repeated registrations of response while the fish remained close to each other.

Each session lasted for 20 min, excluding food-delivery time. During food delivery, both feeders operated simultaneously and a white LED attached on each feeder flashed five times at 0.2-s intervals. Thereafter, the feeder remained inoperative until 5 s elapsed since the onset of reinforcement. The food reinforcer (shrimp eggs) fell onto the surface of the water approximately 1 cm away from the side wall of the aquarium for each fish whereas the target response was measured near the glass partition placed in the middle of the aquarium, thereby dissociating the target response from the goal tracking of the food. Sessions were conducted around the same time of the day, seven days a week.

Experimental conditions. For each pair of fish, the sequence of conditions was exposure to extinction (EXT), variable-time (VT), fixed-ratio (FR), and yokedVT contingencies. Table 1 shows the number of sessions conducted with each condition in effect for each pair of fish.

In the EXT condition, the feeders remained inoperative throughout sessions. This condition assessed the operant level of approach responding in the absence of food delivery. In the VT condition, the feeders operated independently of responding according to a VT 30-s schedule, which consisted of 10 intervals derived from Fleshler and Hoffman’s (1962) distribution. The intervals were selected randomly without replacement. The VT condition assessed the level of approach responding in the presence of food delivery when the delivery was not dependent on responding. This condition also served as magazine training. In the FR condition, every approach response led to food delivery according to an FR 1 schedule. This condition assessed the level of approach responding when food delivery was dependent on approach responses. Lastly, in the yoked-VT condition, food was delivered independently of responding according to a VT .-s schedule. The value of . was determined for each pair of fish in such a way that the rate of food delivery was yoked to the mean obtained rate in the last five sessions of the preceding FR condition. The yoked-VT schedule consisted of 10 intervals derived from the Fleshler and Hoffman distribution, which were selected randomly without replacement. Overall, the VT and yoked-VT conditions served as baselines before and after the FR condition. This sequence of conditions allowed for an assessment of the effect of response-reinforcer dependency on the target approach response while controlling effects associated with food delivery per se.

Results

The target response was a pair of fish approaching one another within the Euclidean distance of 1.5 cm. Figure 2 shows the frequency of target responses across sessions for each pair of fish. The target response generally remained low in the EXT condition for all four pairs except for a few occasions (e.g., Pair 1) and then increased after the transition to the VT condition for Pairs 2, 3, and 4. The transition to the FR condition further increased and maintained the target response for Pairs 1 and 2. For these two pairs, the response decreased in the yoked-VT condition relative to the FR condition. For Pairs 3 and 4, in contrast, the target response initially increased but was not maintained in the FR condition; thus, they did not proceed to a yoked-VT condition. Overall, two of the four pairs acquired the approach response as a result of the reinforcement contingency after controlling the rate of food delivery.

Figure 3 shows representative 3D trajectories of fish motion: The trajectories were for Pair 1 in the last session of each condition. The black lines in the trajectory represent the motion during the 5-s food-delivery time. In the EXT condition, the fish appear to have spent a large portion of time near the glass partition. Comparisons of the VT, FR, and yoked-VT conditions suggest that the fish were more likely to be near the water surface in the FR condition, especially, during food delivery. Results of the other pairs of fish in these conditions generally were similar to Pair 1 (data not shown). For a clearer analysis, Figures 4a (Pair 1), 4b (Pair 2), and 4c (Pairs 3 and 4) show the time spent at each 1-cm. segment of the aquarium in the last session of each condition. The grayscale on the right side of the graphs represents the time spent, ranging from gray to black with the latter indicating longer times. The analysis excluded the food-delivery time to equate total duration across the conditions. Moreover, the time of less than 1 s spent in a location is not displayed to minimize overlaps among the dots in the graph. Pairs 1, 3, and 4 spent a large portion of time near the glass partition in the EXT condition. In the VT, FR, and yoked-VT conditions, the location where the fish spent most time shifted to the water surface when food was delivered either dependent on or independently of responding. Thus, food delivery per se affected the distribution of behavior in the aquarium. Among these three conditions, however, the time spent near the water surface and near the glass partition was prominent in the FR condition for each pair of fish. Although the response-rate graph (Figure 3) suggests failures in response acquisition for the fish in Pairs 3 and 4, the time-spent graph (Figure 4a -4c) shows the fish in Pair 4 were more likely to be near the glass partition and near the water surface in the FR condition than the preceding VT condition. Thus, the reinforcement contingency actually may have been effective in decreasing the distance between these fish but not as close enough as required for response registration (i.e., the Euclidean distance of 1.5 cm).

Informal observations indicated that the fish occasionally tracked the motion of each other, swimming in the same direction like “schooling” near the glass partition. This form of behavior differed from the target approach response in that schooling was a more global description of fish motion rather than a discrete momentary response. Figures 5a-5c show a portion of the 3D trajectory in the last session of each condition for each pair of fish, which can be considered schooling. Inclusion criteria for the analysis were as follows: 1) Across successive detections, the fish in a pair had to remain close in space to each other, namely, within the distance of 1 cm on the .-axis, 2.3 cm on the .-axis (i.e., 1 cm away from each side of the glass partition for each fish), and 0.5 cm on the z-axis (which approximately was the body size of zebrafish on the dorsoventral axis); and 2) the distance for which the fish swam while keeping the physical proximity had to be greater than 1.5 cm. The value in the right top corner of each graph in Figures 5a-5c indicates the frequency of schooling. For all four pairs of fish, the frequency was higher in the VT, FR, and yoked-VT (when conducted) conditions than the EXT condition and this form of behavior occurred mostly near the water surface, suggesting an effect of food delivery. Moreover, the frequency was the highest in the FR condition for Pairs 1 and 2. This indicates that, for these two pairs of fish, schooling increased as a function of arranging the reinforcement contingency for the approach response. In contrast, the frequency of schooling was higher in the VT condition than in the FR condition for Pairs 3 and 4.

Discussion

In the present experiment, a reinforcement contingency was arranged in such a way that a pair of zebrafish could receive food by approaching one another across a glass partition. Two out of four pairs (Pairs 1 and 2) acquired the approach response. For these two pairs, response acquisition resulted from the reinforcement contingency after controlling the rate of food delivery. Further analyses revealed that, when the contingency was in effect, these fish more frequently showed schooling (i.e., swimming in the same direction) in the presence of the reinforcement contingency than in its absence. In contrast, the other two pairs of fish (Pairs 3 and 4) initially showed an increase in the approach response when the reinforcement contingency was introduced but approach responding did not maintain. Implications of the positive and negative results are discussed below.

To the present author’s knowledge, this was the first attempt to study zebrafish social behavior in the context of operant conditioning. Zebrafish naturally show some forms of social behavior, such as approaching conspecifics and shoaling (e.g., Stewart, Nguyen et al., 2014; Suriyampola et al., 2016). Similarly, the tendency to approach other zebrafish across a glass partition also was observed when food delivery was absent in the present experiment. Previous research showed that individual zebrafish learn to approach a discrete mechanical operandum (i.e., a sensor) when the approach response results in food delivery (Kuroda & Mizutani, 2018; Kuroda et al., 2017a,b; Manabe et al., 2013a,b). This has been replicated with a “virtual” operandum set at a corner of the aquarium using a real-time 3D tracking system (Kuroda, 2018). Applying the tracking system to a pair of zebrafish, the results of present experiment revealed that zebrafish also can learn to approach other zebrafish when the approach response leads to food delivery.

The present results seem to indicate that reinforcement contingencies can increase social behavior in zebrafish. The nature of the behavior of approaching conspecifics needs to be clarified, however. Approaching conspecifics appears, at least, to be a precursor to social behavior because many forms of social behavior (e.g., cooperation) generally occur when two or more individuals are in close physical proximity. Yet there remains a question as to whether the behavior of the zebrafish in this experiment indeed was social in a behavior-analytic sense. As noted in the Introduction, it is a common practice to define social behavior by its structure in biological research and by its function in behavioral research. In a sense, approaching conspecifics might be considered as social behavior that is defined both structurally and functionally. It was functional, on one hand, because the function of this behavior was to produce food. It was structural, on the other hand, because the presence of other zebrafish (structure) was responsible for the approach response appearing social. Whether approaching conspecifics is or is not fundamentally different from approaching nonliving objects being in motion, namely, sign tracking (Hearst & Jenkins, 1974), needs to be clarified.

Skinner (1953/2014) defined social behavior as “the behavior of two or more people with respect to one another or in concert with respect to a common environment” (p. 297). His definition implies that there are two types of social behavior. The first type is that the behavior mutually affects both co-actors. The second type is that the behavior of two or more individuals is under the control of a common contingency, but their behavior does not necessarily affect one another. Of these two, approaching conspecifics seems to belong to the first type of social behavior. Specifically, such approach can be considered as a form of approaching so-called social stimuli. If an individual’s behavior is affected by the social stimulus but not vice versa, then approaching conspecifics probably is not fundamentally different from sign tracking. In the present experiment, however, each zebrafish in a pair was a social stimulus for its partner and their behavior mutually affected one another. Therefore, approaching conspecifics seems to meet one of Skinner’s implied types of social behavior. Likewise, a mutual relation was present between the behavior of a pair of subjects in the case of pigeons pecking a corresponding pair of keys (Skinner, 1962), rats pressing levers contiguously in time (Tan & Hackenberg, 2016), and the schooling of zebrafish observed in the present experiment. Thus, these forms of behavior seem to belong to the same category of social behavior as approaching conspecifics. In contrast, the communication between Jack and Jill (Epstein et al., 1980) seems to belong to Skinner’s (1953/2014) second type of social behavior. Communication such as verbal behavior (Skinner, 1957/2014) requires a stimulus (e.g., vocabulary) having a common function among individuals, which implies the presence of a common contingency for them.

The acquisition of approaching conspecifics was unreliable in the present experiment. Unlike Pairs 1 and 2, Pairs 3 and 4 did not acquire the approach response. Perhaps, the present experimental arrangement was not optimal for training the approach response. The failure of response acquisition could be attributed to several variables related to the presence of the glass partition. First, the distance criterion for reinforcer delivery may have been too stringent for Pairs 3 and 4. The criterion was set at the Euclidean distance of 1.5 cm including 0.3 cm for the glass partition; thus, actually 1.2 cm. The center of the fish body, not its head part, was tracked (see Kuroda, 2018 for details) so that the fish had to be very close to one another in space to register a response. Therefore, this condition could be more disadvantageous for bigger fish than smaller fish. Their body sizes were not specifically measured, but loosening the distance criterion, depending on the body size of the fish, or differential reinforcement of successive approximations (i.e. shaping) with adjustments in the criterion could increase the likelihood of response acquisition. Alternatively, tracking the head, instead of the center part, of zebrafish also might increase the likelihood. Tracking a specific part of body is possible with a sophisticated technology in computer science such as deep learning (e.g., Mathis, Mamidanna, Cury, Abe, Murthy, Mathis, & Bethge, 2018). Thus, improvements in the present tracking system may lead to greater reliability in the acquisition of approaching conspecifics.

A second possible factor contributing to the failure in response acquisition was the use of the built-in IR projector. Providing infrared light generally is a recommended way for improving the precision of depth measurement. It may have been inappropriate in the present case, however. Specifically, the infrared light created glare on some parts of the aquarium resulting in “blind spots” in depth frames, especially, near the glass partition. When the system was unable to measure the depth, the fish was considered undetected (i.e., a ”miss”) in the present tracking system. Thus, some portion of approach responses may have gone unregistered and unreinforced, resulting in a lean schedule of reinforcement. After the completion of the present experiment, an informal test was conducted to assess the usefulness of an external IR projector while the built-in projector was turned off. Glare on the glass plates was reduced with the external IR projector placed next to a side wall of the aquarium. Thus, replacement of the built-in projector with an external one could increase the likelihood of response acquisition in future research.

Removing the glass partition from the aquarium is another possible route for improving the experimental setup. Placing a transparent partition between a pair of subjects has been a common method for studying social behavior in operant research (e.g., de Carvalho et al., 2018; Epstein, 1980; Łopuch & Popik, 2011; Skinner, 1962; Tan & Hackenberg, 2016) but social behavior also has been studied in the absence of such a partition in some of the studies (Łopuch & Popik, 2011; Tan & Hackenberg, 2016). The glass partition was used in the present experiment to aid the identification of each fish for tracking them individually. However, the partition precludes direct social contact among subjects, which may omit critical and dynamic elements of social interaction. For example, an individual’s behavior (e.g., chasing) can affect another’s behavior (e.g., running away), which in turn further reinforces the former’s behavior. Such a dynamic interaction may be a critical element for the establishment of social relation among the fish. Several different methods have been proposed for identifying zebrafish (see Delcourt, Ovidio, Denoël, Muller, Pendeville, Deneubourg, & Poncin, 2018, for a summary). One possible approach is implanting a microchip inside their body. The microchips currently implantable in zebrafish, however, are not suitable for behavioral research because reading the radio emitted from the microchip is limited to a distance of 5-12 mm. Another possible way is injecting chemicals into the zebrafish’s body for an artificial color. The chemical, however, can be toxic for the fish and the color fades with time (Bashey, 2004). Moreover, artificial colors per se can affect zebrafish social behavior (Ruberto, Clément, Spinello, Neri, Macri, & Porfiri, 2018). Perhaps a more promising approach is a sophisticated technique developed in the field of computer vision for multi-object tracking. Andriyenko, Schindler, and Roth (2012), for example, offered an algorithm for identifying each object by finding an optimized set of trajectories from all detected locations in a space. Such an identification method would make the presence of a glass partition unnecessary. Moreover, it may be possible to extend the scope of the experimental analysis of social behavior to a situation where a larger number of subjects interact with each other rather than only two subjects.

In summary, the present experiment demonstrated the acquisition of approaching conspecifics under a reinforcement contingency for two out of four pairs of zebrafish. The likelihood of the response acquisition might increase with the improvement in the experimental arrangement for studying the behavior. Approaching conspecifics is a rather elementary form of social behavior and is involved in many other forms of social behavior so that the present research can be considered as a starting point for the study of zebrafish social behavior. This species can serve as a model of social behavior across biological and behavioral research. For example, exposures to certain chemicals during the embryonic stage, such as ethanol (e.g., Buske & Gerlai, 2011a) and valproate (Zimmermann, Gaspary, Leite, De Paula, Cognato, & Bona, 2015), result in deficits in social interactions among zebrafish. Modifications in such genes as samdori 2 can affect social structures in a shoal of zebrafish (Choi et al., 2018; Chou et al., 2016). Behavior analysts might work together with researchers in biology, for example, to examine how reinforcement contingencies change the social structure of zebrafish altered by the manipulation of those biological variables. Together with the availability of technologies for environmental and biological manipulations, zebrafish are a promising animal model for studying how biological factors interact with environmental factors for determining not only individual behavior but also social behavior.

Acknowledgments

This research was approved by the Research Ethics Committee at Aichi Bunkyo University, where the experiment was conducted, and was supported by a research grant from the Japan Society for the Promotion of Science (Kakenhi #16K17366). The author thanks Chris Podlesnik at Auburn University for his helpful comments on a previous version of this manuscript.

References

Andriyenko, A., & Schindler, K., & Roth, S. (2012). Discrete-continuous optimization for multi-target tracking. IEEE Conference on Computer Vision and Pattern Recognition. doi: 10.1109/CVPR.2012.6247893

Aoki, R, Tsuboi, T, & Okamoto, H. (2015). Y-maze avoidance: an automated and rapid associative learning paradigm in zebrafish. Neuroscience Research, 91, 6972. doi: 10.1016/j.neures.2014.10.012

Bashey, F. (2004). A comparison of the suitability of alizarin red S and calcein for inducing a nonlethally detectable mark in juvenile guppies. Transactions of the American Fisheries Society, 133, 1516–1523. https://doi.org/10.1577/T03-073.1

Bass, S. L. S., & Gerlai, R. (2008). Zebrafish (Danio rerio) responds differentially to stimulus fish: The effects of sympatric and allopatric predators and harmless fish. Behavioural Brain Research, 186, 107-117. https://doi.org/10.1016/j. bbr.2007.07.037

Buske, C., & Gerlai, R. (2011a). Early embryonic ethanol exposure impairs shoaling and the dopaminergic and serotoninergic systems in adult zebrafish. Neurotoxicology and Teratology, 33:698–707. https://doi.org/10.1016/j.ntt.2011.05.009

Buske, C., & Gerlai, R. (2011b). Shoaling develops with age in Zebrafish (Danio rerio). Progress in Neuropsychopharmacology & Biological Psychiatry, 35:1409–1415. https://doi.org/10.1016/j.pnpbp.2010.09.003

Buske, C., & Gerlai, R. (2012). Maturation of shoaling behavior is accompanied by changes in the dopaminergic and serotoninergic systems in zebrafish. Developmental psychobiology, 54:28–35. https://doi.org/10.1002/dev.20571

Chalmeau, R., & Gallo, A. (1995). Cooperation in primates: Critical analysis of behavioural criteria. Behavioural processes, 35, 101–111. https://doi. org10.1016/0376-6357(95)00049-6

Choi, J. H., Jeong, Y. M., Kim, S., Lee, B., Ariyasiri, K., Kim, H. T., et al. (2018). Targeted knockout of a chemokine-like gene increases anxiety and fear responses. Proceedings of the National Academy of Sciences of the United States of America, 115, E1041-E1050. https://doi.org/10.1073/pnas.1707663115

Chou, M. Y., Amo, R., Kinoshita, M., Cherng, B. W., Shimazaki, H., Agetsuma, M., et al. (2016). Social conflict resolution regulated by two dorsal habenular subregions in zebrafish. Science, 352, 87-90. DOI: 10.1126/science.aac9508

Colwill, R. M., Raymond, M. P., Ferreira, L., & Escudero, H. (2005). Visual discrimination learning in zebrafish (Danio rerio). Behavioural Processes, 70, 19-31. https://doi.org/10.1016/j.beproc.2005.03.001

Crawford, M. P. (1937). The cooperative solving of problems by young chimpanzees. Baltimore, MD: Johns Hopkins Press.

de Carvalho, L. C., dos Santos, L., Regaço, A., Barbosa, T. B., da Silva, R. F., de Souza, D. D. G., & Sandaker, I. (2018). Cooperative responding in rats maintained by fixed- and variable-ratio schedules. Journal of the Experimental Analysis of Behavior, 110, 105-126. https://doi.org/10.1002/jeab.457

Delcourt, J., Ovidio, M., Denoël, M., Muller, M., Pendeville-Samain, H., Deneubourg, J.-L., & Poncin, P. (2018). Individual identification and marking techniques for zebrafish. Reviews in Fish Biology and Fisheries, 28, 839-864. https:// doi.org/10.1007/s11160-018-9537-y

Epstein, R., Lanza, R. P., & Skinner, B. F. (1980). Symbolic communication between two pigeons. Science, 207, 543-545. DOI: 10.1126/science.207.4430.543

Fleshler, M., & Hoffman, H. S. (1962). A progression for generating variable-interval schedules. Journal of the Experimental Analysis of Behavior, 5, 529-530. https://doi.org/10.1901/jeab.1962.5-529

Gerlai, R. (2015). Zebrafish phenomics: behavioral screens and phenotyping of mutagenized fish. Current Opinion in Behavioral Sciences, 2, 21-27. https://doi.org/10.1016/j.cobeha.2014.07.007

Gerlai, R., Fernandes,Y., & Pereira, T. (2009). Zebrafish (Danio rerio) responds to the animated image of a predator: Towards the development of an automated aversive task. Behavioural Brain Research, 201, 318-324. https://doi.org/10.1016/j.bbr.2009.03.003

Gerlai R., Lahav, M., Guo, S., & Rosenthal, A. (2000). Drinks like a fish: Zebra fish (Danio rerio) as a behavior genetic model to study alcohol effects. Pharmacology Biochemistry and Behavior, 67, 773-782. https://doi.org/10.1016/S0091-3057(00)00422-6

Gould, G. G. (2011). Zebrafish Biogenic amine transporters and behavior in novel environments: Targets of reuptake inhibitors and pesticide action as tools for neurotoxicology research. In A. V. Kalueff & J. M. Cachat (Eds.), Zebrafish models in neurobehavioral research(pp. 181-210). New York: Humana Press.

Hearst, E., & Jenkins, H. M. (1974). Sign-tracking : The stimulus-reinforcer relation and directed action. Madison, WI: Psychonomic Society.

Howe, K., Clark, M. D., Torroja, C. F., Torrance, J., Berthelot, C., Muffato, M. (2013). The zebrafish reference genome sequence and its relationship to the human genome. Nature, 496, 498-503. http://dx.doi.org/10.1038/nature12111

Kalueff, A. V., Stewart, A. M., & Gerlai, R. (2014). Zebrafish as an emerging model for studying complex brain disorders. Trends in Pharmacological Sciences, 35, 6375. https://doi.org/10.1016/j.tips.2013.12.002

Kuroda, T. (2018). A system for the real-time tracking of operant behavior as an application of 3D camera. Journal of the Experimental Analysis of Behavior, 110, 522-544. https://doi.org/10.1002/jeab.471

Kuroda, T., & Mizutani, Y. (2018). Response acquisition by zebrafish (Danio rerio) with delayed reinforcement. Journal of the Experimental Analysis of Behavior, 109, 520-532. doi: 10.1002/jeab.324

Kuroda, T., Mizutani, Y., Cançado, C. R. X., & Podlesnik, C. A. (2017a). Reversal learning and resurgence of operant behavior in zebrafish (Danio rerio). Behavioural Processes, 142, 79-83. doi: 10.1016/j.beproc.2017.06.004

Kuroda, T., Mizutani, Y., Cançado, C. R. X., & Podlesnik, C. A. (2017b). Operant models of relapse in zebrafish (Danio rerio): Resurgence, renewal, and reinstatement. Behavioural Brain Research, 335, 215-222. doi: 10.1016/j.bbr.2017.08.023

Kuroda, T., Mizutani, Y., Cançado, C. R. X., & Podlesnik, C. A. (2019). Predator videos and electric shock function as punishers for zebrafish (Danio rerio). Journal of the Experimental Analysis of Behavior, 111, 116-129. https://doi.org/10.1002/jeab.494

Łopuch, S., & Popik, P. (2011). Cooperative behavior of laboratory rats (Rattus norvegicus) in an instrumental task. Journal of Comparative Psychology, 125, 250253. DOI: 10.1037/a0021532

Mahabir, S., Chatterjee, D., Buske, C., & Gerlai, R. (2013). Maturation of shoaling in two zebrafish strains: A behavioral and neurochemical analysis. Behavioural Brain Research, 247, 1-8. https://doi.org/10.1016/j.bbr.2013.03.013

Manabe, K., Dooling, R. J., & Takaku, S. (2013a). An automated device for appetitive conditioning in zebrafish (Danio rerio). Zebrafish, 10, 518-523. https://doi.org/10.1089/zeb.2012.0776

Manabe, K., Dooling, R. J., & Takaku, S. (2013b). Differential reinforcement of an approach response in zebrafish (Danio rerio). Behavioural Processes, 98, 106-111. https://doi.org/10.1016/j.beproc.2013.05.013

Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018). DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience, 21, 1281-1289. DOI: 10.1038/s41593-018-0209-y

Morin, C., de Souza Silva, M. A., Müller, C. P., Hardigan, P., & Spieler, R. E. (2013). Active avoidance learning in zebrafish (Danio rerio) — The role of sensory modality and inter-stimulus interval. Behavioural Brain Research, 248, 141-143. https://doi.org/10.1016/j.bbr.2013.04.009

Qin, M., Wong, A., Seguin, D., & Gerlai, R. (2014). Induction of social behavior in zebrafish: Live versus computer animated fish as stimuli. Zebrafish, 11, 185-197. https://doi.org/10.1089/zeb.2013.0969

Ruberto, T., Clément, R. J. G., Spinello, C., Neri, D., Macrì, S., & Porfiri, M. (2018). The tagging procedure of visible implant elastomers influences zebrafish individual and social behavior. Zebrafish, 15, 433-444. https://doi.org/10.1089/zeb.2018.1616

Skinner, B. F. (1962). Two “synthetic social relations.” Journal of the Experimental Analysis of Behavior, 5, 531-533. doi: 10.1901/jeab.1962.5-531

Skinner, B. F. (2014). Science and human behavior. Cambridge, MA: The B. F. Skinner Foundation. (Reprinted from Science and human behavior, 1953, New York: Macmillan)

Skinner, B. F. (2014) Verbal behavior. Cambridge, MA: The B. F. Skinner Foundation. (Reprinted from Verbal behavior, 1957, Upper Saddle River, NJ: Prentice-Hall)

Stewart, A. M., Braubach, O., Spitsbergen, J., Gerlai, R., & Kalueff, A. V. (2014). Zebrafish models for translational neuroscience research: from tank to bedside. Trends in Neurosciences, 37, 264-278. https://doi.org/10.1016/j.tins.2014.02.011

Stewart, A. M., Nguyen, M., Wong, K., Poudel, M. K., & Kalueff, A. V. (2014). Developing zebrafish models of autism spectrum disorder (ASD). Progress in Neuro-Psychopharmacology and Biological Psychiatry, 50, 27-36. https://doi.org/10.1016/j.pnpbp.2013.11.014

Suriyampola, P. S., Shelton, D. S., Shukla, R., Roy, T., Bhat, A., & Martins, E. P. (2016). Zebrafish social behavior in the wild. Zebrafish, 13, 1-8. http://doi.org/10.1089/zeb.2015.1159

Tan, L., & Hackenberg, T. D. (2016). Functional analysis of mutual behavior in laboratory rats (Rattus norvegicus). Journal of Comparative Psychology, 130, 13-23. DOI: 10.1037/com0000015

Zimmermann, F. F., Gaspary, K. V., Leite, C. E., De Paula Cognato, G., & Bonan, C. D. (2015). Embryological exposure to valproic acid induces social interaction deficits in zebrafish (Danio rerio): A developmental behavior analysis. Neurotoxicology and Teratology, 52, 36-41. https://doi.org/10.1016/j.ntt.2015.10.002