If two trajectories meet both time constraint and space constraint at the same time, then they maybe belong to the trajectory fragments of one trajectory. Next, we conduct feature matching on the observation variables represented by the start tag and end tag. If the matching is successful, connect the two trajectories. Figure 7 shows an example of trajectory linking. We have conducted experiments to evaluate the performance of the proposed method in tracking multiple swimming fish.
The experimental apparatus is shown in Figure 1 a. Fish behavior is relatively quiet and several motion modes regular acceleration and deceleration, glide-and-burst, rapid and explosive motion are present in the experimental data. In order to evaluate the proposed method more challengingly, we leave the noise at the bottom of fish tank and the disturbance of the suspended matter in the water with no special processing. The computing facility includes a desktop computer with Intel I5 2.
In order to test the tracking performance of different fish schools, we choose zebrafish Danio rerio with different densities in 3 groups: A1 10 fish , A2 20 fish , A3 40 fish , the video for each fish group contains frames. The parameter settings are shown in Table 1. We first carry out target detection for each frame image in the video.
In order to quantify the performance of the proposed detection method, the precision and recall ratios that are widely adopted for evaluating object detection methods are used in the experiment. They are defined as follows: 21 22 where true positive is the total number of correctly detected regions in all frames; false negative is the total number of missed regions; false positive is the total number of wrongly detected regions. In addition, in order to better evaluate the detection performance of the proposed method in the case of fish occlusion, we set up three additional evaluation criteria: OR occlusion ratio , ODR occlusion detection ratio and DT detection time.
The detection results are as shown in Table 2.
From the results we can see that with the increase of fish school density, the occlusion ratio rises and the detection performance gradually declines. The fish school occlusion makes the head region invisible and then leads to the detection errors. In spite of this, the Precision ratio of the three groups of videos are maintained at over 0. Furthermore, occlusion detection ratio shows that, although fish occlusion brings some difficulties, the proposed detection method still demonstrates strong detection ability under occlusion.
Because most occlusions are caused by fish body or tail rather than head, our method is then able to detect most occluded targets. Finally, seen from the detection time, the detection time in three groups are all within 1. Figure 8 shows some detection results. After detecting and locating each fish, we then track them throughout the video to obtain their motion trajectories.
To evaluate the proposed tracking method quantitatively, we associate the obtained trajectories with ground truth trajectories G by using the approach proposed by [24]. Make to indicate the frames where and overlap, then the distance between two trajectories is defined as: 26 where x t represents the target location on t. The above equation indicates the average distance between the obtained target position and ground truth target position over all frames. The cost of an association is defined as the sum of distances between the obtained trajectories and the associated ground truth trajectories.
Define two evaluation metrics to evaluate the tracking method. It indicates the average ratio of one ground truth trajectory length covered by the obtained trajectories. The smaller the value is, the little the accuracy is. The second evaluation metric TFF trajectory fragmentation factor can be defined as:. Larger value means worse effect of the method in tracking the targets. The tracking method consists of three parts: motion prediction, data association and trajectory linking. In order to better evaluate the proposed method, we use five methods with different schemes and compare them, and compared methods are shown in Table 3.
Apple’s ARKit 3 Introduces Human Occlusion, Body Tracking, And Motion Capture - VRScout
Figure 9 shows the comparison results of different methods. Seen from the comparison of motion prediction, the adding of motion prediction performs much better than the single use of feature matching, especially in that, the more fish schools are, the more searching space for feature matching is needed, which will gradually lower the probability of successful matching and then lead to more tracking errors. By adding motion prediction into feature matching, the matching calculation drops and accuracy increases, thus tracking results are significantly improved. In actual tracking, the motion state of fish school is quite complex, with frequent occlusion.
- free public records directory broward florida.
- deschutes county oregon tax assessor records.
- Motion capture.
The motion information itself can hardly complete accurate association, while feature matching takes full advantage of the fish school's appearance information and keeps the consistency of targets in the complex motion. With the increase of fish schools, the tracking performance of all three methods declines because of more occlusion. However, the comparison finds that, the tracking performance of our tracking method declines much more slightly than the other two, showing that the proposed tracking method has strong robustness in multi-target tracking.
Seen from the comparison of trajectory linking, the TCF and TFF using trajectory linking method is superior to the unconnected, which indicates that the gained trajectories becomes more intact after trajectory linking.
- vehicle identification number harley davidson.
- first right of refusal texas divorce.
- how to find my machine id.
- delaware county ohio courthouse marriages.
- canadian ontario long form birth certificate.
- alexander christie mary walters marriage illinois.
In addition, with the increase of fish schools, the number of gained trajectories after trajectory linking also increases, with a more obvious effect in high-density population than low-density population, indicating that the proposed method can better deal with trajectory fragmentation problem caused by occlusion. Acquired trajectories using the proposed method in different groups are shown in Figure As fish density increases, tracking performance of all five methods falls. In comparison, the proposed method offers highest TCF values and lowest TFF values, indicating its performance is the best among the compared methods.
Left column: trajectory acquisition results with the time axis.
3D Pose Detection of Closely Interactive Humans Using Multi-View Cameras
Right column: trajectory acquisition results without the time axis. In experimenting, we find out that for fish with rapid transition of swimming mode, Kalman filter will likely fail to predict a reasonable new state. Then we solve this problem by using a compensation window and trajectory linking method. In addition, we have performed preliminary experiments on images of golden shiner, paracherirodom innesi, tadpole and sperm, and results show that the proposed method can also detect and track the regions of their heads.
The performance of the proposed method is closely related to the occlusion ratio. When the fish head region is occluded by other fish, the coordinate data of the target will be lost.
Navigation menu
The longer the occlusion time, the longer the coordinate data are lost. When the similarity between several matching regions is very high, feature matching may fail, which will lead to identity switch. The higher the density of fish group, the higher the probability of the occurrence of this situation.
Conversely, if fish swim in polarized schools, it will cause severe occlusions, which will significantly increase tracking difficulties. The occlusion problem is the most difficult problem in multi-target tracking. Although we have tried to overcome it, the detection errors and tracking errors caused by occlusion cannot be completely avoided. This paper proposes an effective method for detecting and tracking multiple fish swimming in shallow water with frequent occlusion. Our contributions include a novel method for detecting multiple fish with possible occlusions based on robust image features around the head region.
The method integrates the local extremum and ellipse fitting to locate the fish head region and better deal with the difficulties in fish school detection caused by factors such as variable appearances, frequent occlusion and small discrimination of texture region. Our second contribution is an effective method for first-pass tracking that combines Kalman filtering with feature matching, taking full advantage of the motion and appearance information of fish school to better cope with the tracking in complex motions.
Our third contribution is a robust trajectory linking method as the second-pass of the tracking process in order to deal with frequent occlusion among fish. We have evaluated the proposed method on zebrafish schools of various densities in laboratory environment, and the results show its effectiveness and accuracy.
3D Pose Detection of Closely Interactive Humans Using Multi-View Cameras
Developed the software used in the experiments: ZMQ. Browse Subject Areas? Click through the PLOS taxonomy to find articles in your field. Abstract Due to its universality, swarm behavior in nature attracts much attention of scientists from many fields. Introduction There has been growing research interest in animal collective behavior due to its high scientific values and a wide range of potential applications [1] — [3].
Download: PPT. The Proposed Detection and Tracking Method From Figure 1 b , we observe that the fish appears in a top view image as consisting of two parts: a rigid anterior part and a deformable posterior part that may swing to propel it. The DoH of the matrix can be expressed as: 2 Then the result in the blob detection is the extreme point of DoH responses for the position space and scale space: 3 Since the head region gray value is less than the background region, we retain only the minimum extreme point. Figure 4.
Candidate constraints based on width, contrast and angle. Figure 5. The final feature matching result is defined as follows: 18 Figure 6 shows the process of feature matching. To solve this problem, we propose the following approach for trajectory handling on the basis of [23] : If a state variable of the associated observation variable is found, update according to equation 12 , and mark the state variable effective. If no state variable of the associated observation variable is found, associate with a virtual observation variable, update according to and mark the state variable ineffective.
If no observation variable on the trajectory is associated in T 1 consecutive frames, then the target probably keeps still, mark the trajectory incomplete and record the time et and position ep of the observation variable of the last effective state as the end tag of the trajectory.
If no observation variable of the associated state variable is found, we initialize the tracking and record the time st and position sp of the observation variable as the start tag of a new trajectory. The following tracking will see the two situations: A. Define the constraint as below: 19 The above equation indicates that, if the initial time of trajectory is later than the end time of trajectory , and time difference is less than T 2 , then the two trajectories meet time constraint.
Experiments and Discussions We have conducted experiments to evaluate the performance of the proposed method in tracking multiple swimming fish. Figure 8. The second evaluation metric TFF trajectory fragmentation factor can be defined as: 28 It describes the average number of gained trajectories used to match one ground truth trajectory. Figure 9. Performance of compared methods on two evaluation metrics. Figure Tracking results on different groups with Conclusion This paper proposes an effective method for detecting and tracking multiple fish swimming in shallow water with frequent occlusion.
Supporting Information.
Navigation menu
Movie S1. Tracking result's demo video of 40 fish. References 1. Science : — View Article Google Scholar 2. Nature Communications 4: View Article Google Scholar 3. PNAS 13 : — View Article Google Scholar 4.