Select Page

Using Zebra Finch data and deep learning classification to identify individual bird calls from audio recordings

Ines Moran, PhD Candidate, Yen Yi Loo, PhD Candidate, Dr Kristal Cain, Senior Lecturer, Biological Sciences; Nick Young, Senior Solutions Specialist, Centre for eResearch

  1. Home
    2.  • 
  2. Project
    3.  • Using Zebra Finch data and deep learning classification to identify individual bird calls from audio recordings

Introduction

The Zebra Finch is widely used by researchers and teachers in many scientific disciplines for investigations ranging from anatomy and physiology to behavioural development and evolutionary ecology1. Zebra Finches are highly vocal social birds and apart from male song, wild Zebra Finches use up to eleven distinct call types1, a repertoire that can also be observed in captive birds 2-5.

Our study used Zebra Finch calls 6 to develop and train a deep learning classifier to identify individual bird calls from audio recordings. The next step of our project is to test our classifier on recordings of rifleman, New Zealand’s smallest birds. We will deploy recording units engineered at the University of Auckland to record both the identity of individual rifleman as well as the vocalizations of individual rifleman before entering their nests. We will then test whether our classifier can accurately classify rifleman individual calls. If our classifier succeeds at classifying individual rifleman calls, this will support that rifleman individual calls are unique enough to be distinguished by our classifier. We will then use this classifier on hundreds of hours of rifleman recordings to classify and identify the calls of individuals.

 

 

The objective and hypothesis

Our objective is to determine whether rifleman nest calls can be individually classified using a deep learning classifier. We aim to understand how rifleman vocally recognize each other. We hypothesized that if individual vocal recognition is accurately performed by deep learning algorithms, then similarly, rifleman should be using the unique call properties used by deep learning algorithms to differentiate between one another.

Workflow method

We used wave data file collected from previous study6 of Zebra Finch by Pietro B. D’Amelio.  We applied the following workflow method:

  1. Load the wave files. Split dataset into train and test. The test datasets contain one row for each combination of bird ID and call type. (see figure 1)
  2. Separated out the metadata, and render some spectrograms (see examples of figure 2a and 2b)
  3. Train a random forest classifier based on Mel-frequency Cepstral Coefficients (MFCC) (see table 1)
  4. Through deep learning classification with the R interface to Keras based on MFCC. (see figure 3)

 

 

Figure 1: Split dataset into train and test. The test datasets contain one row for each combination of bird ID and call type.

Figure 2a (left) and figure 2b. (right) are examples of audio recording of  individual female Zebra Finch that their metadata are separated out and some spectrograms are rendered.  

Figure 3. Train a random forest classifier based on Mel-frequency Cepstral Coefficients (MFCC)
Figure 4. Deep learning classification with the R interface to Keras based on MFCC.

Conclusion

Our analyses shown: 1) Each call type seems to have a unique spectrogram. (refer to examples of figure 2a and 2b), 2) Accuracy across whole dataset is 68.91%, 3) Random forest model through deep learning classification with the R interface to Keras based on Mel-frequency Cepstral Coefficients (MFCC) shown 52 out of 63 wavs in the test datasets correctly identified with accuracy of 82.54%, 4) Kera model shown 47 out of 63 wavs in test dataset correctly identified with accuracy of 74.6%, 5) Hence, random forest method did better.

Next steps­­

The deep learning classifier that we trained with Zebra Finch’s audio data has successfully identified individual bird vocalisations. We are now conducting field research to collect rifleman birds’ vocal data for further analysis in early 2020.

Reference:

  1. Zann, R. A. The zebra finch: a synthesis of field and laboratory studies (Oxford Univ. Press, Oxford, 1996).
  2. Elie, J. E. & Theunissen, F. E. Meaning in the avian auditory cortex: neural representation of communication calls. The European journal of neuroscience 41, 546–67 (2015).
  3. Ter Maat, A., Trost, L., Sagunsky, H., Seltmann, S. & Gahr, M. Zebra finch mates use their forebrain song system in unlearned call communication. PloS one 9, e109334 (2014).
  4. Gill, L. F., Goymann, W., Ter Maat, A. & Gahr, M. Patterns of call communication between group-housed zebra finches change during the breeding cycle. eLife 4, e07770 (2015).
  5. Elie, J. E. & Theunissen, F. E. The vocal repertoire of the domesticated zebra finch: a data-driven approach to decipher the information-bearing acoustic features of communication signals. Animal Cognition 19, 285–315 (2016).
  6. Pietro B. D’Amelio, Milena Klumb, Mauricio N. Adreani, Manfred L. Gahr & Andries ter Maat, “Individual recognition of opposite sex vocalizations in the zebra finch”. July 2017. nature.com/scientificreports