For this project we have used free software, to enable reproduction of our processes by others.
Hence we have used the free BatClassify to perform the Auto ID of recordings made with Audiomoth. This “open source and scientifically robust software” was built as part of the study “Woodland bat survey methods” undertaken for for DEFRA in 2011-13. BatClassify was written by Chris Scott who is part of the “Altringham Lab” research group at Leeds University. The project focused on the Myotis species: Alcathoe, Bechstein’s, Brandt’s, Natterer’s and whiskered bats, along with brown long-eared bats and barbastelles
We do not believe that BatClassify has been updated since this project, and hence some consider it as outdated.
Summary
BatClassify is surprisingly accurate at AutoIding even very faint or short calls of the following species:
- Common pipistrelle
- Soprano pipistrelle
There were few calls of the following, but BatClassify identified them correctly. It also produced a small number of false positives, that were easy to manually validate.
- Lesser horseshoe
- Greater horseshoe
BatClassify is very inaccurate at AutoIding the large bats, with several times more false positive IDs than correct IDs for the “NSL” species; Noctule, Serotine, Leisler’s.
As the survey site is not often visited by Myotis or long-eared species, the effectiveness of BatClassify on these was not assessed.
Testing method
The testing of BatClassify was done using the results of the sample rate testing. In this a bank of 8 AudioMoth were set up with three different firmwares, and various sample rates, but all with 50kHz HiPass filter and high setting for gain.
The results of AutoID of recordings from sunset to midnight, were then compared to manual identification for all the same recordings using the spectrograms viewed in Audacity.
This manual verification was done on 5 sets of recordings, each of up to 900, ten second recordings.
Results
The results for number of bats detected by AutoID using Batclassify are in the table below:
| AudioMoth Configuration | Big bat (NSL) AutoIDed | PipPip AutoIDed | PipPyg AutoIDed | RhiFer AutoIDed | RhiHip AutoIDed |
|---|---|---|---|---|---|
| Firmware 1.0.9 (triggering) – 192kHz sample rate | 38 | 29 | 27 | 0 | 0 |
| Firmware 1.4.3 (Amp.Th.) – 192kHz sample rate | 92 | 65 | 124 | 0 | 0 |
| Firmware 1.4.3 (Amp.Th.) – 256kHz sample rate | 90 | 60 | 95 | 1 | 0 |
| Firmware 1.0.10 (triggering) – 394kHz sample rate | 19 | 13 | 16 | 0 | 0 |
| Firmware 1.4.3 (Amp.Th.) – 394kHz sample rate | 67 | 43 | 58 | 1 | 4 |
The number of bats manually identified in the same recordings are in the table below.
- Blue numbers are were the manual validation is identical to the Auto ID
- Green numbers are were the AutoID is close to the manual validation.
- Red numbers denote a wide difference.
| AudioMoth Configuration | Big bat (NSL) Manually IDed | PipPip Manually IDed | PipPyg Manually IDed | RhiFer Manually IDed | RhiHip Manually IDed |
|---|---|---|---|---|---|
| Firmware 1.0.9 (triggering) – 192kHz sample rate | 15 | 30 | 25 | 0 | 0 |
| Firmware 1.4.3 (Amp.Th.) – 192kHz sample rate | 17 | 67 | 100 | 0 | 0 |
| Firmware 1.4.3 (Amp.Th.) – 256kHz sample rate | 18 | 60 | 95 | 0 | 0 |
| Firmware 1.0.10 (triggering) – 394kHz sample rate | 8 | 12 | 14 | 0 | 0 |
| Firmware 1.4.3 (Amp.Th.) – 394kHz sample rate | 14 | 43 | 58 | 0 | 2 |
Full details of all configuration testing results are in the data tab of this spreadsheet.
Discussion of Results
The recordings AutoIded with a false positive for “big bats” were looked at carefully manually. Most of these recordings contained Cricket calls, some Pip social calls, and occasionally the creak of the nearby back door. Hence BatClassify results are best ignore for the “NSL”, big bats”.
In contrast BatClassify is superb at AutoIding pipistrelle calls. Even a single call will be Ided correctly.
For the horseshoe bats, the false AutoId’s were mostly due to the internal noise of some of the AudioMoths, some of which produce constant frequency noise at near the same frequencies as horseshoes.
Further Work
- Another survey site could be chosen to assess the effectiveness of BatClassify on Myotis amd long-eared species.
- A full comparison of BatClassify against commercial industry standard bat AutoId programs such as Kaleidoscope Pro and SonoBat
- In 2019 we did a brief comparison of Kaleidoscope Pro (KPro) vs. BatClassify on AudioMoth recordings, and found KPro (at the default settings) IDed far fewer bats than BatClassify. This may be due to KPro needing settings needing to be tweaked, or due to it been developed to work best on less noisy recordings from more expensive equipment. This needs more work.