Artificial Intelligence for animal sounds is getting more intelligent

Most researchers at Naturalis Biodiversity Center focus on a single group of species, but Burooj Ghani does a bit of everything. Frogs, bats, birds, insects: as long as it makes a sound, it’s fair game. Ghani is a computer scientist and he works on artificial intelligence that recognizes animals based on their sounds.

This has been possible with birds for a while now: for instance the animal sound website Xeno-canto that recognizes European bird species, thanks to an AI that Ghani developed. Currently, he's mainly working on insects. They present their own unique challenges, he explains: "Species recognition based on sound has borrowed a lot from the existing experience in recognizing general audio and human speech, but humans only speak at pitches that other humans can hear. Many insects make sounds that are partly within the human audible spectrum, and partly higher – ultrasonic. So you need special recording equipment, and the way the files are stored can lead to data loss with very high-frequency sounds."

Making a little go a long way

Another problem, not unique to insects: there are many species for which only a handful of sound recordings exist. So-called transfer learning helps a bit: "You first teach your AI to generally recognize sounds using a database of sounds from YouTube full of rappers and barking dogs and so on. After that, you can fine-tune the AI on your ten cicada recordings. But a lack of data remains a problem."

Om dit object in te laden dien je marketing-cookies te accepteren.

Midwife Toad (Alytes obstetricans) amplexus (Source: Herpsafari)

Ghani has a plan to tackle this problem more intelligently. Just like human listeners, sound recognition software could use context. Where are you, what time of year is it, what time of day is it, and so on? Especially with the noise of the wind in the background, it might not be so easy to hear whether a recorded chirp is from a coastal bird or a midwife toad. But, if the AI knows you were on the beach, the choice is easier. "It will basically check if its identification matches everything it knows about the species. That means species recognition software will become more like Large Language Models such as ChatGPT and Gemini. That's ambitious, but it's the way to go."

Weather stations for biodiversity

The ambitions for species identification based on sound are sky-high. In national and international collaborations, significant progress is being made on systems with microphones that you leave somewhere and which then autonomously monitor nature, whether or not as part of a 'biological weather station' that collects even more data. Better information about which species are where is crucial for nature policy and protection, especially in a time when many species are struggling.

However, Ghani emphasizes, "Volunteers remain indispensable." The high-throughput biodiversity data processing pipelines, bird-recognizing artificial intelligences in your smartphone, and real-time monitoring of nature are only possible thanks to thousands of people worldwide who went into nature with a recording device. "The involvement of amateur experts is also essential for validation – checking whether the AI assigns the correct species name to a sound and training it accordingly. In that sense, computer science remains human work."

More information

• AI sound recognition for biodiversity monitoring is part of the ARISE and MAMBO projects in which Naturalis is involved.

Text: Naturalis Biodiversity Center
Images: Naturalis Biodiversity Center; Getty Images; Herpsafari