The instrument
A general model of the world's birds, cut down to one archipelago
Google DeepMind's Perch 2.0 knows 14,795 classes of sound. New Zealand does not need them. The model was opened, inspected, and modified - four tensors, kept bit-for-bit at the indices that survive - leaving the birds you can actually hear here, and twelve classes of noise so the app can say what is drowning them out.
The slice is not an approximation. At the kept indices the reduced model's logits are identical to the original's; converting to 16-bit for the phone costs an embedding cosine of 0.999928 and no change at all in which bird comes first.
Nothing is filtered. Wind, rain, traffic, aircraft, dogs and voices are detected, not removed - Perch was trained through a plain log-mel frontend on noisy field recordings, so cleaning the audio moves it off-distribution and the artefacts of denoising look like faint birds.
The slice
The whole bar is Perch 2.0 - every class it knows. 1.45% of it is New Zealand.
From 491 taxa in DOC's threat classification, after merging subspecies Perch cannot separate and dropping the extinct and the vagrant. 52 families.
Twelve FSD50K classes collapsing to 9 plain words: wind, rain, traffic, voices, engine, aircraft, thunder, running water, a dog.
Shipped anyway, marked unhearable. Silence about a bird must mean no class for it, never no bird called.
- On disk
- 27.4megabytes, down from 413. Runs on the phone; nothing is sent anywhere.
- Te reo names
- 161159 of them from the OSNZ Checklist, 2022 - the authority, not a guess.
The register of standing
What the Department of Conservation says about the birds this thing can hear
Threat classification · 190 birds
- Of concern - declining or worse
- All other categories
59 of 190 are declining or worse. Hearing a kōkako must not look like hearing a blackbird, so the threat status travels with every detection and the interface gives it weight.
The categories are ordinal and the code treats them that way: they can be compared, never averaged. There is deliberately no numeric value on them to tempt an arithmetic mean of a bird's peril.
A further 25 species report no single status. Perch cannot separate the subspecies DOC assessed differently - one class, two verdicts - so the app returns nothing rather than picking the more alarming of the two, and the interface describes the range instead. The first implementation of this got it wrong in an instructive way, and the decision record keeps the wrong version on the page.
The ledger of refusals
The most important page in the book
Perch's own model card says the species logits are uncalibrated and possibly unreliable for rare species, and recommends tuning thresholds on your own data. So 7,489 recordings were scored, and a threshold was fitted per species by cross-validation. 123 species earned one. 90 did not, and the app will not name them.
- Fitted
- 123Median precision 0.80, fitted on a median of 20 clips.
- Declined
- 9060 for want of recordings; 30 because the model is measurably wrong.
A refusal is not a bug being hidden. It is the recorded result of a species whose false positives were counted and found unacceptable - and in most cases the recordings can say precisely which other bird the model is hearing instead.
Where the recordings came from
Mostly Creative Commons; NZ Birds
Online's are used for non-commercial research and never redistributed. A bare
except: continue once hid 1,814 of these - every AAC file -
and still produced a confident-looking table. Decode failures are now
counted by reason.
Read the top of that list carefully. Perch has essentially one concept for the brown kiwi: it calls tokoeka “kiwi-nui” at about the same rate it calls kiwi-nui “kiwi-nui”. Kiwi pukupuku, just as congeneric, separates cleanly - so this is a specific failure, not a general one. Tūī and korimako, the two loudest birds in the forest, take each other down together. All 30 are listed in the appendix.