Research Projects

Coloration is a complex, multidimensional phenotype that is difficult to compare consistently across large numbers of samples. Its variation can reflect shared ancestry, convergence, and environmental selection. Image embeddings provide a scalable way to quantify this variation: a vision model converts each specimen photograph into a numerical representation. These embeddings place visually similar specimens near one another in trait space. This makes appearance measurable across hundreds of thousands of digitized specimens. I use these embeddings to identify visually similar species, map the morphospace of butterfly wing patterns, and test whether phenotypic similarity recovers known mimicry relationships. Combined with language models for natural-language search, this work shows how I use emerging AI tools to extract evolutionary information from natural history collections.

Each point is a species centroid in a principal-component space of image embeddings of digitized butterfly specimens. Switch between the dorsal and ventral surfaces to see the shift in morphospace. Some families, such as the skippers (Hesperiidae) and whites and sulphurs (Pieridae) separate clearly, while many lineages change position between sides. Hover a point for its name, or pick a family to highlight it.

Mimicry is a direct test of whether image embeddings capture what predators perceive. I compiled published Müllerian and Batesian butterfly mimicry pairs and, using dorsal wings, ranked how similar each partner is to the other among all species in the collection. Mimicry pairs rank far above chance, and above congeners. The misses are informative: in Papilio polytes and P. glaucus only some females mimic, and Heliconius timareta varies across its range. More work is needed to account for these variations. However, this initial work shows the potential of image embeddings to capture biologically meaningful visual similarity, and that this similarity can be used to study mimicry, species interactions, and phenotypic evolution. Use the interactive plots below to explore the results.

I designed natural-language search for a large specimen collection. The system uses a language model to parse a question such as “orange butterflies with black stripes from Peru” into structured search calls (color, country, trait, or similarity). To choose the best model for production, I benchmark open-weight models on the same set of queries, comparing accuracy, consistency, latency, and token cost. Open-weight models keep the search reproducible and let any institution with limited resources, run it on its own server. It protects sensitive data, and allows for fine-tuning on domain-specific text. See the interactive plot below to explore the results of the benchmark.

BioCosmos brings this work together in a low-cost web platform, deployed through LepiVerse.com, for more than 600,000 digitized Lepidoptera specimen images. Image embeddings power visual-similarity search and interactive morphospace plots, an open-weight language model turns plain-language questions into searches, and taxonomic harmonization and geographic validation keep the underlying data clean. The result shows that emerging AI tools can make large natural history collections explorable for anyone with a browser. To see how the system is designed, explore the BioCosmos architecture in the Software section.

I hope to extend this framework to Indo-Australian rodents, asking whether phenotypic divergence among isolated island and mountain populations parallels genomic divergence or reflects local environments, and, through collaborations, to frogs, reptiles, and birds.

My 13 years of experience working in the complex island systems of Southeast Asia exposed the limitations of conventional biodiversity research workflows. These include difficulties in preserving the breadth of information needed to understand complex biological systems, integrating genomic and other diverse data types, providing real-time feedback to reduce observer bias in the field, and enabling contributions from researchers with limited computational resources. These challenges motivate my software development efforts.

I develop free, open-source software spanning field data collection (NAHPU), phylogenomics (ULLAR), genomic data processing (SEGUL), taxonomy (Mammal Diversity Database), and AI-enabled specimen exploration and trait analyses (BioCosmos). Across my public repositories, I have added 1.3M lines of code and removed 623K. These projects aim to make biodiversity research more efficient, reproducible, and accessible, particularly to underfunded laboratories, while enabling new research directions, such as using vector embedding approaches to study multi-dimensional trait variation.

My tools work on anything, from a phone to a high-performance computing cluster. It installs like everyday software, either with one click from the app stores or a single command in the terminal, without requiring additional applications, such as Docker or Singularity. Beyond speed, they capture information that was difficult to record before. NAHPU, for example, links every specimen part, field observation (e.g., habitat and environmental conditions), parasite collection, and derivative data (e.g., audio, video, and photographs) to its voucher from the moment of collection. It generates statistics that detects outlier measurements and aids decision-making in the field. This includes prioritizing specimens for complete tissue sampling when preservation resources are limited and reducing observer bias toward specimens that are easier or more convenient to collect.

NAHPU is a field cataloging app for collection-based research, from specimens to the parasites collected from them and their screening status. It has to work on a phone or a tablet at a remote field camp and on a laptop back at the museum, with or without internet. I build it in layers: a Flutter interface on top, application state and domain workflows in the middle, a SQLite database for records, and performance-critical, reusable code in Rust crates interfaced through a generated bridge. Each layer has one job, which keeps the app fast, testable, and easy for new contributors to learn. The full design is documented in the NAHPU developer docs.

BioCosmos, the platform behind LepiVerse, lets anyone search more than 600,000 butterfly specimen images by name, image, or a plain-language question. Its design problem is that it must answer open-ended questions over hundreds of thousands of images and their embeddings on a single, modest server. I built it as one FastAPI process that holds everything needed for the search. An open-weight language model plans the search, the CLIP and UNICOM vision models encode it, and two embedded databases execute it. DuckDB, an in-process analytical (OLAP) database is for metadata, taxonomy, localities, and traits, and LanceDB, an in-process vector database is for the image embeddings. Neither needs a database server, making the system lightweight and efficient. Both query columnar files in place and exchange results as Arrow tables through Polars.

The language model never touches the data. It reads a question once and returns typed tool calls for location, traits, common name, color, or visual similarity. This design keep token usage low and data stay private. Code then runs them in two stages: filters run concurrently as SQL in DuckDB and intersect into an allowlist of species, and ranking tools search the vector index only within that allowlist. Hard constraints stay exact, and vector search spends its time only on candidates that can match. Use the interactive diagram below to explore the architecture.

Data preparation is handled by command-line packages that are decoupled from the backend and can run independently. This design enables the code to be used outside Biocosmos. The packages share the same uv workspace, ensuring consistent dependency management and versioning across the project. They match taxonomic names against the Catalogue of Life, validate coordinates against GADM, and precompute visual similarity and morphospace tables. Each run writes a manifest containing the SHA-256 hash of every generated output, providing a verifiable record for traceability and reproducibility. The same workspace also benchmarks candidate planner models, which informed the selection of the open-weight model used in production (see the AI section).

Between assembling loci and inferring trees lies an often overlooked step: exploring the data. SEGUL summarizes alignments, including taxon occupancy, missing data, and parsimony-informative sites, as well as raw read and contigs. It also filters, concatenates, splits, converts, and renames thousands of files in a single workflow. Its ability to filter alignments by length, completeness, and informative sites is unique among comparable tools. In an accompanying perspective, Chan (2024) described SEGUL as a proof of concept that efficient software can make genomic-scale analyses more tractable and phylogenomics accessible to a broader range of users. SEGUL received the 2024 Ernst Mayr Award from the Society of Systematic Biologists. I am now extending SEGUL and ULLAR with automated detection of outlier alignments and clearer diagnostics.

Every interface runs on the same high-performance Rust core, so users can choose whichever platform best fits their workflow without sacrificing performance. I benchmarked SEGUL against widely used alternatives (AMAS, goalign, and Phyluce) using six published datasets spanning 239.3–522.5 million bases. Each benchmark was repeated 5 times on a desktop (AMD Ryzen 9 5900X, openSUSE Linux), an iPad, and an Android device (Handika & Esselstyn 2024). The interactive plot below lets you explore the results across tools, datasets, and platforms.

I develop the website, cross-platform app, and data tooling for the Mammal Diversity Database of the American Society of Mammalogists, which tracks taxonomic changes to every living and recently extinct mammal species. Version 2.5 recognizes 6,904 species, each with names, synonyms, distribution, and images.

Continuously growing collections keep natural history museums relevant: newly acquired specimens are the best material for genomic, microbiome, and phenomic studies, and many regions remain poorly documented. Over 13 years, my work has both drawn on existing museum collections and built new ones, expanding sampling to under-surveyed islands and mountains.

Museums are critical interfaces between scientific discovery and public literacy. At outreach events, such as those at the LSU Museum of Natural Science, I share representative specimens from our collections and the tools we use in the field. Engagement also continues in the field, with the local communities who make our expeditions possible.

This project is led by Dr. Nurainas at the Herbarium Universitas Andalas (ANDA) funded through the collection and data mobilization grant provided by the Global Biodiversity Information Facility (GBIF). We digitized >65,000 sheets of the herbarium specimens at ANDA and make the data accessible through the GBIF portal.

Nurainas et. al. ( 2024a: Flora of Sumatra : Vascular plant collection from Batang Toru Forest deposited in ANDA Herbarium, show links

Flora of Sumatra : Vascular plant collection from Batang Toru Forest deposited in ANDA Herbarium

Nurainas N, Amolia R R, Taufiq A, Handika H, Syamsuardi S. 2024a

Article (opens in new tab)

, 2024b: Flora of Sumatra: Vascular plant collection of selected families deposited at Herbarium of Andalas University (ANDA), show links

Flora of Sumatra: Vascular plant collection of selected families deposited at Herbarium of Andalas University (ANDA)

Nurainas N, Taufiq A, Handika H, Syamsuardi S. 2024b

Article (opens in new tab)

, 2024c: Flora Sumatra: Digitizing and data basing specimens of the Sumatran Flora deposited at Herbarium Universitas Andalas (ANDA)-Part 2, show links

Flora Sumatra: Digitizing and data basing specimens of the Sumatran Flora deposited at Herbarium Universitas Andalas (ANDA)-Part 2

Nurainas N, Taufiq A, Handika H, Harapan T S, Syamsuardi S. 2024c

Article (opens in new tab)

)