When AI Meets Fluorescence Microscopy
What if you could instruct a fluorescence microscope using natural language instead of navigating software menus or writing scripts? And what if the same AI assistant could analyze the resulting images, identify regions of interest, perform additional measurements, and help document the experiment?
We explored this approach by connecting Claude, an AI assistant developed by Anthropic, to our FLIM mircoscope Luminosa through LumiPy, its Python programming interface.
In our demonstration, Claude interacts with a real microscope, acquires fluorescence lifetime imaging (FLIM) data, identifies structures within the sample, initiates measurements at selected positions, and generates a draft scientific report.
The result is a demonstration of how AI assistants can be integrated into experimental workflows, extending their role beyond the analysis of previously acquired data.
From Natural Language to Instrument Control
The connection between Claude and Luminosa is made possible by LumiPy, PicoQuant’s Python interface for microscope control and data acquisition.
LumiPy provides programmatic access to Luminosa’s hardware and measurement functions. Researchers can control microscope components, define positions and regions of interest, configure and execute image and point measurements, and access online measurement results. These capabilities support custom workflows in which experimental steps can be adapted based on previously acquired data.
Learn more about feedback-adaptive microscopy in our article Smart Microscopy with Luminosa and LumiPy.
By connecting Claude to LumiPy, these capabilities become accessible through natural language interaction. Rather than manually executing each step, a user can describe an experimental task to the AI assistant, which translates the request into operations performed through the programming interface.
Importantly, LumiPy extends rather than replaces the existing Luminosa software. The microscope remains responsible for acquiring experimental data, while Claude coordinates the requested operations and processes the information made available through the software interface.
From Fluorescence Imaging to Data Analysis
To demonstrate this approach, we used Luminosa to investigate a sample of daisy pollen using fluorescence lifetime imaging.
The workflow begins with Claude establishing a connection to the microscope through LumiPy. Once connected, the AI assistant initiates a FLIM measurement, acquiring an image that provides spatially resolved fluorescence lifetime information.
The next step goes beyond image acquisition. Claude identifies six individual pollen grains in the overview image and defines a separate region of interest (ROI) for each, ensuring that every ROI contains exactly one pollen grain.
These regions then serve as the basis for additional measurements. Claude uses LumiPy to acquire a five-frame FLIM image of each ROI, selecting an image resolution of 256 × 256 pixels. The acquired images are subsequently analyzed to determine the average fluorescence intensity, average fluorescence lifetime, ROI area, and pollen area. Pollen areas are determined using Otsu segmentation, while average intensity and lifetime values are calculated within the resulting pollen masks.
The results are compiled into a structured table, providing quantitative information for each of the six pollen grains. Claude also documents potential limitations of the analysis, including cases where ROI boundaries may lead to an underestimation of pollen area.
Finally, the assistant uses the acquired data and analysis results to generate a draft Methods and Results section describing the experiment and its findings. The demonstration also shows how the generated code can be exported for further use.
This sequence connects several tasks that would otherwise require separate interactions with the microscope software and data analysis tools. Image acquisition informs the selection of measurement locations, which in turn provides data for further analysis and documentation. The final step is the creation of a python script with well written comments which describe the whole process and can be reused without the Agent in the future.
What Does This Mean for Experimental Workflows?
The significance of this demonstration extends beyond controlling a microscope with a chatbot. It illustrates how programmable scientific instruments can be connected to AI assistants to coordinate multiple stages of an experiment. Traditionally, researchers interact with measurement instruments through graphical user interfaces or custom scripts. Natural language interaction introduces another option, allowing experimental tasks to be expressed as instructions that an AI assistant translates into executable operations.
The potential becomes particularly interesting when individual steps are linked. An initial image can guide the selection of measurement positions, newly acquired data can inform subsequent actions, and analysis results can be incorporated into documentation without manually transferring information between separate tools. Such approaches could be useful for repetitive measurements, adaptive acquisition strategies, and experimental workflows that involve multiple software environments.
The generation of a Methods and Results section adds another dimension. It demonstrates how information about the measurement procedure and acquired data can be used to prepare scientific documentation. However, AI-generated descriptions and interpretations still require verification by the researcher, particularly regarding experimental parameters, data quality, and scientific conclusions.
The demonstration therefore does not establish fully autonomous scientific experimentation. Instead, it shows a practical way of bringing AI-assisted interaction into an existing measurement workflow while keeping scientific evaluation and responsibility with the researcher. The benefits are clear: easier interaction, faster implementation of tailor-made workflows, trustworthy efficient reporting.
Looking ahead, several questions remain. Will research groups and institutions develop their own AI assistants for instrument control? Will researchers embrace natural language interaction alongside traditional graphical interfaces? And will scientific communities establish common standards for AI-assisted instrumentation?
One particularly promising direction is the integration of scientific context into experimental workflows. Imagine describing a research question, providing information about the sample, fluorescent labels, experimental conditions, and relevant published findings, and having an AI assistant use this context to suggest suitable measurement parameters and help design an automated workflow. While such approaches will require careful validation and continued researcher oversight, the combination of programmable instruments and context-aware AI brings this possibility increasingly within reach. The possibilities are evolving, but the direction is clear: AI is opening new ways to interact with scientific instruments.
Exploring AI-Assisted Microscopy with Luminosa and LumiPy
Connecting Claude to Luminosa demonstrates what becomes possible when scientific instruments provide flexible programming interfaces. With Luminosa providing the measurement capabilities and LumiPy enabling programmatic access to instrument functions and data, researchers can develop workflows that combine microscopy, automation, and external analysis tools.
The integration with Claude offers one example of how these capabilities can be used. More broadly, by making microscope functions accessible to external software, LumiPy provides a foundation for exploring new approaches to instrument interaction, experimental automation, and AI-assisted microscopy.
