Oral Session 10

Millisecond-scale behaviors of plankton quantified in situ and in vitro using the Event-based Vision Sensor (EVS)

The Event-based Vision Sensor (EVS) is a bio-inspired sensor that captures detailed motions of objects, developed with the applicability to become the ‘eyes’ of machines. Cameras using EVS are also known as event cameras. Compared to conventional frame-based image sensors as employed in video cameras, EVS has an extremely fast motion capture equivalent to 10,000-fps even with standard optical settings and additionally has high dynamic ranges for brightness and also lower consumption of memory and energy. Here, we developed 22 characteristic features for analyzing the motions of aquatic particles from the raw data of the EVS, and deployed the EVS system in both natural environments and laboratory aquariums to test its applicability to filming and analyzing plankton behavior. Our EVS monitoring at the bottom of Lake Biwa and the hydrothermal deposit in Myojin sea hill identified several particles exhibiting distinct cumulative trajectory with periodicities in their motion (up to 16 Hz), suggesting that they were living organisms with rhythmic behavior. Furthermore, we used the EVS to observe laboratory cultures of six species of zooplankton and phytoplankton, confirming that they have species-specific motion periodicities of up to 41 Hz. We applied machine learning to automatically classify particles into five categories (four categories of zooplankton plus passive particles), which achieved an accuracy up to 86%. Our attempts to use the EVS for biological observations, especially focusing on its millisecond-scale temporal resolution and wide dynamic range provide a new avenue to investigate rapid and periodical motion and behavior in small organisms.

Susumu Takatsuka, Sony Group Corporation and Japan Agency for Marine-Earth Science and Technology (JAMSTEC), Japan, https://orcid.org/0000-0002-1229-1963

Norio Miyamoto, Japan Agency for Marine-Earth Science and Technology (JAMSTEC), Japan, https://orcid.org/0000-0002-3051-3945

Hidehito Sato, Sony Group Corporation, Japan, https://orcid.org/0009-0003-2311-4457

Hitoshi Azumi, Sony Group Corporation, Japan

Yasuhito Hayashi, Japan Agency for Marine-Earth Science and Technology (JAMSTEC), Japan, https://orcid.org/0000-0002-8894-1184

Shinsuke Kawagucci, Japan Agency for Marine-Earth Science and Technology (JAMSTEC), Japan, https://orcid.org/0000-0002-7807-024X

Code of Conduct
Privacy Policy