Scripps Institute of Oceanography

Software Development Team Member

UC San Diego
April 2017 - June 2017

Role person

I was a Software Development Team Member during a 3 month project carried out by a researcher at the Scripps Institute of Oceanography. My focus was as a front-end software developer.

Team group

I worked in a team of 4 people under the supervision of a Scripps researcher. The team comprised of 4 software developers, who are all currently UCSD undergraduate students.

Work border_color

The team worked on continuing the development of a Convolutional Neural Network that can categorize and classify recorded fish noises according to species.

Underwater Acoustics: When fishes can make noise

Underwater Acoustics is a Scripps project, implemented under the guidance of Dr. Ana Sirovic, a researcher at the Marine Acoustics Lab of the Scripps Institute of Oceanography. The project came about by researchers at the Scripps Institute of Oceanography when they decided to deploy a detector which recorded ocean sounds following the Deepwater Horizon Oil Spill of 2010. The detector was left in the area for 7 years and has since collected a significant amount of acoustic data, much of which is comprised of fish calls.

Overview

It is often unclear to what extent human activities have damaged ocean environments and whether or not our actions have been effective in repairing damaged habitats and restoring marine populations to their former states. Our project aims to address this issue by allowing researchers to monitor trends in sea life over long periods of time.

Problem

Of the 7 years of acoustic data, researchers have been able to successfully anotate 2 years worth, classifying recurring acoustic patterns by the fish species that are believed to have produced them. This, however, is not an efficient process and even if researchers were to successfully parse all of the data, the question remains: What should be done with all of this information?

Solution

Convolutional Neural Networks. By taking the 2 years of annotated data and feeding it into a custom CNN as training data, the remaining data can be systematically organized and classified. Further, by developing a detector in parallel, the Neural Network can be used as a research tool for the long-term surveilance of sea-life populations.

Deliverables

Our project deliverables included three things: (1) An Automated Detector - takes in acoustic data and outputs the time of signal occurence., (2) A User Friendly Interface - an executable with means of interfacing between detector and classifier to allow researchers without a deep knowledge of MATLAB to use the tool, and (3) The Classifier - a CNN designed to classify fish species by the noises they produce.

What I Did

My task, along with one other team member, was to implement a user-friendly, graphical user interface of the Convolutional Neutral Network program to improve usability.

In addition to the GUI, my teammate and I developed and maintained a website to document progress. Please check it out for more detailed information on this project and the team.

View Website →


Overview

My team decided to divide into 2 sub-teams to take on the deliverables. With that, I and one other teammate took on the task of implementing the User Friendly Interface program for the CNN.

Skills

The development and technology tools used for this project included MATLAB for development and GitHub for version control.

Task

My teammate and I worked on implementing a GUI for the CNN so researchers of any background can easily use and navigate with data they had. I developed the interface using MATLAB's GUIDE tool and consistently tested the program throughout the development phase.

In-Progress Images

Below are some images of the progress my teammate and I made midway of the project of the interface.

The images below show how the interface looks when it is opened for the first time. The user is shown a menu of what they would like to do and they select sound files they would like to begin testing.

***Unfortunately, I do not have images of how the original program worked before implementation of this project began.***


The Process

Here was my process through working on this task (quarter-long project):

  • Began looking through the existing code legacy to understand what each file did
  • Replicated the program to get an idea of how the program worked with the data passed in by users
  • Researched how to create a GUI in MATLAB and began implementation of task
  • Ported over the existing code base to the GUI program and modified the files to work towards a working interface
  • Extensive testing to ensure the end product met the baseline functionality

More Images

Below are some more images of the progress my teammate and I made near the end of the project of the interface.

Users are able to automate the CNN and train the data. The last photo shows the overall final user interface, where users are able to complete their tasks within one whole interface.


Reflection

Overall, the project was a great way to get experience on working in a team and gaining technical skills. I learned new skills, such as programming in MATLAB and developing a GUI using MATLAB'S GUIDE tool. However, that happened to also be a challenge for me. Prior to this project, I had no knowledge in programming in MATLAB. Additionally, I did not know about Convolutional Neural Networks. I overcame these obstacles by consulting my teammates for help when I needed to and doing lots of research while developing this project.

Also, throughout the project, we were well-monitored by the Scripps researcher by having bi-weekly check-ins to give updates on our progress and receiving feedback on how we were doing. At the end of the quarter, we turned in the final product and it was used to demo at a science conference the next month!