Importing Legacy Data into MERMAID

Have existing data from past surveys in spreadsheets? You can import these directly into MERMAID using mermaidr to have all your past and present data in one platform.

Step 1: Set Up a Project in MERMAID Collect

• Set up a new project in MERMAID Collect (or check that you have an existing project ready for the data import): 

  • Add your organization(s).

  • Add users and assign their roles. 

  • Set the data sharing policies. 

  • Add sites and management regimes associated with the data you are importing.

• If using an existing project, ensure you are an Admin of that project.

Read more in Setting Up a Project.

Step 2: Download the MERMAID Template in R

• Install and load the R package mermaidr in RStudio. Read more in Getting Started with the MERMAID R Package.

• Load your project to import data using the function mermaid_get_my_projects(). You’ll need to authenticate using your MERMAID credentials.

• Load the MERMAID data template for the survey method you need using the function mermaid_import_get_template_and_options(). Use the survey method syntaxes as listed in Accessing Project Data from MERMAID. More information about this function can be found here. 

Step 3: Reformat Your Data to Match the Template

• Load your data and sample event spreadsheets (including your sites, management regimes and survey dates) as a CSV file using read.csv(). 

• If your data are separated between the observations and the site characteristics (e.g., site, names, management regimes, survey dates, observers, etc), join these two datasets together by the site name using the left_join() function. Join these two datasets together by the site name using the left_join() function. 

• Reformat your data spreadsheet to match the selected MERMAID template. 

  • Mandatory fields are marked with an asterisk (*) in the template and these fields must be present in your spreadsheet as individual columns with the exact same name and required field options. These fields cannot be blank.

  • Non-mandatory columns can be blank or even removed from your data spreadsheet. 

  • Mutate any values in your data spreadsheet to match the available options in specific fields in the MERMAID template

Step 4: Address Errors and Warnings

• Check for errors and warnings using mermaid_import_check_options(). You’ll need to check the columns one by one to ensure data is accepted by the MERMAID template and fix any issues.  This function will compare the information in the MERMAID template to the information in your data. More information about this function can be found here. 

Step 5: Import Data to MERMAID

• Once all checks are done, import the dataset with mermaid_import_project_data(). 

• First do a dry run (dryrun=TRUE in the function), then do the real import (dryrun=FALSE in the function). More information about this function can be found here. 

Step 6: Validate and Submit Data in Your Project

• Once your data has been imported into your project in MERMAID Collect, it’ll appear in your Collecting page as individual sample units. 

• Validate and submit each sample unit, following the standard procedure for data validation and submission. Read more in Saving, Validating and Submitted Data. 

• Alternatively, you can validate, submit and edit all sample units at once in R using the following functions: mermaid_import_bulk_validate(),  mermaid_import_bulk_submit(), mermaid_import_bulk_edit(). 

For more information, access the links: Bulk validation function, Bulk submission function, Bulk edits

Once submitted, sample units become available to all project’s users in the Submitted page.

Detailed Guide and Import Example

Download the data import guide: Step-by-step ingestion guide (PDF)

Example of fish belt data import: Ingesting historical fishbelt data

Are You New to R? 

If you’re new to programming in R, we recommend:

• R Users Guide for MERMAID – a quick-start guide designed for MERMAID users.

• Seascape Models R Courses – a free training in R using marine ecological data.

  • Start with: Cleaning ecological survey data for conservation scientists

These resources will build your confidence using R while helping you work with familiar marine datasets.

Useful Resources

• MERMAID R package documentation

• Accessing project data

• Importing legacy data

• Analysis templates for MERMAID data