Below is a record of the data products synthesised during MiMeMo. I’ve kept track of the structure of the data to make it easier to quickly check what we have. This will also function as a README for anyone else who follows and wants signposting to sources, or to know what additional data could be downloaded.
Shortwave surface Irradiance inputs to NEMO-MEDUSA.
Available on the idrive at Strathclyde, supplied by Andy Yool at NOC.
Global.
NA
Surface air temperatures as inputs to NEMO-MEDUSA.
Available on the idrive at Strathclyde, supplied by Andy Yool at NOC.
Global.
NA
Freshwater inputs to NEMO-MEDUSA.
Available on the idrive at Strathclyde, supplied by Andy Yool at NOC.
Global.
NA
NEMO-MEDUSA model OUTPUTS for multiple Variables.
Available on the idrive at Strathclyde, supplied by Andy Yool at NOC.
North Atlantic into Arctic..
❗ Note, these matrices are different for velocity files (cell corners not centres). The grid otherwise appears the same.
❗ Note, grid_W files contain different depths for layers.
Global terrain model for ocean and land at 15 arc-second intervals.
Global.
NA
Sediment classes in the Barents Sea, an overview document of likely bottom types from expert synthesis of multiple data sets (‘SEABED SEDIMENTS (GRAIN SIZE), N750 OVERVIEW’).
Barents Sea. .
Polygon boundaries embedded in shapefile.
NA
Significant wave height and direction from the CERA-20C ‘Ocean Wave Synoptic Monthly Means’.
I downloaded the Northern hemisphere only.
Global.
I only downloaded as far back as 2000.
Located within netcdf dims, not as variables.
Additional variables are available, but have not been downloaded from the web portal.
‘10 metre wind direction’, ‘10 metre wind speed’, ‘Air density over the oceans’, ‘Benjamin-Feir index’, ‘Coefficient of drag with waves’, ‘Free convective velocity over the oceans’, ‘Maximum individual wave height’, ‘Mean direction of total swell’, ‘Mean direction of wind waves’, ‘Mean period of total swell’, ‘Mean period of wind waves’, ‘Mean square slope of waves’, ‘Mean wave direction of first swell partition’, ‘Mean wave direction of second swell partition’, ‘Mean wave direction of third swell partition’, ‘Mean wave period based on first moment’, ‘Mean wave period based on first moment for swell’, ‘Mean wave period based on first moment for wind waves’, ‘Mean wave period based on second moment for swell’, ‘Mean wave period based on second moment for wind waves’, ‘Mean wave period of first swell partition’, ‘Mean wave period of second swell partition’, ‘Mean wave period of third swell partition’, ‘Mean zero-crossing wave period’, ‘Normalized energy flux into ocean’, ‘Normalized energy flux into waves’, ‘Normalized stress into ocean’, ‘Peak wave period’, ‘Period corresponding to maximum individual wave height’, ‘Significant height of total swell’, ‘Significant height of wind waves’, ‘Significant wave height of first swell partition’, ‘Significant wave height of second swell partition’, ‘Significant wave height of third swell partition’, ‘U-component stokes drift’, ‘V-component stokes drift’, ‘Wave Spectral Skewness’, ‘Wave spectral directional width’, ‘Wave spectral directional width for swell’, ‘Wave spectral directional width for wind waves’, ‘Wave spectral kurtosis’, ‘Wave spectral peakedness’.
Tidally resolved water velocity fields in the Arctic. Numerical ocean model output.
Available on the idrive at Strathclyde, supplied by Ingrid Ellingsen at SINTEF
Arctic.
w_east’, ‘w_north’, ‘w_velocity’, ‘vertical_diffusivity’, ‘NO3’, ‘PhytoF’, ‘PhytoD’, ‘temperature’, ‘salinity’, ‘ice_thickness’, ‘ice_compactness’, ‘freezing_rate’, ‘salinity_ice’, ‘uIce_east’, ‘vIce_north’, ‘uUnderIceE’, ‘vUnderIceN’, ‘elevation’.
‘Global Nutrient Export from Watersheds’. Modeled N and P input to oceans from rivers.
Global.
Nretention’, Pload’, ‘Pretention’.
‘Arctic Great Rivers Observatory’. Water quality field recordings from the 6 largest Arctic rivers.
Arctic.
Cond’, ‘Alk’, ‘H20-2H’, ‘H20-180’, ‘DOC’, ‘DOC-13C’, ‘DOC-14C’, ‘DOC-fm’, ‘NO3’, ‘TDP’, ‘PO4’, ‘SiO2’, ‘Na’, ‘K’, ‘Mg’, ‘Ca’, ‘As’, ‘Ba’, ‘Cl’, ‘SO4’, ‘Li’, ‘Rb’, ‘Sr’, ‘U’, ‘F’, ‘UTHg’, ‘UMeHg’, ‘TSS’, ‘POC’, ‘POC-13C’, ‘POC-14C’, ‘POC-fm’, ‘PON’, ‘PON-15N’, ‘Sample depth 1’, ‘TSS (1)’, ‘POC (1)’, ‘POC-13C (1)’, ‘PON (1)’, ‘PON-15N (1)’, ‘Sample depth 2’,‘TSS (2)’, ‘POC (2)’, ‘POC-13C (2)’, ‘PON (2)’, ‘PON-15N (2)’.
Suspended particulate matter from binned monthly satellite colour data.
Downloaded across an FTP connection. The code below should automatically do this for you, but you will need to get your own username and password from Globcolour, and replace this in the script.
## Automate the downloading of supsended particulate matter from GloColour server
#### Set up ####
rm(list=ls())
library(tidyverse)
library(RCurl)
download <- function(Source, File) {
Sys.sleep(pause) # Pause so we're not locked out
download.file(url = paste0(Source, File), destfile = paste("./Data/SPM/", File))
} # Create path, download, and specify destination file name
url <- "ftp://ftp.hermes.acri.fr/GLOB/merged/month/" # Path to directory of interest
userpwd <- "ftp_gc_*******:*******" # Your username and password
pause <- 120 # How long to leave between requests to the server
#### Build folder structure and download SPM files ####
files <- getURL(url, userpwd = userpwd, ftp.use.epsv = F, dirlistonly = T) %>% # List the sub-directories for years
strsplit("\n") %>% # Seperate names into individual entries
map(function(x) paste0(url, x,"/")) %>% # Create a full path for each subdirectory found
unlist() %>% # Strip back to a vector of yearly subdirectories
map(~ { # For each year folder
Sys.sleep(pause) # Pause so we're not locked out
url <- .x # Rememeber the name of this folder
print(url)
months <- getURL(.x, userpwd = userpwd, ftp.use.epsv = F, dirlistonly = T) %>% # Check which months are reported this year
strsplit("\n") %>% # Seperate names again
map(function(x) paste0(url, x,"/01/")) %>% # Create full paths for each source/year/month paths
unlist()
}) %>% # How many months are in each year folder? pause so I'm not locked out
unlist() %>%
map(~ {
Sys.sleep(pause) # Pause between checking each folder to stop getting locked out
url <- .x # Remember the route to this subdirectory
print(url)
files <- getURL(.x, userpwd = userpwd, ftp.use.epsv = F, dirlistonly = T) %>% # Which files are in here?
strsplit("\n") %>% # Separate files
map(function(x) paste0(url, x)) %>% # Create full path for each of these files
unlist() %>%
as.data.frame(col.names = "Files") %>% # Get a dataframe of all file names
filter(grepl(".nc", `.`, fixed = TRUE)) %>% # Limit to netcdf files
filter(grepl("SPM", `.`, fixed = TRUE)) %>% # Limit to suspended particulate matter files
filter(grepl("L3b", `.`, fixed = TRUE)) # Limit to files on their Binned grid (high resolution I think)
}) %>% # Which files are in each folder? with a pause so I'm not locked out
bind_rows() %>% # Bind file names into dataframe
rename(value = 1) %>%
separate(value, into = c("Source", "File"), sep = "L3b_") %>% # Isolate file name to use when saving files
mutate(File = paste0("L3b_", File), # Replace dropped separator
Source = str_replace(Source, "ftp.hermes", paste0(userpwd,"@ftp.hermes"))) # Add password into link request
pmap(files, download) # Download all the files, don't parallelise, we don't want to overload the server!
Global.
Two dimensions are also required:
To get to coordinates in latitude and longitude:
index <- row[bin] - row[1]+1
lat_bin = center_lat[index]
lon_bin = center_lon[index] + col[bin] * lon_step[index]
Additional variables are available, but have not been downloaded from the server.
CHL1’, ‘CHL-OC5’, ‘CHL2’, ‘TSM’, ‘PIC’, ‘POC’, ‘NFLH’, ‘WVCS’, ‘Txxx’, ‘Axxx’, ‘CF’, ‘ABSD’, ‘(N)RRSxxx’, ‘PAR’, ‘BBP’, ‘CDM’, ‘KD490’, ‘KDPAR’, ‘ZHL’, ‘ZEU’, ‘ZSD’, ‘CHL-CIA’, ‘BBPxxx-LOG’, ‘BBPS-LOG’, ‘PSD-XXX’, ‘POC-SURF’, ‘POC-INT’, ‘PP’, ‘PP-AM’, ‘PP-UITZ’, ‘PHYSAT’.
Modelled atmospheric deposition of reduced and oxidised Nitrogen from EMEP MSC-W.
Downloaded from the EMEP server. The code below should automatically do this for you.
## Automate the downloading of N atmospheric depostion data from EMEP server
library(tidyverse)
library(rvest)
download <- function(file) {
download <- download.file(url = paste0(source, file), destfile = paste("./Data/EMEP Atmosphere/", file))
} # Create path, download, and specify destination file name
source <- "https://thredds.met.no/thredds/fileServer/data/EMEP/2018_Reporting/" # Where are the files stored?
read_html("https://thredds.met.no/thredds/catalog/data/EMEP/2018_Reporting/catalog.html") %>% # Import the file catalog
html_nodes("a") %>% # Extract links
html_attr("href") %>%
.[which(grepl("month", .), )] %>% # Limit to monthly files
str_remove(., fixed("catalog.html?dataset=EMEP/2018_Reporting/")) %>% # Extract file names from links
map(download) # Download all the files, don't parallelise, we don't want to overload the website!
Europe.
Located within netcdf dims, not as variables.
Additional variables are available, but have not been downloaded from the web portal.
SURF_ugN_NOX’, ‘SURF_ugN_RDN’, ‘SURF_ugN_OXN’, ‘WDEP_PREC’, ‘WDEP_SOX’, ‘SURF_MAXO3’, ‘SURF_PM25water’, ‘SOMO35’, ‘DDEP_SOX_m2Grid’, ‘DDEP_SOX_m2Conif’, ‘DDEP_SOX_m2Decid’, ‘DDEP_SOX_m2Seminat’, ‘DDEP_OXN_m2Conif’, ‘DDEP_OXN_m2Decid’, ‘DDEP_OXN_m2Seminat’, ‘DDEP_RDN_m2Conif’, ‘DDEP_RDN_m2Decid’, ‘DDEP_RDN_m2Seminat’, ‘MMAOT40_IAM_DF’, ‘MMAOT40_IAM_CR’, ‘EUAOT40_Crops’, ‘EUAOT40_Forests’, ‘SURF_ug_SO2’, ‘SURF_ug_NH3’, ‘SURF_ug_HNO3’, ‘SURF_ug_NO2’, ‘SURF_ug_NO’, ‘SURF_ug_SO4’, ‘SURF_ug_NO3_F’, ‘SURF_ug_NO3_C’, ‘SURF_ug_NH4_F’, ‘SURF_ppb_O3’, ‘SURF_ug_PM10’, ‘SURF_ug_PPM25’, ‘SURF_ug_PPM_C’, ‘SURF_ug_SS’, ‘SURF_ug_DUST’, ‘SURF_ug_ECFINE’, ‘SURF_ug_ECCOARSE’, ‘SURF_ug_PM25’, ‘SURF_ug_PM25_rh50’, ‘SURF_ug_PM10_rh50’.
Fishing effort from anonymised VMS signals and a model of vessel movement patterns.
Global.
flag’, ‘vessel_hours’, ‘mmsi_present’.
International report on fisheries landings by species and country.
Landings from ICES were downloaded from Ian’s cleaned portal hosted at Strathclyde.
Major fishing area 27. .
ccode2’, ‘ccode3’, acronym’, ‘tsn’, ‘common’.
Reports of landings and effort by species and gear of Norwegian vessels.
Provided at request by Tom Clegg at IMR Bergen.
Barents & Norwegain Sea. .
‘The raw fish sellers society of Norway’. Value and weight of exploited species sold in different regions of Norway.
Norwegian coast.
Reports of effort by gear for EU vessels.
Global.
Polygon boundaries embedded in shapefile.
vssl_ln’, ‘fshng_t’, ‘msh_sz_’, ‘trgt_ss’,‘metier’, ‘spr_rgn’, ‘sub_rgn’,‘spcn_tc’, ‘deep’,‘icesnam’, ‘cscode’.
Reports of landings by species and gear for EU vessels.
Global.
Polygon boundaries embedded in shapefile.
cscode’, ‘quarter’, ‘vssl_ln’, ‘fshng_t’, ‘msh_sz_’, ‘trgt_ss’, ‘metier’, ‘spr_rgn’, ‘sub_rgn’, ‘spcn_tc’, ‘deep’, ‘icesnam’.