Freva on Levante - what's news?

  • Levante is operational since 3rd of March 22
  • Freva isn't quite yet 😢
  • 2832 CPU computing nodes: 2x AMD EPYC 64 cores @ 3.5 GHz (256 - 1024 GB RAM)

by: Brian Lewis, Christopher Kadow, Etor Lucio Eceiza, Mahesh Ramadoss, Martin Bergemann

What have we actually done during all this time?

  • updated freva core from python2 âžœ python3, (python2 EOL since 2020)
  • did a major infrastructure re-design
    • including the plugin infrastructure
  • added some user features

New features:

The freva python module:

import freva
import xarray as xr
files = freva.databrowser(
    project="cmip6",
    experiment="historical",
    model="mpi*",
    ensemble="r1i1*",
    variable="tas",
    time_frequency="mon"
)
dset = xr.open_mfdataset(files, combine="by_coords", parallel=True)
monthly_data dset.resample(time="mon").mean()

Improved argument completion

Other changes and add-ons:

  • Facelift for the website
  • Slight changes of the command line interface:
    • freva --databrowser âžœ freva databrowser or freva-databrowser etc
  • metadata-inspector: ncdump -h on steroids
  • utility to calculate global warming level
  • kernel-install
  • rechunk-data

Data

  • More advanced data crawling mechanism allowing to add data from /pool/data to the databrowser
  • Slightly more data than before
  • cordex-nukleus data project âžœ 5 x 5 Model matrix nukleus recommends to the model regions
  • nukleus data project âžœ Model simulations and bias corrected data.

Plugins

  • Most of the existing plugins have also been updated: python2 âžœ python3
    • Few are causing hiccups for example CWT
  • Plugins are deployed in their own anaconda environment:
    • Better reproducibility
    • We offer a plugin template to get users started with the setup
  • Most notable new plugins for nukleus/regiklim
    • EnsemblePlotter: Create and compare mulit-model ensembles - still in prototype phase
    • ClimateChangeProfile: Compare climate change signal for selected area
    • ClimPact: This has actually been quite successful for model regions
  • But ... there's room for improvement

Suggestion: Convective Cell tracking algorithm (tint)

Studying Convective Extreme Events (Hector the convector)

A few other things to think about

  • We have 20 TB cloud storage - let's make use of it!
  • Do we really need 3 hourly, 6 hourly, daily and monthly data if we have hourly data?
  • I think we do want to move away from redmine as a documentation platform.