Note

Download this example as a Jupyter notebook here: https://github.com/pypsa/atlite/examples/create_cutout_SARAH.ipynb

Creating a Cutout with the SARAH-2 dataset

This walkthrough describes the process of creating a cutout using the SARAH-2 dataset by EUMETSAT.

The SARAH-2 dataset contains extensive information on solar radiation variables, like surface incoming direct radiation (SID) or surface incoming shortwave radiation (SIS). It serves as an addition to the ERA5 dataset and as such requires the cdsapi to be setup properly.

Recommendation

This is a reduced version for cutout creation. Creating cutouts with ERA-5 is simpler and explained in more details. We therefore recommend you have a look at this example first.

Note:

For creating a cutout from this dataset, you need to download large files and your computers memory needs to be able to handle these as well.

Downloading the data set

To download the dataset, head to the EUMETSTATs website (the link points to the current 2.1 edition)

https://wui.cmsaf.eu/safira/action/viewDoiDetails?acronym=SARAH_V002_01

On the bottom, select the products you want to include in the cutout, i.e. for us:

variable

time span

time resolution

Surface incoming direct radiation (SID)

2013

Instantaneous

Surface incoming shortwave radiation (SIS)

2013

Instantaneous

  • Add each product to your cart and register with the website.

  • Follow the instructions to activate your account, confirm your order and wait for the download to be ready.

  • You will be notified by email with the download instructions.

  • Download the ordered files of your order into a directory, e.g. sarah-2.

  • Extract the tar files (e.g. for linux systems tar -xvf * or with 7zip for windows) into the same folder

You are now ready to create cutouts using the SARAH-2 dataset.

Specifying the cutout

Import the package and set recommended logging settings:

[1]:
import atlite

import logging
logging.basicConfig(level=logging.INFO)
Disable urllib3 and cdsapi warnings.

To download the dataset, head to the EUMETSTATs website (the link points to the current 2.1 edition)

https://wui.cmsaf.eu/safira/action/viewDoiDetails?acronym=SARAH_V002_01

On the bottom, select the products you want to include in the cutout, i.e. for us:

variable

time span

time resolution

Surface incoming direct radiation (SID)

2013

Instantaneous

Surface incoming shortwave radiation (SIS)

2013

Instantaneous

  • Add each product to your cart and register with the website.

  • Follow the instructions to activate your account, confirm your order and wait for the download to be ready.

  • You will be notified by email with the download instructions.

  • Download the ordered files of your order into a directory, e.g. sarah-2.

  • Extract the tar files (e.g. for linux systems tar -xvf * or with 7zip for windows) into the same folder

You are now ready to create cutouts using the SARAH-2 dataset.

[2]:
cutout = atlite.Cutout(path="western-europe-2011-01.nc",
                       module=["sarah", "era5"],
                       sarah_dir="/home/vres-climate/data/sarah_v2",
                       x=slice(-13.6913, 1.7712),
                       y=slice(49.9096, 60.8479),
                       time="2013-01",
                       chunks={'time': 100}
                       )
INFO:atlite.cutout:Building new cutout western-europe-2011-01.nc

Let’s see what the available features that is the available weather data variables are.

[3]:
cutout.available_features.to_frame()
[3]:
variables
module feature
sarah influx influx_direct
influx influx_diffuse
era5 height height
wind wnd100m
wind roughness
influx influx_toa
influx influx_direct
influx influx_diffuse
influx albedo
temperature temperature
temperature soil temperature
runoff runoff

Preparing the Cutout

No matter which dataset you use, this is where all the work actually happens. This can be fast or take some or a lot of time and resources, among others depending on your computer ressources (especially memory for SARAH-2).

[4]:
cutout.prepare()
INFO:atlite.data:Calculating and writing with module sarah:
[########################################] | 100% Completed |  3min 19.2s
INFO:atlite.data:Calculating and writing with module era5:
INFO:atlite.datasets.era5:Downloading data for feature 'height' to /tmp/tmpk92au7s_.
INFO:atlite.datasets.era5:Downloading data for feature 'wind' to /tmp/tmpk92au7s_.
[########################################] | 100% Completed | 11.2s
INFO:atlite.datasets.era5:Downloading data for feature 'influx' to /tmp/tmpk92au7s_.
[########################################] | 100% Completed |  4.1s
INFO:atlite.datasets.era5:Downloading data for feature 'temperature' to /tmp/tmpk92au7s_.
[########################################] | 100% Completed | 10.9s
INFO:atlite.datasets.era5:Downloading data for feature 'runoff' to /tmp/tmpk92au7s_.
[########################################] | 100% Completed | 10.8s
[########################################] | 100% Completed |  1.3s
[4]:
<Cutout "western-europe-2011-01">
 x = -13.50 ⟷ 1.75, dx = 0.25
 y = 50.00 ⟷ 60.75, dy = 0.25
 time = 2013-01-01 ⟷ 2013-01-31, dt = H
 prepared_features = ['influx', 'height', 'wind', 'temperature', 'runoff']

Querying the cutout gives us some basic information on which data is contained in it an can already be used.

Inspecting the Cutout

[5]:
cutout # basic information
[5]:
<Cutout "western-europe-2011-01">
 x = -13.50 ⟷ 1.75, dx = 0.25
 y = 50.00 ⟷ 60.75, dy = 0.25
 time = 2013-01-01 ⟷ 2013-01-31, dt = H
 prepared_features = ['influx', 'height', 'wind', 'temperature', 'runoff']
[6]:
cutout.data.attrs # cutout meta data
[6]:
{'module': ['sarah', 'era5'],
 'prepared_features': ['influx', 'runoff', 'height', 'wind', 'temperature'],
 'chunksize_time': 100,
 'sarah_dir': '/home/vres-climate/data/sarah_v2'}
[7]:
cutout.prepared_features # included weather variables
[7]:
module  feature
sarah   influx            influx_direct
        influx           influx_diffuse
era5    height                   height
        wind                    wnd100m
        wind                  roughness
        influx               influx_toa
        influx                   albedo
        temperature         temperature
        temperature    soil temperature
        runoff                   runoff
Name: variables, dtype: object
[8]:
cutout.data # access to underlying xarray data
[8]:
Show/Hide data repr Show/Hide attributes
xarray.Dataset
    • time: 744
    • x: 62
    • y: 44
    • lon
      (x)
      float64
      dask.array<chunksize=(62,), meta=np.ndarray>
      standard_name :
      longitude
      axis :
      X
      long_name :
      longitude
      units :
      degrees_east
      Array Chunk
      Bytes 496 B 496 B
      Shape (62,) (62,)
      Count 2 Tasks 1 Chunks
      Type float64 numpy.ndarray
      62 1
    • lat
      (y)
      float64
      dask.array<chunksize=(44,), meta=np.ndarray>
      standard_name :
      latitude
      axis :
      Y
      long_name :
      latitude
      units :
      degrees_north
      Array Chunk
      Bytes 352 B 352 B
      Shape (44,) (44,)
      Count 2 Tasks 1 Chunks
      Type float64 numpy.ndarray
      44 1
    • y
      (y)
      float64
      50.0 50.25 50.5 ... 60.5 60.75
      standard_name :
      latitude
      axis :
      Y
      long_name :
      latitude
      units :
      degrees_north
      array([50.  , 50.25, 50.5 , 50.75, 51.  , 51.25, 51.5 , 51.75, 52.  , 52.25,
             52.5 , 52.75, 53.  , 53.25, 53.5 , 53.75, 54.  , 54.25, 54.5 , 54.75,
             55.  , 55.25, 55.5 , 55.75, 56.  , 56.25, 56.5 , 56.75, 57.  , 57.25,
             57.5 , 57.75, 58.  , 58.25, 58.5 , 58.75, 59.  , 59.25, 59.5 , 59.75,
             60.  , 60.25, 60.5 , 60.75])
    • time
      (time)
      datetime64[ns]
      2013-01-01 ... 2013-01-31T23:00:00
      standard_name :
      time
      axis :
      T
      long_name :
      time
      array(['2013-01-01T00:00:00.000000000', '2013-01-01T01:00:00.000000000',
             '2013-01-01T02:00:00.000000000', ..., '2013-01-31T21:00:00.000000000',
             '2013-01-31T22:00:00.000000000', '2013-01-31T23:00:00.000000000'],
            dtype='datetime64[ns]')
    • x
      (x)
      float64
      -13.5 -13.25 -13.0 ... 1.5 1.75
      standard_name :
      longitude
      axis :
      X
      long_name :
      longitude
      units :
      degrees_east
      array([-13.5 , -13.25, -13.  , -12.75, -12.5 , -12.25, -12.  , -11.75, -11.5 ,
             -11.25, -11.  , -10.75, -10.5 , -10.25, -10.  ,  -9.75,  -9.5 ,  -9.25,
              -9.  ,  -8.75,  -8.5 ,  -8.25,  -8.  ,  -7.75,  -7.5 ,  -7.25,  -7.  ,
              -6.75,  -6.5 ,  -6.25,  -6.  ,  -5.75,  -5.5 ,  -5.25,  -5.  ,  -4.75,
              -4.5 ,  -4.25,  -4.  ,  -3.75,  -3.5 ,  -3.25,  -3.  ,  -2.75,  -2.5 ,
              -2.25,  -2.  ,  -1.75,  -1.5 ,  -1.25,  -1.  ,  -0.75,  -0.5 ,  -0.25,
               0.  ,   0.25,   0.5 ,   0.75,   1.  ,   1.25,   1.5 ,   1.75])
    • influx_direct
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      module :
      sarah
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • influx_diffuse
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      long_name :
      Surface Diffuse Shortwave Flux
      units :
      W m-2
      module :
      sarah
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • height
      (y, x)
      float32
      dask.array<chunksize=(44, 62), meta=np.ndarray>
      module :
      era5
      Array Chunk
      Bytes 10.91 kB 10.91 kB
      Shape (44, 62) (44, 62)
      Count 2 Tasks 1 Chunks
      Type float32 numpy.ndarray
      62 44
    • wnd100m
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      units :
      m s**-1
      long_name :
      100 metre wind speed
      module :
      era5
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • roughness
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      units :
      m
      long_name :
      Forecast surface roughness
      module :
      era5
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • influx_toa
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      units :
      W m**-2
      module :
      era5
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • albedo
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      units :
      (0 - 1)
      long_name :
      Albedo
      module :
      era5
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • temperature
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      units :
      K
      long_name :
      2 metre temperature
      module :
      era5
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • soil temperature
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      units :
      K
      long_name :
      Soil temperature level 4
      module :
      era5
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
    • runoff
      (time, y, x)
      float32
      dask.array<chunksize=(100, 44, 62), meta=np.ndarray>
      module :
      era5
      Array Chunk
      Bytes 8.12 MB 1.09 MB
      Shape (744, 44, 62) (100, 44, 62)
      Count 9 Tasks 8 Chunks
      Type float32 numpy.ndarray
      62 44 744
  • module :
    ['sarah', 'era5']
    prepared_features :
    ['influx', 'runoff', 'height', 'wind', 'temperature']
    chunksize_time :
    100
    sarah_dir :
    /home/vres-climate/data/sarah_v2