Note

Download this example as a Jupyter notebook here: https://github.com/pypsa/atlite/examples/create_cutout_SARAH.ipynb

Creating a Cutout with the SARAH-2 dataset

This walkthrough describes the process of creating a cutout using the SARAH-2 dataset by EUMETSAT.

The SARAH-2 dataset contains extensive information on solar radiation variables, like surface incoming direct radiation (SID) or surface incoming shortwave radiation (SIS). It serves as an addition to the ERA5 dataset and as such requires the cdsapi to be setup properly.

Recommendation

This is a reduced version for cutout creation. Creating cutouts with ERA-5 is simpler and explained in more details. We therefore recommend you have a look at this example first.

Note:

For creating a cutout from this dataset, you need to download large files and your computers memory needs to be able to handle these as well.

Downloading the data set

To download the dataset, head to the EUMETSTATs website (the link points to the current 2.1 edition)

https://wui.cmsaf.eu/safira/action/viewDoiDetails?acronym=SARAH_V002_01

On the bottom, select the products you want to include in the cutout, i.e. for us:

variable	time span	time resolution
Surface incoming direct radiation (SID)	2013	Instantaneous
Surface incoming shortwave radiation (SIS)	2013	Instantaneous

• Add each product to your cart and register with the website.

• Follow the instructions to activate your account, confirm your order and wait for the download to be ready.

• You will be notified by email with the download instructions.

• Download the ordered files of your order into a directory, e.g. sarah-2.

• Extract the tar files (e.g. for linux systems tar -xvf * or with 7zip for windows) into the same folder

You are now ready to create cutouts using the SARAH-2 dataset.

Specifying the cutout

Import the package and set recommended logging settings:

import atlite

import logging
logging.basicConfig(level=logging.INFO)

Disable urllib3 and cdsapi warnings.

To download the dataset, head to the EUMETSTATs website (the link points to the current 2.1 edition)

https://wui.cmsaf.eu/safira/action/viewDoiDetails?acronym=SARAH_V002_01

On the bottom, select the products you want to include in the cutout, i.e. for us:

variable	time span	time resolution
Surface incoming direct radiation (SID)	2013	Instantaneous
Surface incoming shortwave radiation (SIS)	2013	Instantaneous

• Add each product to your cart and register with the website.

• Follow the instructions to activate your account, confirm your order and wait for the download to be ready.

• You will be notified by email with the download instructions.

• Download the ordered files of your order into a directory, e.g. sarah-2.

• Extract the tar files (e.g. for linux systems tar -xvf * or with 7zip for windows) into the same folder

You are now ready to create cutouts using the SARAH-2 dataset.

cutout = atlite.Cutout(path="western-europe-2011-01.nc",
                       module=["sarah", "era5"],
                       sarah_dir="/home/vres-climate/data/sarah_v2",
                       x=slice(-13.6913, 1.7712),
                       y=slice(49.9096, 60.8479),
                       time="2013-01",
                       chunks={'time': 100}
                       )

INFO:atlite.cutout:Building new cutout western-europe-2011-01.nc

Let’s see what the available features that is the available weather data variables are.

cutout.available_features.to_frame()

		variables
module	feature
sarah	influx	influx_direct
	influx	influx_diffuse
era5	height	height
	wind	wnd100m
	wind	roughness
	influx	influx_toa
	influx	influx_direct
	influx	influx_diffuse
	influx	albedo
	temperature	temperature
	temperature	soil temperature
	runoff	runoff

Preparing the Cutout

No matter which dataset you use, this is where all the work actually happens. This can be fast or take some or a lot of time and resources, among others depending on your computer ressources (especially memory for SARAH-2).

cutout.prepare()

INFO:atlite.data:Calculating and writing with module sarah:

[########################################] | 100% Completed |  3min 19.2s

INFO:atlite.data:Calculating and writing with module era5:
INFO:atlite.datasets.era5:Downloading data for feature 'height' to /tmp/tmpk92au7s_.
INFO:atlite.datasets.era5:Downloading data for feature 'wind' to /tmp/tmpk92au7s_.

[########################################] | 100% Completed | 11.2s

INFO:atlite.datasets.era5:Downloading data for feature 'influx' to /tmp/tmpk92au7s_.

[########################################] | 100% Completed |  4.1s

INFO:atlite.datasets.era5:Downloading data for feature 'temperature' to /tmp/tmpk92au7s_.

[########################################] | 100% Completed | 10.9s

INFO:atlite.datasets.era5:Downloading data for feature 'runoff' to /tmp/tmpk92au7s_.

[########################################] | 100% Completed | 10.8s
[########################################] | 100% Completed |  1.3s

<Cutout "western-europe-2011-01">
 x = -13.50 ⟷ 1.75, dx = 0.25
 y = 50.00 ⟷ 60.75, dy = 0.25
 time = 2013-01-01 ⟷ 2013-01-31, dt = H
 prepared_features = ['influx', 'height', 'wind', 'temperature', 'runoff']

Querying the cutout gives us some basic information on which data is contained in it an can already be used.

Inspecting the Cutout

cutout # basic information

<Cutout "western-europe-2011-01">
 x = -13.50 ⟷ 1.75, dx = 0.25
 y = 50.00 ⟷ 60.75, dy = 0.25
 time = 2013-01-01 ⟷ 2013-01-31, dt = H
 prepared_features = ['influx', 'height', 'wind', 'temperature', 'runoff']

cutout.data.attrs # cutout meta data

{'module': ['sarah', 'era5'],
 'prepared_features': ['influx', 'runoff', 'height', 'wind', 'temperature'],
 'chunksize_time': 100,
 'sarah_dir': '/home/vres-climate/data/sarah_v2'}

cutout.prepared_features # included weather variables

module  feature
sarah   influx            influx_direct
        influx           influx_diffuse
era5    height                   height
        wind                    wnd100m
        wind                  roughness
        influx               influx_toa
        influx                   albedo
        temperature         temperature
        temperature    soil temperature
        runoff                   runoff
Name: variables, dtype: object

cutout.data # access to underlying xarray data

xarray.Dataset

• Dimensions:
  • time: 744
  • x: 62
  • y: 44
• Coordinates: (5)
  • lon
    (x)
    float64
    dask.array<chunksize=(62,), meta=np.ndarray>

    standard_name :

    longitude

    axis :

    X

    long_name :

    longitude

    units :

    degrees_east

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     496 B 
     
     496 B 
        
     Shape 
     (62,) 
     
     (62,) 
        
     Count 
     2 Tasks 
     1 Chunks 
        
     Type 
     float64 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • lat
    (y)
    float64
    dask.array<chunksize=(44,), meta=np.ndarray>

    standard_name :

    latitude

    axis :

    Y

    long_name :

    latitude

    units :

    degrees_north

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     352 B 
     
     352 B 
        
     Shape 
     (44,) 
     
     (44,) 
        
     Count 
     2 Tasks 
     1 Chunks 
        
     Type 
     float64 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • y
    (y)
    float64
    50.0 50.25 50.5 ... 60.5 60.75

    standard_name :

    latitude

    axis :

    Y

    long_name :

    latitude

    units :

    degrees_north

    array([50.  , 50.25, 50.5 , 50.75, 51.  , 51.25, 51.5 , 51.75, 52.  , 52.25,
           52.5 , 52.75, 53.  , 53.25, 53.5 , 53.75, 54.  , 54.25, 54.5 , 54.75,
           55.  , 55.25, 55.5 , 55.75, 56.  , 56.25, 56.5 , 56.75, 57.  , 57.25,
           57.5 , 57.75, 58.  , 58.25, 58.5 , 58.75, 59.  , 59.25, 59.5 , 59.75,
           60.  , 60.25, 60.5 , 60.75])

  • time
    (time)
    datetime64[ns]
    2013-01-01 ... 2013-01-31T23:00:00

    standard_name :

    time

    axis :

    T

    long_name :

    time

    array(['2013-01-01T00:00:00.000000000', '2013-01-01T01:00:00.000000000',
           '2013-01-01T02:00:00.000000000', ..., '2013-01-31T21:00:00.000000000',
           '2013-01-31T22:00:00.000000000', '2013-01-31T23:00:00.000000000'],
          dtype='datetime64[ns]')

  • x
    (x)
    float64
    -13.5 -13.25 -13.0 ... 1.5 1.75

    standard_name :

    longitude

    axis :

    X

    long_name :

    longitude

    units :

    degrees_east

    array([-13.5 , -13.25, -13.  , -12.75, -12.5 , -12.25, -12.  , -11.75, -11.5 ,
           -11.25, -11.  , -10.75, -10.5 , -10.25, -10.  ,  -9.75,  -9.5 ,  -9.25,
            -9.  ,  -8.75,  -8.5 ,  -8.25,  -8.  ,  -7.75,  -7.5 ,  -7.25,  -7.  ,
            -6.75,  -6.5 ,  -6.25,  -6.  ,  -5.75,  -5.5 ,  -5.25,  -5.  ,  -4.75,
            -4.5 ,  -4.25,  -4.  ,  -3.75,  -3.5 ,  -3.25,  -3.  ,  -2.75,  -2.5 ,
            -2.25,  -2.  ,  -1.75,  -1.5 ,  -1.25,  -1.  ,  -0.75,  -0.5 ,  -0.25,
             0.  ,   0.25,   0.5 ,   0.75,   1.  ,   1.25,   1.5 ,   1.75])

• Data variables: (10)
  • influx_direct
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    module :

    sarah

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • influx_diffuse
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    long_name :

    Surface Diffuse Shortwave Flux

    units :

    W m-2

    module :

    sarah

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • height
    (y, x)
    float32
    dask.array<chunksize=(44, 62), meta=np.ndarray>

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     10.91 kB 
     
     10.91 kB 
        
     Shape 
     (44, 62) 
     
     (44, 62) 
        
     Count 
     2 Tasks 
     1 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • wnd100m
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    units :

    m s**-1

    long_name :

    100 metre wind speed

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • roughness
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    units :

    m

    long_name :

    Forecast surface roughness

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • influx_toa
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    units :

    W m**-2

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • albedo
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    units :

    (0 - 1)

    long_name :

    Albedo

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • temperature
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    units :

    K

    long_name :

    2 metre temperature

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • soil temperature
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    units :

    K

    long_name :

    Soil temperature level 4

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

  • runoff
    (time, y, x)
    float32
    dask.array<chunksize=(100, 44, 62), meta=np.ndarray>

    module :

    era5

    
    
    
    
      
    
        
     
     Array 
     Chunk 
      
      
    
        
     Bytes 
     8.12 MB 
     
     1.09 MB 
        
     Shape 
     (744, 44, 62) 
     
     (100, 44, 62) 
        
     Count 
     9 Tasks 
     8 Chunks 
        
     Type 
     float32 
     numpy.ndarray 
      
    
    
    
    
    
    
    

• Attributes: (4)

  module :

  ['sarah', 'era5']

  prepared_features :

  ['influx', 'runoff', 'height', 'wind', 'temperature']

  chunksize_time :

  100

  sarah_dir :

  /home/vres-climate/data/sarah_v2