{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Remapping the Initial Dataset\n", "---\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook serves as a template for the interpolation of initial data onto the limited area grid. \n", "The interpolation is performed by the `Climate Data Operators (CDO)`. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The **CDO (Climate Data Operators)** are a collection of command-line operators to manipulate and analyze NetCDF and GRIB data. The CDO package is developed and maintained at the MPI for Meteorology in Hamburg. The CDO are also capable of remapping data to regular grids and triangular grids. Here, we will demonstrate how the CDO command-line tools can be used for the interpolation of the initial datasets." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Preparations" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Internally, the CDOs make use of the ECCodes package for GRIB2 decoding/encoding:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "module load eccodes\n", "export ECCODES_DEFINITION_PATH=/pool/data/ICON/ICON_training/eccodes/definitions.edzw-2.27.0-1:$ECCODES_DEFINITION_PATH" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We need to specify the input and output grid:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "export INGRID=\"/scratch/${USER::1}/$USER/example_data/const/icon_grid_0026_R03B07_G.nc\" # grid for which the input data has been provided\n", "export LOCALGRID=\"/pool/data/ICON/ICON_training/exercise_lam/grids/iconR3B08_DOM01.nc\" # this is the name (and path) of the destination grid (limited area grid)" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "The following command defines the output directory as well as the filename containing the remapped data:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "# Create directory where the pre-processed input (initial and boundary data) for the LAM run is stored\n", "export DATADIR_LAM=/scratch/${USER::1}/$USER/data_lam\n", "mkdir -p ${DATADIR_LAM}\n", "export OUTFILE=\"$DATADIR_LAM/init_ML_20210714T000000Z.nc\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " Exercise: \n", " Define the absolute path of the raw initial data set from the example raw data set that has been discussed in icon_exercise_prepare_lam.ipynb.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "export DATAFILE=???" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "Solution\n", "\n", "`export DATAFILE=\"/scratch/${USER::1}/$USER/example_data/raw_data/init_ML_20210714T000000Z.grb\"`\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Specify the `FIELDS` to be remapped. See table 11.3 in the tutorial for further information." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "export FIELDS=ALB_SEAICE,C_T_LK,EVAP_PL,FR_ICE,FRESHSNW,H_ICE,H_ML_LK,H_SNOW,P,QC,QI,QR,QS,QV,QV_S,RHO_SNOW,SMI,T,T_BOT_LK,T_G,T_ICE,T_MNW_LK,T_SNOW,T_SO,T_WML_LK,U,V,W,W_I,W_SNOW,W_SO_ICE,Z0,HHL" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Identify the correct grid" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Technically, the ICON grid files contain sub-grids, namely cells (triangles), edges and vertices. For the remapping with `CDO`, we need to select the correct grid number in the netCDF grid definition file in order to generate the interpolation weights." ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "
\n", " Exercise: \n", " Inspect the input data grid file INGRID and the data file DATAFILE. Which grid number IGN in the grid file corresponds to the horizontal grid of the variables listed in FIELDS?\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "IGN=???" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "Solution\n", "\n", "`module load cdo`\n", "`cdo sinfov $DATAFILE`\n", "\n", "The number of the vertical grid is displayed in the 7th column, the horizontal grid in the ninth column of the output:\n", "```\n", " -1 : Institut Source T Steptype Levels Num Points Num Dtype : Parameter name\n", " 1 : DWD unknown v instant 1 1 2949120 1 P16 : FR_LAND \n", " 2 : DWD unknown v instant 1 1 2949120 1 P16 : QV_S \n", " 3 : DWD unknown v instant 1 1 2949120 1 P16 : T_G \n", " 4 : DWD unknown v instant 1 1 2949120 1 P16 : W_I \n", "```\n", "\n", "For the variables in `FIELDS`, it is thus the following grid:\n", "```\n", " 1 : unstructured : points=2949120\n", " grid : number=26 position=1\n", " uuid : a27b8de6-18c4-11e4-820a-b5b098c6a5c0\n", "```\n", "\n", "Now this grid needs to be identified in the input grid file:\n", "\n", "`cdo sinfov $INGRID`\n", "\n", "The corresponding grid with `2949120` points has number 1 in `INGRID`:\n", "\n", "```\n", " 1 : unstructured : points=2949120 nvertex=3\n", " grid : number=26 position=0\n", " uri : http://icon-downloads.mpimet.mpg.de/grids/public/icon_grid_0026_R03B07_G.nc\n", " clon : -3.141593 to 3.141593 radian\n", " clat : -1.56928 to 1.56928 radian\n", " available : cellbounds\n", " uuid : a27b8de6-18c4-11e4-820a-b5b098c6a5c0\n", "```\n", "\n", "`IGN=1`\n", "\n", "Please note that grid number 1 is also the default for the `CDO` command that will be used in the following, so in this case, an explicit specification of the grid would not be necessary. This might be different in other grid files, though.\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "\n", "
\n", " Exercise: \n", " At which position in the triangles are the corresponding coordinates located?\n", "
" ] }, { "cell_type": "raw", "metadata": {}, "source": [ "your answer" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "
\n", "Solution\n", "\n", "The dimensions `clon` and `clat` are the longitude/latitude values of the midpoints of triangle circumcircles (Section 2.1.1 of the tutorial).\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "
\n", " Exercise: \n", " Now identify the corresponding grid number TGN in target grid file.\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "TGN=???" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "### Solution" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "
\n", "Solution\n", "\n", "`module load cdo`\n", "`cdo sinfov $LOCALGRID`\n", "\n", "The grid with dimensions `clat` and `clon` has number 2:\n", "```\n", " 2 : unstructured : points=37488 nvertex=3\n", " grid : number=102 position=0\n", " clon : 0.01426315 to 0.2926627 radian\n", " clat : 0.7672772 to 1.005925 radian\n", " available : cellbounds\n", " uuid : 936f1ca2-6345-6e84-7704-52b7cb2c26a0\n", "```\n", "\n", "`TGN=2`\n", "\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Remapping\n", "\n", "We choose 1st order conservative remapping as a remapping method. Since the correct grid number of the input grid (`IGN`) and the target grid (`TGN`) have been identified, we can select these grids in the cdo command:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "module load cdo\n", "cdo -f nc2 remapcon,$LOCALGRID:$TGN -selname,$FIELDS -setgrid,$INGRID:$IGN $DATAFILE $OUTFILE" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Notes:**\n", "- We chose `NetCDF` as file format of the output file by specifying `-f nc2`. The filename `$OUTFILE` was specified accordingly.\n", "- CDO allows also to precalculate remapping weights which can significantly speed up the interpolation in case of multiple files on identical grids.\n", "- Further details on initial data remapping with CDO can be found in Section 2.2.3 of the tutorial." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Further Reading and Resources\n", "\n", "- **ICON Manual:** https://www.dwd.de/DE/leistungen/nwv_icon_tutorial/nwv_icon_tutorial.html
A new draft version of the ICON Tutorial is available here: https://icon-training-2025-scripts-rendering-cc74a6.gitlab-pages.dkrz.de/index.html. It is currently being finalized and will be published soon.\n", " - Grid file description\n", " - ICON Manual, Chapter 2\n", " - Necessary input data tables\n", " - ICON Manual, Chapter 11\n", " - Remapping initial data using CDO\n", " - ICON Manual, Section 2.2.2\n", "- CDO code and documentation: https://code.mpimet.mpg.de/projects/cdo\n", "- CDO usage examples (interpolation onto regular grids): https://www.dwd.de/DE/leistungen/opendata/help/modelle/Opendata_cdo_DE.pdf?__blob=publicationFile&v=4\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "\n", "*Author info: Deutscher Wetterdienst (DWD) 2025 :: icon@dwd.de. For a full list of contributors, see CONTRIBUTING in the root directory. License info: see LICENSE file.*" ] } ], "metadata": { "kernelspec": { "display_name": "Bash", "language": "bash", "name": "bash" }, "language_info": { "codemirror_mode": "shell", "file_extension": ".sh", "mimetype": "text/x-sh", "name": "bash" } }, "nbformat": 4, "nbformat_minor": 4 }