{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Remapping the Lateral Boundary Data for ICON\n",
"---\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook serves as a template for the interpolation of lateral boundary data onto the limited area grid. \n",
"The interpolation is performed by the `Climate Data Operators (CDO)`. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The **CDO (Climate Data Operators)** are a collection of command-line operators to manipulate and analyze NetCDF and GRIB data. The CDO package is developed and maintained at the MPI for Meteorology in Hamburg. The CDO are also capable of remapping data to regular grids and triangular grids. Here, we will demonstrate how the CDO command-line tools can be used for the interpolation of the boundary datasets."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Preparations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Internally, the CDOs make use of the ECCodes package for GRIB2 decoding/encoding:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"module load eccodes\n",
"export ECCODES_DEFINITION_PATH=/pool/data/ICON/ICON_training/eccodes/definitions.edzw-2.27.0-1:$ECCODES_DEFINITION_PATH"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then, we need to specify the input and output grid (the boundary grid):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"export INGRID=\"/scratch/${USER::1}/$USER/example_data/const/icon_grid_0026_R03B07_G.nc\" # grid for which the input data has been provided\n",
"export LOCALGRID=\"/pool/data/ICON/ICON_training/exercise_lam/grids/iconR3B08_DOM01.nc\" # this is the name (and path) of the destination grid (limited area grid)"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"The following command defines the output directory:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Create directory where the pre-processed input (initial and boundary data) for the LAM run is stored\n",
"export DATADIR_LAM=/scratch/${USER::1}/$USER/data_lam\n",
"mkdir -p ${DATADIR_LAM}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"
Exercise: \n",
" Define the absolute path of the raw forcing data from the example raw data set that has been discussed in
icon_exercise_prepare_lam.ipynb.
\n",
"
\n",
"Solution
\n",
"\n",
"`export DATAPATH=\"/scratch/${USER::1}/$USER/example_data/raw_data\"`\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"Create a list of files which are contained in the forcing data directory `DATAPATH`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"cd $DATAPATH\n",
"DATAFILES=`ls forcing_*.grb`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Identical to the remapping of initial data, the correct sub-grids need to be chosen and set:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"IGN=1\n",
"TGN=2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Remapping"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can loop through the files and remap them to the target grid and change the file format to `NetCDF`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" Exercise: \n",
" Perform the remapping.\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"module load cdo\n",
"\n",
"for DATAFILE in $DATAFILES\n",
"do\n",
" # Replace .grb extension by .nc since we remap from grb to netcdf\n",
" OUTFILE=\"$DATADIR_LAM/${DATAFILE%.*}_lbc.nc\"\n",
" # Perform remapping\n",
" cdo -f nc2 remapcon,$LOCALGRID:$TGN -setgrid,$INGRID:$IGN $DATAFILE $OUTFILE.tmp\n",
" # Change name of HHL field to ICON internal z_ifc (could also be changed in the latbc_varnames_map_file)\n",
" cdo chname,HHL,z_ifc $OUTFILE.tmp $OUTFILE\n",
" rm $OUTFILE.tmp\n",
"done"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Notes:**\n",
"- ICON supports reading boundary data on an auxiliary grid which contains only the cells of the boundary zone (Tutorial Section 2.3). While CDO is in capable of remapping to such an auxiliary grid, the generation of this grid file is not possible with CDO.\n",
"- Further details on boundary data remapping with CDO can be found in Section 2.3 of the tutorial."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" Exercise: \n",
" Why did we omit the explicit specifiaction of the fields for remapping?\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"Solution
\n",
"\n",
"The `pamore` call that was used to retrieve the data (see icon_exercise_prepare_lam.ipynb) only retrieves the fields that are needed.\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" Exercise: \n",
" The cdo chname,HHL,z_ifc $OUTFILE.tmp $OUTFILE in the above remapping script generates several warnings. Do you have an idea why?\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"Solution
\n",
"\n",
"The half-level height field `HHL` is only present in the first file. Since this is a constant field, ICON can reuse this information for subsequent lateral boundary files. `pamore` takes this into account when downloading a set of boundary data files.\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
" Optional Exercise: \n",
" The remapping of multiple files could be sped up significantly by precalculating the interpolation weightings. How could this be realized?\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"module load cdo\n",
"..."
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"\n",
"Solution
\n",
"\n",
"```\n",
"module load cdo\n",
"export WEIGHTINGS=\"weightings.tmp.nc\"\n",
"\n",
"# Generate weightings\n",
"cdo -P 4 gencon,$LOCALGRID:$TGN -selgrid,$IGN $INGRID $WEIGHTINGS\n",
"\n",
"for DATAFILE in $DATAFILES\n",
"do\n",
" # Replace .grb extension by .nc since we remap from grb to netcdf\n",
" OUTFILE=\"$DATADIR_LAM/${DATAFILE%.*}_lbc.nc\"\n",
" # Perform remapping\n",
" cdo -f nc2 remap,$LOCALGRID:$TGN,$WEIGHTINGS $DATAFILE $OUTFILE\n",
" # Change name of HHL field to ICON internal z_ifc (could also be changed in the latbc_varnames_map_file)\n",
" cdo chname,HHL,z_ifc $OUTFILE $OUTFILE\n",
"done\n",
"\n",
"rm $WEIGHTINGS\n",
"```\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note:** As pointed out in the ICON Tutorial, this workflow with precalculated weightings will fail when using the data in ICON due to missing values. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Further Reading and Resources\n",
"\n",
"- **ICON Manual:** https://www.dwd.de/DE/leistungen/nwv_icon_tutorial/nwv_icon_tutorial.html
A new draft version of the ICON Tutorial is available here: https://icon-training-2025-scripts-rendering-cc74a6.gitlab-pages.dkrz.de/index.html. It is currently being finalized and will be published soon."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"\n",
"*Author info: Deutscher Wetterdienst (DWD) 2025 :: icon@dwd.de. For a full list of contributors, see CONTRIBUTING in the root directory. License info: see LICENSE file.*"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Bash",
"language": "bash",
"name": "bash"
},
"language_info": {
"codemirror_mode": "shell",
"file_extension": ".sh",
"mimetype": "text/x-sh",
"name": "bash"
}
},
"nbformat": 4,
"nbformat_minor": 4
}