Skip to content

Commit

Permalink
Update data access for deltares-floods from planetary computer
Browse files Browse the repository at this point in the history
Following the changes in microsoft/PlanetaryComputerExamples#301.
The adlfs package is now required for data access and has been added to the environment.
  • Loading branch information
chpolste committed Dec 6, 2024
1 parent 6f55f28 commit 8a34f53
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -117,21 +117,23 @@
"`````{admonition} Find more info about the libraries used in this workflow here\n",
":class: hint dropdown\n",
"In this notebook, we will use the following Python libraries:\n",
"\n",
"- [os](https://docs.python.org/3/library/os.html) - Provides a way to interact with the operating system, allowing the creation of directories and file manipulation.\n",
"- [urllib.parse](https://docs.python.org/3/library/urllib.parse.html) - URL operations.\n",
"- [numpy](https://numpy.org/) - A powerful library for numerical computations in Python, widely used for array operations and mathematical functions.\n",
"- [pandas](https://pandas.pydata.org/) - A data manipulation and analysis library, essential for working with structured data in tabular form.\n",
"- [rasterio](https://rasterio.readthedocs.io/en/stable/) - A library for reading and writing geospatial raster data, providing functionalities to explore and manipulate raster datasets.\n",
"- [xarray](https://docs.xarray.dev/en/stable/) - library for working with labelled multi-dimensional arrays.\n",
"- [rioxarray](https://corteva.github.io/rioxarray/stable/) - An extension of the xarray library that simplifies working with geospatial raster data in GeoTIFF format.\n",
"- [damagescanner](https://damagescanner.readthedocs.io/en/latest/#) - A library designed for calculating flood damages based on geospatial data, particularly suited for analyzing flood impact.\n",
"- [matplotlib](https://matplotlib.org/) - A versatile plotting library in Python, commonly used for creating static, animated, and interactive visualizations.\n",
"- [contextily](https://contextily.readthedocs.io/en/latest/) A library for adding basemaps to plots, enhancing geospatial visualizations.\n",
"- [cartopy](https://scitools.org.uk/cartopy/docs/latest/) A library for geospatial data processing.\n",
"\n",
"- [planetary-computer](https://pypi.org/project/planetary-computer/) A library for interacting with the Microsoft Planetary Computer.\n",
"- [dask](https://www.dask.org/) A library for parallel computing and task scheduling.\n",
"- [pystac-client](https://pystac-client.readthedocs.io/en/stable/) A library for for working with STAC Catalogs and APIs.\n",
"- [shapely](https://shapely.readthedocs.io/en/stable/index.html) A library for manipulation and analysis of geometric objects.\n",
"- [contextily](https://contextily.readthedocs.io/en/latest/) - A library for adding basemaps to plots, enhancing geospatial visualizations.\n",
"- [cartopy](https://scitools.org.uk/cartopy/docs/latest/) - A library for geospatial data processing.\n",
"- [planetary-computer](https://pypi.org/project/planetary-computer/) - A library for interacting with the Microsoft Planetary Computer.\n",
"- [dask](https://www.dask.org/) - A library for parallel computing and task scheduling.\n",
"- [pystac-client](https://pystac-client.readthedocs.io/en/stable/) - A library for for working with STAC Catalogs and APIs.\n",
"- [adlfs](https://github.com/fsspec/adlfs) - Interface to access data from Azure-Datalake storage.\n",
"- [shapely](https://shapely.readthedocs.io/en/stable/index.html) - A library for manipulation and analysis of geometric objects.\n",
"\n",
"These libraries collectively enable the download, processing, analysis, and visualization of geospatial and numerical data.\n",
"`````"
Expand All @@ -146,6 +148,7 @@
"source": [
"# Packages for downloading data and managing files\n",
"import os\n",
"import urllib\n",
"import pystac_client\n",
"import planetary_computer\n",
"\n",
Expand Down Expand Up @@ -302,9 +305,16 @@
"metadata": {},
"outputs": [],
"source": [
"# select first item from the search and open the dataset \n",
"# select the first item from the search and open the dataset \n",
"item = next(search.items())\n",
"url = item.assets[\"index\"].href\n",
"\n",
"# environment variables with authentication information for the data access\n",
"token = urllib.parse.urlparse(planetary_computer.sign_url(url)).query\n",
"os.environ[\"AZURE_STORAGE_SAS_TOKEN\"] = token\n",
"os.environ[\"AZURE_STORAGE_ANON\"] = \"false\"\n",
"os.environ[\"AZURE_STORAGE_ACCOUNT_NAME\"] = \"deltaresfloodssa\"\n",
"\n",
"ds = xr.open_dataset(f\"reference::{url}\", engine=\"zarr\", consolidated=False, chunks={})"
]
},
Expand Down
1 change: 1 addition & 0 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ channels:
- conda-forge

dependencies:
- adlfs=2024.7.0
- aiohttp=3.9.5
- cartopy=0.23.0
- cdsapi=0.7.0
Expand Down

0 comments on commit 8a34f53

Please sign in to comment.