Skip to content

Commit

Permalink
ADD: info regarding s3 writing and reading in main.rst
Browse files Browse the repository at this point in the history
  • Loading branch information
wolfidan committed Jan 30, 2025
1 parent 4f6d353 commit ab4f5eb
Showing 1 changed file with 10 additions and 7 deletions.
17 changes: 10 additions & 7 deletions doc/source/overview/main.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,21 +16,24 @@ locationConfigFile STRING File name (with full path) of the location config
productConfigFile STRING File name (with full path) of the product configuration file. Described in Section 4.
lastStateFile STRING File name (with full path) of the file containing the time of the last processed scan. Used in particular for real-time processing.
imgformat STRING/ File format(s) of the images. The following formats are supported: eps, png, and jpg. If ``saveimg`` is set to 0, this field is not used.
STRARR
STRARR
saveimgbasepath STRING Base directory for the images to save. The directory structure looks as follows:
``<saveimgbasepath>/<name>/<YYYY-MM-DD>/<datasetname>/<prodname>/<outputname>``
If ``saveimg`` is set to 0, this field is not used.
s3copypath STRING OPTIONAL. Path to an S3 bucket. If provided all generated products will be written there as well using the same data structure. The format must be https://bucket_name.endpoint.domain, for example https://tests.fr-par-1.linodeobjects.com/. The S3 copy procedure will only work if the
environment variables AWS_KEY and AWS_SECRET are defined in the pyrad scope. AWS_KEY contains the S3 bucket AWS key and AWS_SECRET the associated secret.
s3EndpointRead STRING OPTIONAL. Url to an S3 endpoint containing input radar data. The format must be https://endpoint.domain (e.g. https://eu-central-1.linodeobjects.com/), the https:// is not mandatory.
s3BucketRead STRING OPTIONAL. Name of an S3 bucket containing input radar data. It has to be used together with ``s3EndpointRead`` and ``s3PathRead`` to be able to read input radar data from a bucket. The procedure will only work by setting the environment variables S3_KEY_READ and S3_SECRET_READ.
s3PathRead STRING OPTIONAL. Path where to find input radar data in an S3 bucket. The data will be retrieved at url https://s3BucketRead.s3EndpointRead/s3PathRead/*
rm_s3_file INT OPTIONAL. If set input radar data files downloaded from and S3 bucket will be removed after reading.
s3EndpointWrite STRING OPTIONAL. Url to an S3 endpoint where to store output data. The format must be https://endpoint.domain (e.g. https://eu-central-1.linodeobjects.com/), the https:// is not mandatory.
s3BucketWrite STRING OPTIONAL. Name of an S3 bucket containing input radar data. It has to be used together with ``s3EndpointWrite`` and ``s3PathWrite`` to be able to save output data to a bucket. The procedure will only work by setting the environment variables S3_KEY_WRITE and S3_SECRET_WRITE.
s3PathWrite STRING OPTIONAL. Path where to save output radar data in an S3 bucket. The data will be saved at url https://s3BucketRead.s3EndpointRead/s3PathRead/filename
loadbasepath STRING OPTIONAL. Base path of saved data. By default, this field is set to ``saveimgbasepath``.
loadname STRING OPTIONAL. Name of the saved data processing. Used for saved volume loading. By default, this field is set to ``name``.
gecsxbasepath STRING OPTIONAL. Base path of saved visibility fields generated by the GECSX tool
gecsxname STRING OPTIONAL. Name of the dataset generated by GEXSX in which the visibility fields are contained. Used for GECSX volume loading.
gecsxbasepath STRING OPTIONAL. Base path of saved visibility fields generated by the GECSX tool
gecsxname STRING OPTIONAL. Name of the dataset generated by GEXSX in which the visibility fields are contained. Used for GECSX volume loading.
dempath STRING OPTIONAL. Base directory of the Digital Elevation Model (DEM) files. Basically to load the radar visibility (Optional).
smnpath STRING OPTIONAL. Base directory of the SwissMetNet stations data. Used in the comparison between radar data and rain gauges (Optional).
disdropath STRING OPTIONAL. Base directory of the disdrometer data. Used in the comparison between radar data and disdrometers (Optional).
solarfluxpath STRING OPTIONAL. Base directory of the solar flux data. Used to plot the calibration bias based on sun monitoring (Optional).
iconpath STRING OPTIONAL. Base directory of the ICON (NWP model) data files.
==================== ======= =======================================================================================


0 comments on commit ab4f5eb

Please sign in to comment.