Skip to content

Commit

Permalink
Release 0.13.0 cleanup.
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 569559421
Change-Id: I1c201749debee5e1a2e8367a18462aaeab68349d
  • Loading branch information
tfboyd authored and copybara-github committed Sep 29, 2023
1 parent 58f5f01 commit 7a31baa
Show file tree
Hide file tree
Showing 6 changed files with 25 additions and 31 deletions.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,13 +85,15 @@ paired with a specific version of TensorFlow. If installing Reverb as
installed. The table below lists the version of TensorFlow that each release of
Reverb is associated with and some versions of interest:

* 0.13.0 dropped Python 3.8 support.
* 0.11.0 first version to support Python 3.11.
* 0.10.0 last version to support Python 3.7.


Release | Branch / Tag | TensorFlow Version
------- | ---------------------------------------------------------- | ------------------
Nightly | [master](https://github.com/deepmind/reverb) | tf-nightly
0.13.0 | [v0.12.0](https://github.com/deepmind/reverb/tree/v0.13.0) | 2.14.0
0.12.0 | [v0.12.0](https://github.com/deepmind/reverb/tree/v0.12.0) | 2.13.0
0.11.0 | [v0.11.0](https://github.com/deepmind/reverb/tree/v0.11.0) | 2.12.0
0.10.0 | [v0.10.0](https://github.com/deepmind/reverb/tree/v0.10.0) | 2.11.0
Expand Down
6 changes: 1 addition & 5 deletions docker/release.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,10 @@ RUN ${APT_COMMAND} update && ${APT_COMMAND} install -y --no-install-recommends \
libzmq3-dev \
lsof \
pkg-config \
python3.8-dev \
python3.9-dev \
python3.10-dev \
python3.11-dev \
# python >= 3.8 needs distutils for packaging.
python3.8-distutils \
# python >= 3.9 needs distutils for packaging.
python3.9-distutils \
python3.10-distutils \
python3.11-distutils \
Expand Down Expand Up @@ -82,13 +80,11 @@ RUN for python in ${python_version}; do \
RUN rm get-pip.py

# Removes existing links so they can be created to point where we expect.
RUN rm /dt9/usr/include/x86_64-linux-gnu/python3.8
RUN rm /dt9/usr/include/x86_64-linux-gnu/python3.9
RUN rm /dt9/usr/include/x86_64-linux-gnu/python3.10
RUN rm /dt9/usr/include/x86_64-linux-gnu/python3.11

# Needed until this is included in the base TF image.
RUN ln -s "/usr/include/x86_64-linux-gnu/python3.8" "/dt9/usr/include/x86_64-linux-gnu/python3.8"
RUN ln -s "/usr/include/x86_64-linux-gnu/python3.9" "/dt9/usr/include/x86_64-linux-gnu/python3.9"
RUN ln -s "/usr/include/x86_64-linux-gnu/python3.10" "/dt9/usr/include/x86_64-linux-gnu/python3.10"
RUN ln -s "/usr/include/x86_64-linux-gnu/python3.11" "/dt9/usr/include/x86_64-linux-gnu/python3.11"
Expand Down
13 changes: 5 additions & 8 deletions oss_build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@
#
# Example usage after building release docker:
# docker run --rm -it -v ${REVERB_DIR}:/tmp/reverb tensorflow:reverb_release \
# bash oss_build.sh --python 3.8
# bash oss_build.sh --python 3.9

# Exit if any process returns non-zero status.
set -e
set -o pipefail

# Flags
PYTHON_VERSIONS=3.9 # Options 3.9 (default), 3.8, 3.10 or 3.11.
PYTHON_VERSIONS=3.9 # Options 3.9 (default), 3.10 or 3.11.
CLEAN=false # Set to true to run bazel clean.
CLEAR_CACHE=false # Set to true to delete Bazel cache folder. b/279235134
OUTPUT_DIR=/tmp/reverb/dist/
Expand All @@ -39,7 +39,7 @@ PIP_PKG_EXTRA_ARGS="" # Extra args passed to `build_pip_package`.
if [[ $# -lt 1 ]] ; then
echo "Usage:"
echo "--release [Indicates this is a release build. Otherwise nightly.]"
echo "--python [3.9(default)|3.8|3.10|3.11]"
echo "--python [3.9(default)|3.10|3.11]"
echo "--clean [true to run bazel clean]"
echo "--clear_bazel_cache [true to delete Bazel cache folder]"
echo "--tf_dep_override [Required tensorflow version to pass to setup.py."
Expand Down Expand Up @@ -103,10 +103,7 @@ for python_version in $PYTHON_VERSIONS; do
bazel clean
fi

if [ "$python_version" = "3.8" ]; then
export PYTHON_BIN_PATH=/usr/bin/python3.8 && export PYTHON_LIB_PATH=/usr/local/lib/python3.8/dist-packages
ABI=cp38
elif [ "$python_version" = "3.9" ]; then
if [ "$python_version" = "3.9" ]; then
export PYTHON_BIN_PATH=/usr/bin/python3.9 && export PYTHON_LIB_PATH=/usr/local/lib/python3.9/dist-packages
ABI=cp39
elif [ "$python_version" = "3.10" ]; then
Expand All @@ -116,7 +113,7 @@ for python_version in $PYTHON_VERSIONS; do
export PYTHON_BIN_PATH=/usr/bin/python3.11 && export PYTHON_LIB_PATH=/usr/local/lib/python3.11/dist-packages
ABI=cp311
else
echo "Error unknown --python. Only [3.8|3.9|3.10|3.11]"
echo "Error unknown --python. Only [3.9|3.10|3.11]"
exit 1
fi

Expand Down
28 changes: 14 additions & 14 deletions reverb/pip_package/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,34 +41,34 @@ Execute from the root of the git repository. The end result will end up in
##################################
# Creates the Docker container.
##################################
# Builds the container with Python 3.8, 3.9, 3.10, and 3.11. Set the
# Builds the container with Python 3.9, 3.10, and 3.11. Set the
# `build-arg tensorflow_pip` to the version of TensorFlow to build against.
# If building against an RC, use == rather than ~= for `tensorflow_pip`.
$ docker build --pull --no-cache \
--tag tensorflow:reverb_release \
--build-arg tensorflow_pip=tensorflow~=2.12.0 \
--build-arg python_version="python3.8 python3.9 python3.10 python3.11" \
--build-arg tensorflow_pip=tensorflow~=2.14.0 \
--build-arg python_version="python3.9 python3.10 python3.11" \
- < "$REVERB_DIR/docker/release.dockerfile"

#################################################
# Builds Reverb against TensorFlow stable or rc.
#################################################

# Builds Reverb against most recent stable release of TensorFlow and
# requires `tensorflow~=2.8.0` if using `pip install reverb[tensorflow]`.
# Packages for Python 3.8, 3.9, 3.10, and 3.11 are created.
# requires `tensorflow~=2.14.0` if using `pip install reverb[tensorflow]`.
# Packages for Python 3.9, 3.10, and 3.11 are created.
$ docker run --rm --mount "type=bind,src=$REVERB_DIR,dst=/tmp/reverb" \
tensorflow:reverb_release bash oss_build.sh --clean true \
--clear_bazel_cache true --tf_dep_override "tensorflow~=2.12.0" \
--release --python "3.8 3.9 3.10 3.11"
--clear_bazel_cache true --tf_dep_override "tensorflow~=2.14.0" \
--release --python "3.9 3.10 3.11"

# Builds Reverb against an RC of TensorFlow. `>=` and `~=` are not effective
# because pip does not recognize 2.4.0rc0 as greater than 2.3.0. RC builds need
# to have a strict dependency on the RC of TensorFlow used.
$ docker run --rm --mount "type=bind,src=$REVERB_DIR,dst=/tmp/reverb" \
tensorflow:reverb_release bash oss_build.sh --clean true \
--clear_bazel_cache true --tf_dep_override "tensorflow==2.12.0rc0" \
--release --python "3.8 3.9 3.10 3.11"
--clear_bazel_cache true --tf_dep_override "tensorflow==2.14.0rc0" \
--release --python "3.9 3.10 3.11"

# Builds a debug version of Reverb. The debug version is not labeled as debug
# as that can result in a user installing both the debug and regular packages
Expand All @@ -78,22 +78,22 @@ $ docker run --rm --mount "type=bind,src=$REVERB_DIR,dst=/tmp/reverb" \
$ docker run --rm --mount "type=bind,src=$REVERB_DIR,dst=/tmp/reverb" \
tensorflow:reverb_release bash oss_build.sh --clean true --debug_build true \
--clear_bazel_cache true --output_dir /tmp/reverb/dist/debug/ \
--tf_dep_override "tensorflow~=2.12.0" --release --python "3.8 3.9 3.10 3.11"
--tf_dep_override "tensorflow~=2.14.0" --release --python "3.9 3.10 3.11"

```

<a id='Develop'></a>
## Develop Reverb inside a Docker container

1. Build the Docker container. By default the container is setup for python 3.9.
Use the `python_version` arg to configure the container with 3.8 or 3.9.
Use the `python_version` arg to configure the container with 3.10 or 3.11.

```shell
$ docker build --tag tensorflow:reverb - < "$REVERB_DIR/docker/release.dockerfile"

# Alternatively you can build the container with Python 3.8 support.
# Alternatively you can build the container with Python 3.10 support.
$ docker build --tag tensorflow:reverb \
--build-arg python_version=python3.8 \
--build-arg python_version=python3.10 \
- < "$REVERB_DIR/docker/release.dockerfile"
```

Expand All @@ -109,7 +109,7 @@ $ docker run --rm --mount "type=bind,src=$REVERB_DIR,dst=/tmp/reverb" \
1. Compile Reverb.

```shell
$ python3.8 configure.py
$ python3.9 configure.py
$ bazel build -c opt //reverb/pip_package:build_pip_package
```

Expand Down
2 changes: 1 addition & 1 deletion reverb/pip_package/reverb_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

# We follow Semantic Versioning (https://semver.org/)
_MAJOR_VERSION = '0'
_MINOR_VERSION = '13'
_MINOR_VERSION = '15'
_PATCH_VERSION = '0'

# When building releases, we can update this value on the release branch to
Expand Down
5 changes: 2 additions & 3 deletions reverb/pip_package/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
import reverb_version

# Defaults if doing a release build.
TENSORFLOW_VERSION = 'tensorflow~=2.7.0'
TENSORFLOW_VERSION = 'tensorflow~=2.14.0'


class BinaryDistribution(Distribution):
Expand Down Expand Up @@ -145,7 +145,6 @@ def run_setup(self):
'Intended Audience :: Education',
'Intended Audience :: Science/Research',
'License :: OSI Approved :: Apache Software License',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
Expand Down Expand Up @@ -174,7 +173,7 @@ def run_setup(self):
type=str,
default=None,
help='Overrides TF version required when Reverb is installed, e.g.'
'tensorflow==2.5.0')
'tensorflow==2.14.0')
FLAGS, unparsed = parser.parse_known_args()
# Go forward with only non-custom flags.
sys.argv.clear()
Expand Down

0 comments on commit 7a31baa

Please sign in to comment.