Fly_trainer.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mortocks/Core-audio-exersizes/blob/master/Fly_trainer.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fF8ysCfYKgTP"
      },
      "source": [
        "# TensorFlow Lite Object Detection API in Colab\n",
        "**Author:** Evan Juras, [EJ Technology Consultants](https://ejtech.io)\n",
        "\n",
        "**Last updated:** 1/28/23\n",
        "\n",
        "**GitHub:** [TensorFlow Lite Object Detection](https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi)\n",
        "\n",
        "# Introduction\n",
        "\n",
        "This notebook uses [the TensorFlow 2 Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection) to train an SSD-MobileNet model or EfficientDet model with a custom dataset and convert it to TensorFlow Lite format. By working through this Colab, you'll be able to create and download a TFLite model that you can run on your PC, an Android phone, or an edge device like the Raspberry Pi.\n",
        "\n",
        "<p align=center>\n",
        "<img src=\"https://s3.us-west-1.amazonaws.com/evanjuras.com/img/CoinDetectorDemo.gif\" height=\"360\"><br>\n",
        "<i>Custom SSD-MobileNet-FPNLite model in action!</i>\n",
        "</p>\n",
        "\n",
        "I also made a YouTube video that walks through this guide step by step. I use a coin detection model as an example for the video. I recommend following along with the video while working through this notebook.\n",
        "\n",
        "<p align=center>\n",
        "<a href=\"https://www.youtube.com/watch?v=XZ7FYAMCc4M\" target=\"_blank\"><img src=\"https://s3.us-west-1.amazonaws.com/evanjuras.com/img/thumbnail-tflite-colab-1.png\" height=\"240\"><br>\n",
        "<i>Click here to go to the video!</i></a>\n",
        "</p>\n",
        "\n",
        "**Important note: This notebook will be continuously updated to make sure it works with newer versions of TensorFlow. If you see any differences between the YouTube video and this notebook, always follow the notebook!**\n",
        "\n",
        "### Working in Colab\n",
        "Colab provides a virtual machine in your browser complete with a Linux OS, filesystem, Python environment, and best of all, a free GPU. It comes with most TensorFlow backend requirements (like CUDA and cuDNN) pre-installed. Simply click the play button on sections of code in this notebook to execute them on the virtual machine.\n",
        "\n",
        "> *Note: Make sure you're using a GPU-equipped machine by going to \"Runtime\" -> \"Change runtime type\" in the top menu bar, and then selecting \"GPU\" from the Hardware accelerator dropdown.*\n",
        "\n",
        "This Colab notebook uses TensorFlow 2. If you'd like to use TensorFlow 1, please see my [TF1 Colab notebook](https://colab.research.google.com/github/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/Train_TFLite1_Object_Detection_Model.ipynb).\n",
        "\n",
        "### Navigation\n",
        "This is a long notebook! Each step of the training process has its own section. Click the arrow next to the heading for each section to expand it. You can use the table of contents in the left sidebar to jump from section to section."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 1.&nbsp;Gather and Label Training Images"
      ],
      "metadata": {
        "id": "4VAvZo8qE4u5"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ag0qD4XiBDcz"
      },
      "source": [
        "Before we start training, we need to gather and label images that will be used for training the object detection model. A good starting point for a proof-of-concept model is 200 images. The training images should have random objects in the image along with the desired objects, and should have a variety of backgrounds and lighting conditions.\n",
        "\n",
        "Watch the YouTube video below for instructions and tips on how to gather and label images for training an object detection model.\n",
        "\n",
        "<p align=center>\n",
        "<a href=\"https://www.youtube.com/watch?v=v0ssiOY6cfg\" target=\"_blank\"><img src=\"https://s3.us-west-1.amazonaws.com/evanjuras.com/img/thumbnail-capture-label-data.png\" height=\"240\"><br>\n",
        "<i>Watch this video to learn how to capture and label images.</i></a>\n",
        "</p>\n",
        "\n",
        "When you've finished gathering and labeling images, you should have a folder full of images and corresponding .xml data annotation file for each image. An example of a labeled image and the image folder for my coin detector model are shown below.\n",
        "\n",
        "![](https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/doc/labeled_image_example2.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#2.&nbsp;Install TensorFlow Object Detection Dependencies"
      ],
      "metadata": {
        "id": "sxb8_h-QFErO"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l7EOtpvlLeS0"
      },
      "source": [
        "First, we'll install the TensorFlow Object Detection API in this Google Colab instance. This requires cloning the [TensorFlow models repository](https://github.com/tensorflow/models) and running a couple installation commands. Click the play button to run the following sections of code.\n",
        "\n",
        "The latest version of TensorFlow this Colab has been verified to work with is TF v2.8.0.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "ypWGYdPlLRUN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "68831c18-1183-461f-e409-e8031042512b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Found existing installation: Cython 3.0.5\n",
            "Uninstalling Cython-3.0.5:\n",
            "  Successfully uninstalled Cython-3.0.5\n",
            "Cloning into 'models'...\n",
            "remote: Enumerating objects: 4043, done.\u001b[K\n",
            "remote: Counting objects: 100% (4043/4043), done.\u001b[K\n",
            "remote: Compressing objects: 100% (3072/3072), done.\u001b[K\n",
            "remote: Total 4043 (delta 1185), reused 2772 (delta 914), pack-reused 0\u001b[K\n",
            "Receiving objects: 100% (4043/4043), 54.68 MiB | 28.82 MiB/s, done.\n",
            "Resolving deltas: 100% (1185/1185), done.\n",
            "Updating files: 100% (3659/3659), done.\n"
          ]
        }
      ],
      "source": [
        "# Clone the tensorflow models repository from GitHub\n",
        "!pip uninstall Cython -y # Temporary fix for \"No module named 'object_detection'\" error\n",
        "!git clone --depth 1 https://github.com/tensorflow/models"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "6QPmVBSlLTzM"
      },
      "outputs": [],
      "source": [
        "# Copy setup files into models/research folder\n",
        "%%bash\n",
        "cd models/research/\n",
        "protoc object_detection/protos/*.proto --python_out=.\n",
        "#cp object_detection/packages/tf2/setup.py ."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Modify setup.py file to install the tf-models-official repository targeted at TF v2.8.0\n",
        "import re\n",
        "with open('/content/models/research/object_detection/packages/tf2/setup.py') as f:\n",
        "    s = f.read()\n",
        "\n",
        "with open('/content/models/research/setup.py', 'w') as f:\n",
        "    # Set fine_tune_checkpoint path\n",
        "    s = re.sub('tf-models-official>=2.5.1',\n",
        "               'tf-models-official==2.8.0', s)\n",
        "    f.write(s)"
      ],
      "metadata": {
        "id": "NRBnuCKjM4Bd"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "OLDnCkLLwLr6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "09d96a93-dc24-41e6-9b18-8bb73fd9262a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting pyyaml==5.3\n",
            "  Downloading PyYAML-5.3.tar.gz (268 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.2/268.2 kB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Building wheels for collected packages: pyyaml\n",
            "  Building wheel for pyyaml (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pyyaml: filename=PyYAML-5.3-cp310-cp310-linux_x86_64.whl size=44243 sha256=98feea2f794cce08d89d56014e1c4ee8064070b88fd555ab778f9d7b1eee7b6f\n",
            "  Stored in directory: /root/.cache/pip/wheels/0d/72/68/a263cfc14175636cf26bada99f13b735be1b60a11318e08bfc\n",
            "Successfully built pyyaml\n",
            "Installing collected packages: pyyaml\n",
            "  Attempting uninstall: pyyaml\n",
            "    Found existing installation: PyYAML 6.0.1\n",
            "    Uninstalling PyYAML-6.0.1:\n",
            "      Successfully uninstalled PyYAML-6.0.1\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "llmx 0.0.15a0 requires openai, which is not installed.\n",
            "llmx 0.0.15a0 requires tiktoken, which is not installed.\n",
            "dask 2023.8.1 requires pyyaml>=5.3.1, but you have pyyaml 5.3 which is incompatible.\n",
            "distributed 2023.8.1 requires pyyaml>=5.3.1, but you have pyyaml 5.3 which is incompatible.\n",
            "flax 0.7.5 requires PyYAML>=5.4.1, but you have pyyaml 5.3 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed pyyaml-5.3\n",
            "Processing ./models/research\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting avro-python3 (from object-detection==0.1)\n",
            "  Downloading avro-python3-1.10.2.tar.gz (38 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting apache-beam (from object-detection==0.1)\n",
            "  Downloading apache_beam-2.51.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.7 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m14.7/14.7 MB\u001b[0m \u001b[31m27.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pillow in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (9.4.0)\n",
            "Requirement already satisfied: lxml in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (4.9.3)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (3.7.1)\n",
            "Collecting Cython (from object-detection==0.1)\n",
            "  Downloading Cython-3.0.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.6/3.6 MB\u001b[0m \u001b[31m57.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: contextlib2 in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (21.6.0)\n",
            "Requirement already satisfied: tf-slim in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (1.1.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (1.16.0)\n",
            "Requirement already satisfied: pycocotools in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (2.0.7)\n",
            "Collecting lvis (from object-detection==0.1)\n",
            "  Downloading lvis-0.5.3-py3-none-any.whl (14 kB)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (1.11.3)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (1.5.3)\n",
            "Collecting tf-models-official==2.8.0 (from object-detection==0.1)\n",
            "  Downloading tf_models_official-2.8.0-py2.py3-none-any.whl (2.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.2/2.2 MB\u001b[0m \u001b[31m51.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tensorflow_io (from object-detection==0.1)\n",
            "  Downloading tensorflow_io-0.34.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (28.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m28.8/28.8 MB\u001b[0m \u001b[31m57.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: keras in /usr/local/lib/python3.10/dist-packages (from object-detection==0.1) (2.14.0)\n",
            "Collecting pyparsing==2.4.7 (from object-detection==0.1)\n",
            "  Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.8/67.8 kB\u001b[0m \u001b[31m10.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting sacrebleu<=2.2.0 (from object-detection==0.1)\n",
            "  Downloading sacrebleu-2.2.0-py3-none-any.whl (116 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.6/116.6 kB\u001b[0m \u001b[31m16.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: gin-config in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (0.5.0)\n",
            "Requirement already satisfied: google-api-python-client>=1.6.7 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (2.84.0)\n",
            "Requirement already satisfied: kaggle>=1.3.9 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (1.5.16)\n",
            "Requirement already satisfied: numpy>=1.15.4 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (1.23.5)\n",
            "Requirement already satisfied: oauth2client in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (4.1.3)\n",
            "Requirement already satisfied: opencv-python-headless in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (4.8.1.78)\n",
            "Requirement already satisfied: psutil>=5.4.3 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (5.9.5)\n",
            "Requirement already satisfied: py-cpuinfo>=3.3.0 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (9.0.0)\n",
            "Requirement already satisfied: pyyaml<6.0,>=5.1 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (5.3)\n",
            "Collecting sentencepiece (from tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m84.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting seqeval (from tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading seqeval-1.2.2.tar.gz (43 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.6/43.6 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting tensorflow-addons (from tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorflow_addons-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (612 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m612.3/612.3 kB\u001b[0m \u001b[31m48.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tensorflow-datasets in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (4.9.3)\n",
            "Requirement already satisfied: tensorflow-hub>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from tf-models-official==2.8.0->object-detection==0.1) (0.15.0)\n",
            "Collecting tensorflow-model-optimization>=0.4.1 (from tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorflow_model_optimization-0.7.5-py2.py3-none-any.whl (241 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m241.2/241.2 kB\u001b[0m \u001b[31m16.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tensorflow-text~=2.8.0 (from tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorflow_text-2.8.2-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.9 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.9/4.9 MB\u001b[0m \u001b[31m84.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tensorflow~=2.8.0 (from tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorflow-2.8.4-cp310-cp310-manylinux2010_x86_64.whl (498.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m498.1/498.1 MB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->object-detection==0.1) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->object-detection==0.1) (2023.3.post1)\n",
            "Collecting portalocker (from sacrebleu<=2.2.0->object-detection==0.1)\n",
            "  Downloading portalocker-2.8.2-py3-none-any.whl (17 kB)\n",
            "Requirement already satisfied: regex in /usr/local/lib/python3.10/dist-packages (from sacrebleu<=2.2.0->object-detection==0.1) (2023.6.3)\n",
            "Requirement already satisfied: tabulate>=0.8.9 in /usr/local/lib/python3.10/dist-packages (from sacrebleu<=2.2.0->object-detection==0.1) (0.9.0)\n",
            "Collecting colorama (from sacrebleu<=2.2.0->object-detection==0.1)\n",
            "  Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\n",
            "Requirement already satisfied: absl-py>=0.2.2 in /usr/local/lib/python3.10/dist-packages (from tf-slim->object-detection==0.1) (1.4.0)\n",
            "Collecting crcmod<2.0,>=1.7 (from apache-beam->object-detection==0.1)\n",
            "  Downloading crcmod-1.7.tar.gz (89 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m89.7/89.7 kB\u001b[0m \u001b[31m12.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting orjson<4,>=3.9.7 (from apache-beam->object-detection==0.1)\n",
            "  Downloading orjson-3.9.10-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m19.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting dill<0.3.2,>=0.3.1.1 (from apache-beam->object-detection==0.1)\n",
            "  Downloading dill-0.3.1.1.tar.gz (151 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m152.0/152.0 kB\u001b[0m \u001b[31m22.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: cloudpickle~=2.2.1 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (2.2.1)\n",
            "Collecting fastavro<2,>=0.23.6 (from apache-beam->object-detection==0.1)\n",
            "  Downloading fastavro-1.9.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m108.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting fasteners<1.0,>=0.3 (from apache-beam->object-detection==0.1)\n",
            "  Downloading fasteners-0.19-py3-none-any.whl (18 kB)\n",
            "Requirement already satisfied: grpcio!=1.48.0,<2,>=1.33.1 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (1.59.2)\n",
            "Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam->object-detection==0.1)\n",
            "  Downloading hdfs-2.7.3.tar.gz (43 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.5/43.5 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: httplib2<0.23.0,>=0.8 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (0.22.0)\n",
            "Collecting js2py<1,>=0.74 (from apache-beam->object-detection==0.1)\n",
            "  Downloading Js2Py-0.74-py3-none-any.whl (1.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.0/1.0 MB\u001b[0m \u001b[31m71.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting objsize<0.7.0,>=0.6.1 (from apache-beam->object-detection==0.1)\n",
            "  Downloading objsize-0.6.1-py3-none-any.whl (9.3 kB)\n",
            "Requirement already satisfied: packaging>=22.0 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (23.2)\n",
            "Collecting pymongo<5.0.0,>=3.8.0 (from apache-beam->object-detection==0.1)\n",
            "  Downloading pymongo-4.6.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (677 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m677.1/677.1 kB\u001b[0m \u001b[31m51.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: proto-plus<2,>=1.7.1 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (1.22.3)\n",
            "Requirement already satisfied: protobuf!=4.0.*,!=4.21.*,!=4.22.0,!=4.23.*,!=4.24.0,!=4.24.1,!=4.24.2,<4.25.0,>=3.20.3 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (3.20.3)\n",
            "Requirement already satisfied: pydot<2,>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (1.4.2)\n",
            "Requirement already satisfied: requests<3.0.0,>=2.24.0 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (2.31.0)\n",
            "Requirement already satisfied: typing-extensions>=3.7.0 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (4.5.0)\n",
            "Collecting zstandard<1,>=0.18.0 (from apache-beam->object-detection==0.1)\n",
            "  Downloading zstandard-0.22.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.4/5.4 MB\u001b[0m \u001b[31m80.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pyarrow<12.0.0,>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from apache-beam->object-detection==0.1) (9.0.0)\n",
            "Requirement already satisfied: cycler>=0.10.0 in /usr/local/lib/python3.10/dist-packages (from lvis->object-detection==0.1) (0.12.1)\n",
            "Requirement already satisfied: kiwisolver>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from lvis->object-detection==0.1) (1.4.5)\n",
            "Requirement already satisfied: opencv-python>=4.1.0.25 in /usr/local/lib/python3.10/dist-packages (from lvis->object-detection==0.1) (4.8.0.76)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->object-detection==0.1) (1.2.0)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->object-detection==0.1) (4.44.0)\n",
            "Requirement already satisfied: tensorflow-io-gcs-filesystem==0.34.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow_io->object-detection==0.1) (0.34.0)\n",
            "Requirement already satisfied: google-auth<3.0.0dev,>=1.19.0 in /usr/local/lib/python3.10/dist-packages (from google-api-python-client>=1.6.7->tf-models-official==2.8.0->object-detection==0.1) (2.17.3)\n",
            "Requirement already satisfied: google-auth-httplib2>=0.1.0 in /usr/local/lib/python3.10/dist-packages (from google-api-python-client>=1.6.7->tf-models-official==2.8.0->object-detection==0.1) (0.1.1)\n",
            "Requirement already satisfied: google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5 in /usr/local/lib/python3.10/dist-packages (from google-api-python-client>=1.6.7->tf-models-official==2.8.0->object-detection==0.1) (2.11.1)\n",
            "Requirement already satisfied: uritemplate<5,>=3.0.1 in /usr/local/lib/python3.10/dist-packages (from google-api-python-client>=1.6.7->tf-models-official==2.8.0->object-detection==0.1) (4.1.1)\n",
            "Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam->object-detection==0.1)\n",
            "  Downloading docopt-0.6.2.tar.gz (25 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: tzlocal>=1.2 in /usr/local/lib/python3.10/dist-packages (from js2py<1,>=0.74->apache-beam->object-detection==0.1) (5.2)\n",
            "Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam->object-detection==0.1)\n",
            "  Downloading pyjsparser-2.7.1.tar.gz (24 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from kaggle>=1.3.9->tf-models-official==2.8.0->object-detection==0.1) (2023.7.22)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from kaggle>=1.3.9->tf-models-official==2.8.0->object-detection==0.1) (4.66.1)\n",
            "Requirement already satisfied: python-slugify in /usr/local/lib/python3.10/dist-packages (from kaggle>=1.3.9->tf-models-official==2.8.0->object-detection==0.1) (8.0.1)\n",
            "Requirement already satisfied: urllib3 in /usr/local/lib/python3.10/dist-packages (from kaggle>=1.3.9->tf-models-official==2.8.0->object-detection==0.1) (2.0.7)\n",
            "Requirement already satisfied: bleach in /usr/local/lib/python3.10/dist-packages (from kaggle>=1.3.9->tf-models-official==2.8.0->object-detection==0.1) (6.1.0)\n",
            "Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam->object-detection==0.1)\n",
            "  Downloading dnspython-2.4.2-py3-none-any.whl (300 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m300.4/300.4 kB\u001b[0m \u001b[31m32.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.24.0->apache-beam->object-detection==0.1) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3.0.0,>=2.24.0->apache-beam->object-detection==0.1) (3.4)\n",
            "Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (1.6.3)\n",
            "Requirement already satisfied: flatbuffers>=1.12 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (23.5.26)\n",
            "Requirement already satisfied: gast>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (0.5.4)\n",
            "Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (0.2.0)\n",
            "Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (3.9.0)\n",
            "Collecting keras-preprocessing>=1.1.1 (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.6/42.6 kB\u001b[0m \u001b[31m5.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: libclang>=9.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (16.0.6)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (3.3.0)\n",
            "INFO: pip is looking at multiple versions of tensorflow to determine which version is compatible with other requirements. This could take a while.\n",
            "Collecting tensorflow~=2.8.0 (from tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorflow-2.8.3-cp310-cp310-manylinux2010_x86_64.whl (498.5 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m498.5/498.5 MB\u001b[0m \u001b[31m2.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading tensorflow-2.8.2-cp310-cp310-manylinux2010_x86_64.whl (498.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m498.0/498.0 MB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading tensorflow-2.8.1-cp310-cp310-manylinux2010_x86_64.whl (498.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m498.0/498.0 MB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (67.7.2)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (2.3.0)\n",
            "Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (1.14.1)\n",
            "Collecting tensorboard<2.9,>=2.8 (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorboard-2.8.0-py3-none-any.whl (5.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.8/5.8 MB\u001b[0m \u001b[31m80.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tensorflow-estimator<2.9,>=2.8 (from tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorflow_estimator-2.8.0-py2.py3-none-any.whl (462 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.3/462.3 kB\u001b[0m \u001b[31m39.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting keras (from object-detection==0.1)\n",
            "  Downloading keras-2.8.0-py2.py3-none-any.whl (1.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.4/1.4 MB\u001b[0m \u001b[31m67.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: dm-tree~=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow-model-optimization>=0.4.1->tf-models-official==2.8.0->object-detection==0.1) (0.1.8)\n",
            "Requirement already satisfied: pyasn1>=0.1.7 in /usr/local/lib/python3.10/dist-packages (from oauth2client->tf-models-official==2.8.0->object-detection==0.1) (0.5.0)\n",
            "Requirement already satisfied: pyasn1-modules>=0.0.5 in /usr/local/lib/python3.10/dist-packages (from oauth2client->tf-models-official==2.8.0->object-detection==0.1) (0.3.0)\n",
            "Requirement already satisfied: rsa>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from oauth2client->tf-models-official==2.8.0->object-detection==0.1) (4.9)\n",
            "Requirement already satisfied: scikit-learn>=0.21.3 in /usr/local/lib/python3.10/dist-packages (from seqeval->tf-models-official==2.8.0->object-detection==0.1) (1.2.2)\n",
            "Collecting typeguard<3.0.0,>=2.7 (from tensorflow-addons->tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading typeguard-2.13.3-py3-none-any.whl (17 kB)\n",
            "Requirement already satisfied: array-record in /usr/local/lib/python3.10/dist-packages (from tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (0.5.0)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (8.1.7)\n",
            "Requirement already satisfied: etils[enp,epath,etree]>=0.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (1.5.2)\n",
            "Requirement already satisfied: promise in /usr/local/lib/python3.10/dist-packages (from tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (2.3)\n",
            "Requirement already satisfied: tensorflow-metadata in /usr/local/lib/python3.10/dist-packages (from tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (1.14.0)\n",
            "Requirement already satisfied: toml in /usr/local/lib/python3.10/dist-packages (from tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (0.10.2)\n",
            "Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (0.41.3)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from etils[enp,epath,etree]>=0.9.0->tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (2023.6.0)\n",
            "Requirement already satisfied: importlib_resources in /usr/local/lib/python3.10/dist-packages (from etils[enp,epath,etree]>=0.9.0->tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (6.1.1)\n",
            "Requirement already satisfied: zipp in /usr/local/lib/python3.10/dist-packages (from etils[enp,epath,etree]>=0.9.0->tensorflow-datasets->tf-models-official==2.8.0->object-detection==0.1) (3.17.0)\n",
            "Requirement already satisfied: googleapis-common-protos<2.0.dev0,>=1.56.2 in /usr/local/lib/python3.10/dist-packages (from google-api-core!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-api-python-client>=1.6.7->tf-models-official==2.8.0->object-detection==0.1) (1.61.0)\n",
            "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3.0.0dev,>=1.19.0->google-api-python-client>=1.6.7->tf-models-official==2.8.0->object-detection==0.1) (5.3.2)\n",
            "Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=0.21.3->seqeval->tf-models-official==2.8.0->object-detection==0.1) (1.3.2)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn>=0.21.3->seqeval->tf-models-official==2.8.0->object-detection==0.1) (3.2.0)\n",
            "Collecting google-auth-oauthlib<0.5,>=0.4.1 (from tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (3.5.1)\n",
            "Collecting tensorboard-data-server<0.7.0,>=0.6.0 (from tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorboard_data_server-0.6.1-py3-none-manylinux2010_x86_64.whl (4.9 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.9/4.9 MB\u001b[0m \u001b[31m89.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting tensorboard-plugin-wit>=1.6.0 (from tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1)\n",
            "  Downloading tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m781.3/781.3 kB\u001b[0m \u001b[31m57.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (3.0.1)\n",
            "Requirement already satisfied: webencodings in /usr/local/lib/python3.10/dist-packages (from bleach->kaggle>=1.3.9->tf-models-official==2.8.0->object-detection==0.1) (0.5.1)\n",
            "Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.10/dist-packages (from python-slugify->kaggle>=1.3.9->tf-models-official==2.8.0->object-detection==0.1) (1.3)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (1.3.1)\n",
            "Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=0.11.15->tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (2.1.3)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow~=2.8.0->tf-models-official==2.8.0->object-detection==0.1) (3.2.2)\n",
            "Building wheels for collected packages: object-detection, avro-python3, crcmod, dill, hdfs, seqeval, pyjsparser, docopt\n",
            "  Building wheel for object-detection (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for object-detection: filename=object_detection-0.1-py3-none-any.whl size=1697350 sha256=a5a5a487ed4428f611ca40ffce91c2c7da63dd66eec5b17600015436704df608\n",
            "  Stored in directory: /tmp/pip-ephem-wheel-cache-z94zq8sx/wheels/53/dd/70/2de274d6c443c69d367bd6a5606f95e5a6df61aacf1435ec0d\n",
            "  Building wheel for avro-python3 (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for avro-python3: filename=avro_python3-1.10.2-py3-none-any.whl size=43993 sha256=dabd6a9816214296a04b406b3301fdd1a5a16e457b47fbb25a5211eff57767c3\n",
            "  Stored in directory: /root/.cache/pip/wheels/bc/85/62/6cdd81c56f923946b401cecff38055b94c9b766927f7d8ca82\n",
            "  Building wheel for crcmod (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for crcmod: filename=crcmod-1.7-cp310-cp310-linux_x86_64.whl size=31407 sha256=33f6275deb991ca367f6434ba68ec1862d0375f1579e0194f923c6cfe5ebfc39\n",
            "  Stored in directory: /root/.cache/pip/wheels/85/4c/07/72215c529bd59d67e3dac29711d7aba1b692f543c808ba9e86\n",
            "  Building wheel for dill (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for dill: filename=dill-0.3.1.1-py3-none-any.whl size=78542 sha256=22afb0d9b3a872f589482def647b8cb313636c21363e87307a9a097031d0581c\n",
            "  Stored in directory: /root/.cache/pip/wheels/ea/e2/86/64980d90e297e7bf2ce588c2b96e818f5399c515c4bb8a7e4f\n",
            "  Building wheel for hdfs (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for hdfs: filename=hdfs-2.7.3-py3-none-any.whl size=34325 sha256=a944c60e8eb977410abc71a8711d2a8ad32628f512009ca097b5b03cc64b66f3\n",
            "  Stored in directory: /root/.cache/pip/wheels/e5/8d/b6/99c1c0a3ac5788c866b0ecd3f48b0134a5910e6ed26011800b\n",
            "  Building wheel for seqeval (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for seqeval: filename=seqeval-1.2.2-py3-none-any.whl size=16161 sha256=ceb81bffd9d75e088a6a40f0298e33e8f152883832fbe71ba91ce7ec899d797a\n",
            "  Stored in directory: /root/.cache/pip/wheels/1a/67/4a/ad4082dd7dfc30f2abfe4d80a2ed5926a506eb8a972b4767fa\n",
            "  Building wheel for pyjsparser (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pyjsparser: filename=pyjsparser-2.7.1-py3-none-any.whl size=25985 sha256=5a9ea177196219799d565b471dc3028dfbb9f3b35585d1d0420519ae9ff95903\n",
            "  Stored in directory: /root/.cache/pip/wheels/5e/81/26/5956478df303e2bf5a85a5df595bb307bd25948a4bab69f7c7\n",
            "  Building wheel for docopt (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13706 sha256=62184b1606e9b979fe214eab4bdbd4bccddcf999881fd22e8d7e15ea98fc6b59\n",
            "  Stored in directory: /root/.cache/pip/wheels/fc/ab/d4/5da2067ac95b36618c629a5f93f809425700506f72c9732fac\n",
            "Successfully built object-detection avro-python3 crcmod dill hdfs seqeval pyjsparser docopt\n",
            "Installing collected packages: tensorflow-estimator, tensorboard-plugin-wit, sentencepiece, pyjsparser, keras, docopt, crcmod, zstandard, typeguard, tensorflow-model-optimization, tensorflow_io, tensorboard-data-server, pyparsing, portalocker, orjson, objsize, keras-preprocessing, js2py, fasteners, fastavro, dnspython, dill, Cython, colorama, avro-python3, tensorflow-addons, sacrebleu, pymongo, hdfs, seqeval, lvis, google-auth-oauthlib, apache-beam, tensorboard, tensorflow, tensorflow-text, tf-models-official, object-detection\n",
            "  Attempting uninstall: tensorflow-estimator\n",
            "    Found existing installation: tensorflow-estimator 2.14.0\n",
            "    Uninstalling tensorflow-estimator-2.14.0:\n",
            "      Successfully uninstalled tensorflow-estimator-2.14.0\n",
            "  Attempting uninstall: keras\n",
            "    Found existing installation: keras 2.14.0\n",
            "    Uninstalling keras-2.14.0:\n",
            "      Successfully uninstalled keras-2.14.0\n",
            "  Attempting uninstall: tensorboard-data-server\n",
            "    Found existing installation: tensorboard-data-server 0.7.2\n",
            "    Uninstalling tensorboard-data-server-0.7.2:\n",
            "      Successfully uninstalled tensorboard-data-server-0.7.2\n",
            "  Attempting uninstall: pyparsing\n",
            "    Found existing installation: pyparsing 3.1.1\n",
            "    Uninstalling pyparsing-3.1.1:\n",
            "      Successfully uninstalled pyparsing-3.1.1\n",
            "  Attempting uninstall: google-auth-oauthlib\n",
            "    Found existing installation: google-auth-oauthlib 1.0.0\n",
            "    Uninstalling google-auth-oauthlib-1.0.0:\n",
            "      Successfully uninstalled google-auth-oauthlib-1.0.0\n",
            "  Attempting uninstall: tensorboard\n",
            "    Found existing installation: tensorboard 2.14.1\n",
            "    Uninstalling tensorboard-2.14.1:\n",
            "      Successfully uninstalled tensorboard-2.14.1\n",
            "  Attempting uninstall: tensorflow\n",
            "    Found existing installation: tensorflow 2.14.0\n",
            "    Uninstalling tensorflow-2.14.0:\n",
            "      Successfully uninstalled tensorflow-2.14.0\n",
            "Successfully installed Cython-3.0.5 apache-beam-2.51.0 avro-python3-1.10.2 colorama-0.4.6 crcmod-1.7 dill-0.3.1.1 dnspython-2.4.2 docopt-0.6.2 fastavro-1.9.0 fasteners-0.19 google-auth-oauthlib-0.4.6 hdfs-2.7.3 js2py-0.74 keras-2.8.0 keras-preprocessing-1.1.2 lvis-0.5.3 object-detection-0.1 objsize-0.6.1 orjson-3.9.10 portalocker-2.8.2 pyjsparser-2.7.1 pymongo-4.6.0 pyparsing-2.4.7 sacrebleu-2.2.0 sentencepiece-0.1.99 seqeval-1.2.2 tensorboard-2.8.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-2.8.1 tensorflow-addons-0.22.0 tensorflow-estimator-2.8.0 tensorflow-model-optimization-0.7.5 tensorflow-text-2.8.2 tensorflow_io-0.34.0 tf-models-official-2.8.0 typeguard-2.13.3 zstandard-0.22.0\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.colab-display-data+json": {
              "pip_warning": {
                "packages": [
                  "pyparsing"
                ]
              }
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting tensorflow==2.8.0\n",
            "  Downloading tensorflow-2.8.0-cp310-cp310-manylinux2010_x86_64.whl (497.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m497.6/497.6 MB\u001b[0m \u001b[31m3.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: absl-py>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (1.4.0)\n",
            "Requirement already satisfied: astunparse>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (1.6.3)\n",
            "Requirement already satisfied: flatbuffers>=1.12 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (23.5.26)\n",
            "Requirement already satisfied: gast>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (0.5.4)\n",
            "Requirement already satisfied: google-pasta>=0.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (0.2.0)\n",
            "Requirement already satisfied: h5py>=2.9.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (3.9.0)\n",
            "Requirement already satisfied: keras-preprocessing>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (1.1.2)\n",
            "Requirement already satisfied: libclang>=9.0.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (16.0.6)\n",
            "Requirement already satisfied: numpy>=1.20 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (1.23.5)\n",
            "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (3.3.0)\n",
            "Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (3.20.3)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (67.7.2)\n",
            "Requirement already satisfied: six>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (1.16.0)\n",
            "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (2.3.0)\n",
            "Requirement already satisfied: typing-extensions>=3.6.6 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (4.5.0)\n",
            "Requirement already satisfied: wrapt>=1.11.0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (1.14.1)\n",
            "Requirement already satisfied: tensorboard<2.9,>=2.8 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (2.8.0)\n",
            "Collecting tf-estimator-nightly==2.8.0.dev2021122109 (from tensorflow==2.8.0)\n",
            "  Downloading tf_estimator_nightly-2.8.0.dev2021122109-py2.py3-none-any.whl (462 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m462.5/462.5 kB\u001b[0m \u001b[31m40.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: keras<2.9,>=2.8.0rc0 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (2.8.0)\n",
            "Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (0.34.0)\n",
            "Requirement already satisfied: grpcio<2.0,>=1.24.3 in /usr/local/lib/python3.10/dist-packages (from tensorflow==2.8.0) (1.59.2)\n",
            "Requirement already satisfied: wheel<1.0,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from astunparse>=1.6.0->tensorflow==2.8.0) (0.41.3)\n",
            "Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow==2.8.0) (2.17.3)\n",
            "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow==2.8.0) (0.4.6)\n",
            "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow==2.8.0) (3.5.1)\n",
            "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow==2.8.0) (2.31.0)\n",
            "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow==2.8.0) (0.6.1)\n",
            "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow==2.8.0) (1.8.1)\n",
            "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.10/dist-packages (from tensorboard<2.9,>=2.8->tensorflow==2.8.0) (3.0.1)\n",
            "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (5.3.2)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (0.3.0)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (4.9)\n",
            "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.10/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (1.3.1)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (3.3.2)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (3.4)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (2.0.7)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (2023.7.22)\n",
            "Requirement already satisfied: MarkupSafe>=2.1.1 in /usr/local/lib/python3.10/dist-packages (from werkzeug>=0.11.15->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (2.1.3)\n",
            "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (0.5.0)\n",
            "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.10/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow==2.8.0) (3.2.2)\n",
            "Installing collected packages: tf-estimator-nightly, tensorflow\n",
            "  Attempting uninstall: tensorflow\n",
            "    Found existing installation: tensorflow 2.8.1\n",
            "    Uninstalling tensorflow-2.8.1:\n",
            "      Successfully uninstalled tensorflow-2.8.1\n",
            "Successfully installed tensorflow-2.8.0 tf-estimator-nightly-2.8.0.dev2021122109\n"
          ]
        }
      ],
      "source": [
        "# Install the Object Detection API\n",
        "# Need to do a temporary fix with PyYAML because Colab isn't able to install PyYAML v5.4.1\n",
        "!pip install pyyaml==5.3\n",
        "!pip install /content/models/research/\n",
        "\n",
        "# Need to downgrade to TF v2.8.0 due to Colab compatibility bug with TF v2.10 (as of 10/03/22)\n",
        "!pip install tensorflow==2.8.0"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6V7TrfUos-9E"
      },
      "source": [
        "You may get warnings or errors related to package dependencies in the previous code block, but you can ignore them for now.\n",
        "\n",
        "Let's test our installation by running `model_builder_tf2_test.py` to make sure everything is working as expected. Run the following code block and confirm that it finishes without errors. If you get errors, try Googling them or checking the FAQ at the end of this Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "wh_HPMOqWH9z",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ba3dabd6-66d0-4014-edb2-9e0ba1a299b1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/__init__.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so']\n",
            "caused by: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl5mutexC1Ev']\n",
            "  warnings.warn(f\"unable to load libtensorflow_io_plugins.so: {e}\")\n",
            "/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/__init__.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so']\n",
            "caused by: ['/usr/local/lib/python3.10/dist-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZNK10tensorflow4data11DatasetBase8FinalizeEPNS_15OpKernelContextESt8functionIFN3tsl8StatusOrINS5_4core11RefCountPtrIS1_EEEEvEE']\n",
            "  warnings.warn(f\"file system plugins are not loaded: {e}\")\n",
            "Running tests under Python 3.10.12: /usr/bin/python3\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_center_net_deepmac\n",
            "2023-11-13 23:10:58.058543: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
            "W1113 23:10:58.414186 137329057406976 model_builder.py:1112] Building experimental DeepMAC meta-arch. Some features may be omitted.\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_deepmac): 2.61s\n",
            "I1113 23:10:59.075690 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_deepmac): 2.61s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_center_net_deepmac\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True)\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True)): 0.95s\n",
            "I1113 23:11:00.028641 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True)): 0.95s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_center_net_model0 (customize_head_params=True)\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False)\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False)): 0.49s\n",
            "I1113 23:11:00.524573 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False)): 0.49s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_center_net_model1 (customize_head_params=False)\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_center_net_model_from_keypoints\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model_from_keypoints): 0.5s\n",
            "I1113 23:11:01.022010 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model_from_keypoints): 0.5s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_center_net_model_from_keypoints\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_center_net_model_mobilenet\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_center_net_model_mobilenet): 4.81s\n",
            "I1113 23:11:05.828544 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_center_net_model_mobilenet): 4.81s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_center_net_model_mobilenet\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_experimental_model\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_experimental_model): 0.0s\n",
            "I1113 23:11:05.849453 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_experimental_model): 0.0s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_experimental_model\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature0 (True)\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature0 (True)): 0.13s\n",
            "I1113 23:11:05.981982 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature0 (True)): 0.13s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature0 (True)\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature1 (False)\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature1 (False)): 0.09s\n",
            "I1113 23:11:06.072928 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature1 (False)): 0.09s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_faster_rcnn_from_config_with_crop_feature1 (False)\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_faster_rcnn_model_from_config_with_example_miner\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_model_from_config_with_example_miner): 0.1s\n",
            "I1113 23:11:06.171965 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_model_from_config_with_example_miner): 0.1s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_faster_rcnn_model_from_config_with_example_miner\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul): 0.62s\n",
            "I1113 23:11:06.795512 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul): 0.62s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_with_matmul\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul): 0.55s\n",
            "I1113 23:11:07.346223 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul): 0.55s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_faster_rcnn_without_matmul\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul): 0.6s\n",
            "I1113 23:11:07.952633 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul): 0.6s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_with_matmul\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul): 0.47s\n",
            "I1113 23:11:08.427026 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul): 0.47s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_faster_rcnn_models_from_config_mask_rcnn_without_matmul\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_rfcn_model_from_config\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_rfcn_model_from_config): 0.23s\n",
            "I1113 23:11:08.663330 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_rfcn_model_from_config): 0.23s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_rfcn_model_from_config\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_ssd_fpn_model_from_config\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_ssd_fpn_model_from_config): 0.06s\n",
            "I1113 23:11:08.724067 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_ssd_fpn_model_from_config): 0.06s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_ssd_fpn_model_from_config\n",
            "[ RUN      ] ModelBuilderTF2Test.test_create_ssd_models_from_config\n",
            "I1113 23:11:09.325854 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b0\n",
            "I1113 23:11:09.326058 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 64\n",
            "I1113 23:11:09.326152 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 3\n",
            "I1113 23:11:09.330573 137329057406976 efficientnet_model.py:144] round_filter input=32 output=32\n",
            "I1113 23:11:09.387110 137329057406976 efficientnet_model.py:144] round_filter input=32 output=32\n",
            "I1113 23:11:09.387306 137329057406976 efficientnet_model.py:144] round_filter input=16 output=16\n",
            "I1113 23:11:09.580436 137329057406976 efficientnet_model.py:144] round_filter input=16 output=16\n",
            "I1113 23:11:09.580640 137329057406976 efficientnet_model.py:144] round_filter input=24 output=24\n",
            "I1113 23:11:10.313529 137329057406976 efficientnet_model.py:144] round_filter input=24 output=24\n",
            "I1113 23:11:10.317445 137329057406976 efficientnet_model.py:144] round_filter input=40 output=40\n",
            "I1113 23:11:10.965502 137329057406976 efficientnet_model.py:144] round_filter input=40 output=40\n",
            "I1113 23:11:10.965731 137329057406976 efficientnet_model.py:144] round_filter input=80 output=80\n",
            "I1113 23:11:11.776137 137329057406976 efficientnet_model.py:144] round_filter input=80 output=80\n",
            "I1113 23:11:11.776370 137329057406976 efficientnet_model.py:144] round_filter input=112 output=112\n",
            "I1113 23:11:12.652084 137329057406976 efficientnet_model.py:144] round_filter input=112 output=112\n",
            "I1113 23:11:12.652317 137329057406976 efficientnet_model.py:144] round_filter input=192 output=192\n",
            "I1113 23:11:13.792592 137329057406976 efficientnet_model.py:144] round_filter input=192 output=192\n",
            "I1113 23:11:13.792811 137329057406976 efficientnet_model.py:144] round_filter input=320 output=320\n",
            "I1113 23:11:14.080472 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=1280\n",
            "I1113 23:11:14.218231 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=1.0, depth_coefficient=1.0, resolution=224, dropout_rate=0.2, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "I1113 23:11:14.495394 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b1\n",
            "I1113 23:11:14.495633 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 88\n",
            "I1113 23:11:14.495734 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 4\n",
            "I1113 23:11:14.524938 137329057406976 efficientnet_model.py:144] round_filter input=32 output=32\n",
            "I1113 23:11:14.669502 137329057406976 efficientnet_model.py:144] round_filter input=32 output=32\n",
            "I1113 23:11:14.671190 137329057406976 efficientnet_model.py:144] round_filter input=16 output=16\n",
            "I1113 23:11:15.255441 137329057406976 efficientnet_model.py:144] round_filter input=16 output=16\n",
            "I1113 23:11:15.255664 137329057406976 efficientnet_model.py:144] round_filter input=24 output=24\n",
            "I1113 23:11:16.128775 137329057406976 efficientnet_model.py:144] round_filter input=24 output=24\n",
            "I1113 23:11:16.134810 137329057406976 efficientnet_model.py:144] round_filter input=40 output=40\n",
            "I1113 23:11:16.873902 137329057406976 efficientnet_model.py:144] round_filter input=40 output=40\n",
            "I1113 23:11:16.874128 137329057406976 efficientnet_model.py:144] round_filter input=80 output=80\n",
            "I1113 23:11:18.520170 137329057406976 efficientnet_model.py:144] round_filter input=80 output=80\n",
            "I1113 23:11:18.520402 137329057406976 efficientnet_model.py:144] round_filter input=112 output=112\n",
            "I1113 23:11:19.121334 137329057406976 efficientnet_model.py:144] round_filter input=112 output=112\n",
            "I1113 23:11:19.121537 137329057406976 efficientnet_model.py:144] round_filter input=192 output=192\n",
            "I1113 23:11:20.176711 137329057406976 efficientnet_model.py:144] round_filter input=192 output=192\n",
            "I1113 23:11:20.176937 137329057406976 efficientnet_model.py:144] round_filter input=320 output=320\n",
            "I1113 23:11:20.415009 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=1280\n",
            "I1113 23:11:20.458376 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=1.0, depth_coefficient=1.1, resolution=240, dropout_rate=0.2, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "I1113 23:11:20.562103 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b2\n",
            "I1113 23:11:20.562307 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 112\n",
            "I1113 23:11:20.562393 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 5\n",
            "I1113 23:11:20.564746 137329057406976 efficientnet_model.py:144] round_filter input=32 output=32\n",
            "I1113 23:11:20.587227 137329057406976 efficientnet_model.py:144] round_filter input=32 output=32\n",
            "I1113 23:11:20.587380 137329057406976 efficientnet_model.py:144] round_filter input=16 output=16\n",
            "I1113 23:11:20.774921 137329057406976 efficientnet_model.py:144] round_filter input=16 output=16\n",
            "I1113 23:11:20.775119 137329057406976 efficientnet_model.py:144] round_filter input=24 output=24\n",
            "I1113 23:11:21.107867 137329057406976 efficientnet_model.py:144] round_filter input=24 output=24\n",
            "I1113 23:11:21.108077 137329057406976 efficientnet_model.py:144] round_filter input=40 output=48\n",
            "I1113 23:11:21.681306 137329057406976 efficientnet_model.py:144] round_filter input=40 output=48\n",
            "I1113 23:11:21.681515 137329057406976 efficientnet_model.py:144] round_filter input=80 output=88\n",
            "I1113 23:11:22.383783 137329057406976 efficientnet_model.py:144] round_filter input=80 output=88\n",
            "I1113 23:11:22.383956 137329057406976 efficientnet_model.py:144] round_filter input=112 output=120\n",
            "I1113 23:11:22.676549 137329057406976 efficientnet_model.py:144] round_filter input=112 output=120\n",
            "I1113 23:11:22.676729 137329057406976 efficientnet_model.py:144] round_filter input=192 output=208\n",
            "I1113 23:11:23.061635 137329057406976 efficientnet_model.py:144] round_filter input=192 output=208\n",
            "I1113 23:11:23.061830 137329057406976 efficientnet_model.py:144] round_filter input=320 output=352\n",
            "I1113 23:11:23.204998 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=1408\n",
            "I1113 23:11:23.236745 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=1.1, depth_coefficient=1.2, resolution=260, dropout_rate=0.3, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "I1113 23:11:23.295989 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b3\n",
            "I1113 23:11:23.296154 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 160\n",
            "I1113 23:11:23.296238 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 6\n",
            "I1113 23:11:23.297784 137329057406976 efficientnet_model.py:144] round_filter input=32 output=40\n",
            "I1113 23:11:23.316257 137329057406976 efficientnet_model.py:144] round_filter input=32 output=40\n",
            "I1113 23:11:23.316375 137329057406976 efficientnet_model.py:144] round_filter input=16 output=24\n",
            "I1113 23:11:23.430774 137329057406976 efficientnet_model.py:144] round_filter input=16 output=24\n",
            "I1113 23:11:23.430922 137329057406976 efficientnet_model.py:144] round_filter input=24 output=32\n",
            "I1113 23:11:23.653529 137329057406976 efficientnet_model.py:144] round_filter input=24 output=32\n",
            "I1113 23:11:23.653722 137329057406976 efficientnet_model.py:144] round_filter input=40 output=48\n",
            "I1113 23:11:23.875052 137329057406976 efficientnet_model.py:144] round_filter input=40 output=48\n",
            "I1113 23:11:23.875227 137329057406976 efficientnet_model.py:144] round_filter input=80 output=96\n",
            "I1113 23:11:24.236381 137329057406976 efficientnet_model.py:144] round_filter input=80 output=96\n",
            "I1113 23:11:24.236548 137329057406976 efficientnet_model.py:144] round_filter input=112 output=136\n",
            "I1113 23:11:24.598376 137329057406976 efficientnet_model.py:144] round_filter input=112 output=136\n",
            "I1113 23:11:24.598545 137329057406976 efficientnet_model.py:144] round_filter input=192 output=232\n",
            "I1113 23:11:25.035322 137329057406976 efficientnet_model.py:144] round_filter input=192 output=232\n",
            "I1113 23:11:25.035496 137329057406976 efficientnet_model.py:144] round_filter input=320 output=384\n",
            "I1113 23:11:25.179217 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=1536\n",
            "I1113 23:11:25.205844 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=1.2, depth_coefficient=1.4, resolution=300, dropout_rate=0.3, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "I1113 23:11:25.269184 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b4\n",
            "I1113 23:11:25.269377 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 224\n",
            "I1113 23:11:25.269457 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 7\n",
            "I1113 23:11:25.271705 137329057406976 efficientnet_model.py:144] round_filter input=32 output=48\n",
            "I1113 23:11:25.287271 137329057406976 efficientnet_model.py:144] round_filter input=32 output=48\n",
            "I1113 23:11:25.287400 137329057406976 efficientnet_model.py:144] round_filter input=16 output=24\n",
            "I1113 23:11:25.401615 137329057406976 efficientnet_model.py:144] round_filter input=16 output=24\n",
            "I1113 23:11:25.401805 137329057406976 efficientnet_model.py:144] round_filter input=24 output=32\n",
            "I1113 23:11:25.988893 137329057406976 efficientnet_model.py:144] round_filter input=24 output=32\n",
            "I1113 23:11:25.989069 137329057406976 efficientnet_model.py:144] round_filter input=40 output=56\n",
            "I1113 23:11:26.281626 137329057406976 efficientnet_model.py:144] round_filter input=40 output=56\n",
            "I1113 23:11:26.281810 137329057406976 efficientnet_model.py:144] round_filter input=80 output=112\n",
            "I1113 23:11:26.715671 137329057406976 efficientnet_model.py:144] round_filter input=80 output=112\n",
            "I1113 23:11:26.715842 137329057406976 efficientnet_model.py:144] round_filter input=112 output=160\n",
            "I1113 23:11:27.161337 137329057406976 efficientnet_model.py:144] round_filter input=112 output=160\n",
            "I1113 23:11:27.161508 137329057406976 efficientnet_model.py:144] round_filter input=192 output=272\n",
            "I1113 23:11:27.735970 137329057406976 efficientnet_model.py:144] round_filter input=192 output=272\n",
            "I1113 23:11:27.736141 137329057406976 efficientnet_model.py:144] round_filter input=320 output=448\n",
            "I1113 23:11:27.884662 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=1792\n",
            "I1113 23:11:27.911715 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=1.4, depth_coefficient=1.8, resolution=380, dropout_rate=0.4, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "I1113 23:11:27.985730 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b5\n",
            "I1113 23:11:27.985895 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 288\n",
            "I1113 23:11:27.985972 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 7\n",
            "I1113 23:11:27.987442 137329057406976 efficientnet_model.py:144] round_filter input=32 output=48\n",
            "I1113 23:11:28.002804 137329057406976 efficientnet_model.py:144] round_filter input=32 output=48\n",
            "I1113 23:11:28.002922 137329057406976 efficientnet_model.py:144] round_filter input=16 output=24\n",
            "I1113 23:11:28.178555 137329057406976 efficientnet_model.py:144] round_filter input=16 output=24\n",
            "I1113 23:11:28.178739 137329057406976 efficientnet_model.py:144] round_filter input=24 output=40\n",
            "I1113 23:11:28.599412 137329057406976 efficientnet_model.py:144] round_filter input=24 output=40\n",
            "I1113 23:11:28.599619 137329057406976 efficientnet_model.py:144] round_filter input=40 output=64\n",
            "I1113 23:11:29.085104 137329057406976 efficientnet_model.py:144] round_filter input=40 output=64\n",
            "I1113 23:11:29.085303 137329057406976 efficientnet_model.py:144] round_filter input=80 output=128\n",
            "I1113 23:11:29.771612 137329057406976 efficientnet_model.py:144] round_filter input=80 output=128\n",
            "I1113 23:11:29.771822 137329057406976 efficientnet_model.py:144] round_filter input=112 output=176\n",
            "I1113 23:11:30.454316 137329057406976 efficientnet_model.py:144] round_filter input=112 output=176\n",
            "I1113 23:11:30.454507 137329057406976 efficientnet_model.py:144] round_filter input=192 output=304\n",
            "I1113 23:11:31.322010 137329057406976 efficientnet_model.py:144] round_filter input=192 output=304\n",
            "I1113 23:11:31.322209 137329057406976 efficientnet_model.py:144] round_filter input=320 output=512\n",
            "I1113 23:11:31.618200 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=2048\n",
            "I1113 23:11:31.653523 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=1.6, depth_coefficient=2.2, resolution=456, dropout_rate=0.4, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "I1113 23:11:32.131402 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b6\n",
            "I1113 23:11:32.131618 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 384\n",
            "I1113 23:11:32.131717 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 8\n",
            "I1113 23:11:32.134050 137329057406976 efficientnet_model.py:144] round_filter input=32 output=56\n",
            "I1113 23:11:32.156071 137329057406976 efficientnet_model.py:144] round_filter input=32 output=56\n",
            "I1113 23:11:32.156217 137329057406976 efficientnet_model.py:144] round_filter input=16 output=32\n",
            "I1113 23:11:32.418747 137329057406976 efficientnet_model.py:144] round_filter input=16 output=32\n",
            "I1113 23:11:32.418955 137329057406976 efficientnet_model.py:144] round_filter input=24 output=40\n",
            "I1113 23:11:33.105487 137329057406976 efficientnet_model.py:144] round_filter input=24 output=40\n",
            "I1113 23:11:33.105720 137329057406976 efficientnet_model.py:144] round_filter input=40 output=72\n",
            "I1113 23:11:33.766938 137329057406976 efficientnet_model.py:144] round_filter input=40 output=72\n",
            "I1113 23:11:33.767147 137329057406976 efficientnet_model.py:144] round_filter input=80 output=144\n",
            "I1113 23:11:34.660667 137329057406976 efficientnet_model.py:144] round_filter input=80 output=144\n",
            "I1113 23:11:34.660901 137329057406976 efficientnet_model.py:144] round_filter input=112 output=200\n",
            "I1113 23:11:35.552789 137329057406976 efficientnet_model.py:144] round_filter input=112 output=200\n",
            "I1113 23:11:35.553000 137329057406976 efficientnet_model.py:144] round_filter input=192 output=344\n",
            "I1113 23:11:36.761912 137329057406976 efficientnet_model.py:144] round_filter input=192 output=344\n",
            "I1113 23:11:36.762120 137329057406976 efficientnet_model.py:144] round_filter input=320 output=576\n",
            "I1113 23:11:37.080013 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=2304\n",
            "I1113 23:11:37.120899 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=1.8, depth_coefficient=2.6, resolution=528, dropout_rate=0.5, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "I1113 23:11:37.288605 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:161] EfficientDet EfficientNet backbone version: efficientnet-b7\n",
            "I1113 23:11:37.288824 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:163] EfficientDet BiFPN num filters: 384\n",
            "I1113 23:11:37.288920 137329057406976 ssd_efficientnet_bifpn_feature_extractor.py:164] EfficientDet BiFPN num iterations: 8\n",
            "I1113 23:11:37.291313 137329057406976 efficientnet_model.py:144] round_filter input=32 output=64\n",
            "I1113 23:11:37.313640 137329057406976 efficientnet_model.py:144] round_filter input=32 output=64\n",
            "I1113 23:11:37.313805 137329057406976 efficientnet_model.py:144] round_filter input=16 output=32\n",
            "I1113 23:11:37.657141 137329057406976 efficientnet_model.py:144] round_filter input=16 output=32\n",
            "I1113 23:11:37.657357 137329057406976 efficientnet_model.py:144] round_filter input=24 output=48\n",
            "I1113 23:11:38.429399 137329057406976 efficientnet_model.py:144] round_filter input=24 output=48\n",
            "I1113 23:11:38.429628 137329057406976 efficientnet_model.py:144] round_filter input=40 output=80\n",
            "I1113 23:11:39.176856 137329057406976 efficientnet_model.py:144] round_filter input=40 output=80\n",
            "I1113 23:11:39.177074 137329057406976 efficientnet_model.py:144] round_filter input=80 output=160\n",
            "I1113 23:11:40.285508 137329057406976 efficientnet_model.py:144] round_filter input=80 output=160\n",
            "I1113 23:11:40.285747 137329057406976 efficientnet_model.py:144] round_filter input=112 output=224\n",
            "I1113 23:11:41.397437 137329057406976 efficientnet_model.py:144] round_filter input=112 output=224\n",
            "I1113 23:11:41.397626 137329057406976 efficientnet_model.py:144] round_filter input=192 output=384\n",
            "I1113 23:11:42.374226 137329057406976 efficientnet_model.py:144] round_filter input=192 output=384\n",
            "I1113 23:11:42.374396 137329057406976 efficientnet_model.py:144] round_filter input=320 output=640\n",
            "I1113 23:11:42.655689 137329057406976 efficientnet_model.py:144] round_filter input=1280 output=2560\n",
            "I1113 23:11:42.694436 137329057406976 efficientnet_model.py:454] Building model efficientnet with params ModelConfig(width_coefficient=2.0, depth_coefficient=3.1, resolution=600, dropout_rate=0.5, blocks=(BlockConfig(input_filters=32, output_filters=16, kernel_size=3, num_repeat=1, expand_ratio=1, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=16, output_filters=24, kernel_size=3, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=24, output_filters=40, kernel_size=5, num_repeat=2, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=40, output_filters=80, kernel_size=3, num_repeat=3, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=80, output_filters=112, kernel_size=5, num_repeat=3, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=112, output_filters=192, kernel_size=5, num_repeat=4, expand_ratio=6, strides=(2, 2), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise'), BlockConfig(input_filters=192, output_filters=320, kernel_size=3, num_repeat=1, expand_ratio=6, strides=(1, 1), se_ratio=0.25, id_skip=True, fused_conv=False, conv_type='depthwise')), stem_base_filters=32, top_base_filters=1280, activation='simple_swish', batch_norm='default', bn_momentum=0.99, bn_epsilon=0.001, weight_decay=5e-06, drop_connect_rate=0.2, depth_divisor=8, min_depth=None, use_se=True, input_channels=3, num_classes=1000, model_name='efficientnet', rescale_input=False, data_format='channels_last', dtype='float32')\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_create_ssd_models_from_config): 34.08s\n",
            "I1113 23:11:42.806054 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_create_ssd_models_from_config): 34.08s\n",
            "[       OK ] ModelBuilderTF2Test.test_create_ssd_models_from_config\n",
            "[ RUN      ] ModelBuilderTF2Test.test_invalid_faster_rcnn_batchnorm_update\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_invalid_faster_rcnn_batchnorm_update): 0.01s\n",
            "I1113 23:11:42.818284 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_invalid_faster_rcnn_batchnorm_update): 0.01s\n",
            "[       OK ] ModelBuilderTF2Test.test_invalid_faster_rcnn_batchnorm_update\n",
            "[ RUN      ] ModelBuilderTF2Test.test_invalid_first_stage_nms_iou_threshold\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_invalid_first_stage_nms_iou_threshold): 0.0s\n",
            "I1113 23:11:42.819875 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_invalid_first_stage_nms_iou_threshold): 0.0s\n",
            "[       OK ] ModelBuilderTF2Test.test_invalid_first_stage_nms_iou_threshold\n",
            "[ RUN      ] ModelBuilderTF2Test.test_invalid_model_config_proto\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_invalid_model_config_proto): 0.0s\n",
            "I1113 23:11:42.820352 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_invalid_model_config_proto): 0.0s\n",
            "[       OK ] ModelBuilderTF2Test.test_invalid_model_config_proto\n",
            "[ RUN      ] ModelBuilderTF2Test.test_invalid_second_stage_batch_size\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_invalid_second_stage_batch_size): 0.0s\n",
            "I1113 23:11:42.821785 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_invalid_second_stage_batch_size): 0.0s\n",
            "[       OK ] ModelBuilderTF2Test.test_invalid_second_stage_batch_size\n",
            "[ RUN      ] ModelBuilderTF2Test.test_session\n",
            "[  SKIPPED ] ModelBuilderTF2Test.test_session\n",
            "[ RUN      ] ModelBuilderTF2Test.test_unknown_faster_rcnn_feature_extractor\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_unknown_faster_rcnn_feature_extractor): 0.0s\n",
            "I1113 23:11:42.823054 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_unknown_faster_rcnn_feature_extractor): 0.0s\n",
            "[       OK ] ModelBuilderTF2Test.test_unknown_faster_rcnn_feature_extractor\n",
            "[ RUN      ] ModelBuilderTF2Test.test_unknown_meta_architecture\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_unknown_meta_architecture): 0.0s\n",
            "I1113 23:11:42.823481 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_unknown_meta_architecture): 0.0s\n",
            "[       OK ] ModelBuilderTF2Test.test_unknown_meta_architecture\n",
            "[ RUN      ] ModelBuilderTF2Test.test_unknown_ssd_feature_extractor\n",
            "INFO:tensorflow:time(__main__.ModelBuilderTF2Test.test_unknown_ssd_feature_extractor): 0.0s\n",
            "I1113 23:11:42.824418 137329057406976 test_util.py:2373] time(__main__.ModelBuilderTF2Test.test_unknown_ssd_feature_extractor): 0.0s\n",
            "[       OK ] ModelBuilderTF2Test.test_unknown_ssd_feature_extractor\n",
            "----------------------------------------------------------------------\n",
            "Ran 24 tests in 46.356s\n",
            "\n",
            "OK (skipped=1)\n"
          ]
        }
      ],
      "source": [
        "# Run Model Bulider Test file, just to verify everything's working properly\n",
        "!python /content/models/research/object_detection/builders/model_builder_tf2_test.py\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 3.&nbsp;Upload Image Dataset and Prepare Training Data"
      ],
      "metadata": {
        "id": "eydREUsMGUUR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "q2sx_1qbmeUI",
        "outputId": "bba290a1-ab98-4b7f-915e-e8c542e20602"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mSZVCxE4nSVI"
      },
      "source": [
        "In this section, we'll upload our data and prepare it for training with TensorFlow. We'll upload our images, split them into train, validation, and test folders, and then run scripts for creating TFRecords from our data.\n",
        "\n",
        "First, on your local PC, zip all your training images and XML files into a single folder called \"images.zip\". The files should be directly inside the zip folder, or in a nested folder as shown below:\n",
        "```\n",
        "images.zip\n",
        "-- images\n",
        "  -- img1.jpg\n",
        "  -- img1.xml\n",
        "  -- img2.jpg\n",
        "  -- img2.xml\n",
        "  ...\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 3.1 Upload images\n",
        "There are three options for moving the image files to this Colab instance."
      ],
      "metadata": {
        "id": "LE1MtX4HGQA4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Option 1. Upload through Google Colab**\n",
        "\n",
        "Upload the \"images.zip\" file to the Google Colab instance by clicking the \"Files\" icon on the left hand side of the browser, and then the \"Upload to session storage\" icon. Select the zip folder to upload it.\n",
        "\n",
        "![](https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/doc/colab_upload_button.png)"
      ],
      "metadata": {
        "id": "sFSJoDEnJotN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Option 2. Copy from Google Drive**\n",
        "\n",
        "You can also upload your images to your personal Google Drive, mount the drive on this Colab session, and copy them over to the Colab filesystem. This option works well if you want to upload the images beforehand so you don't have to wait for them to upload each time you restart this Colab. If you have more than 50MB worth of images, I recommend using this option.\n",
        "\n",
        "First, upload the \"images.zip\" file to your Google Drive, and make note of the folder you uploaded them to. Replace `MyDrive/path/to/images.zip` with the path to your zip file. (For example, I uploaded the zip file to folder called \"change-counter1\", so I would use `MyDrive/change-counter1/images.zip` for the path). Then, run the following block of code to mount your Google Drive to this Colab session and copy the folder to this filesystem."
      ],
      "metadata": {
        "id": "hGsPlloAGIXB"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "id": "tLgAPsQsfTLs",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e2908c23-cf1e-4478-b2cd-939fd17a2d27"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount(\"/content/gdrive\", force_remount=True).\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/gdrive')\n",
        "\n",
        "!cp /content/gdrive/Shareddrives/HatchHead\\ Team\\ Drive/Clients/T-DA/01-Louise-the-fly-AR/ml/training-data.zip /content"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Option 3. Use coin detection dataset**\n",
        "\n",
        "If you don't have a dataset and just want to try training a model, you can download my coin image dataset to use as an example. I've uploaded a dataset containing 750 labeled images of pennies, nickels, dimes, and quarters. Run the following code block to download the dataset."
      ],
      "metadata": {
        "id": "9xAJMKwpFilm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget -O /content/training-data.zip https://www.dropbox.com/s/gk57ec3v8dfuwcp/CoinPics_11NOV22.zip?dl=0  # United States coin images"
      ],
      "metadata": {
        "id": "suu_xPVZIEcH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5f28fd83-3e90-41c2-f785-1efbafb3aa3c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-11-13 13:29:24--  https://www.dropbox.com/s/gk57ec3v8dfuwcp/CoinPics_11NOV22.zip?dl=0\n",
            "Resolving www.dropbox.com (www.dropbox.com)... 162.125.1.18, 2620:100:6016:18::a27d:112\n",
            "Connecting to www.dropbox.com (www.dropbox.com)|162.125.1.18|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: /s/raw/gk57ec3v8dfuwcp/CoinPics_11NOV22.zip [following]\n",
            "--2023-11-13 13:29:24--  https://www.dropbox.com/s/raw/gk57ec3v8dfuwcp/CoinPics_11NOV22.zip\n",
            "Reusing existing connection to www.dropbox.com:443.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com/cd/0/inline/CHd8r6KH4FgqBS5xmFAVKj2DLldoBTniViMIuPPy0JL5zhv_yrNw2_MAudyuk91wmFwm9SpEC8bIXa1gh_ObeE8lw2XJzIyKtrp5mDGi_1KLdQWZ0YJ1Jwp2jzbVeS8MXm_LiUzJaNl2UW0n63kzGzJE/file# [following]\n",
            "--2023-11-13 13:29:24--  https://uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com/cd/0/inline/CHd8r6KH4FgqBS5xmFAVKj2DLldoBTniViMIuPPy0JL5zhv_yrNw2_MAudyuk91wmFwm9SpEC8bIXa1gh_ObeE8lw2XJzIyKtrp5mDGi_1KLdQWZ0YJ1Jwp2jzbVeS8MXm_LiUzJaNl2UW0n63kzGzJE/file\n",
            "Resolving uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com (uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com)... 162.125.6.15, 2620:100:6016:15::a27d:10f\n",
            "Connecting to uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com (uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com)|162.125.6.15|:443... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: /cd/0/inline2/CHdzsc4P6IyUgVDNsGfYkBIfJtbMCE6It9qP7jv_pIB6Dbub9yzBcZFUzFXTSRTokUuoCT_WraPRKp-OswRA5JjQYiltJT0dHRb15NUNb8dAI8vnhH3qT-MdMm3bseO-j2c7XRWv6YdAdRO6q-Zdd546-76evOLMz-I1V2_aAAV3Mqfj3X-9sxpzCQZ9_xE8MZszvjuBcUeZzKKuh9DV_bwaYCpo0H62Xz8VJGFGcP24h9hxqKrkL6aiqj5DVlmiiY41rsCljARgq89wNNs6lhSrMjqQWtDMQIkF-muMgmgba7QKY4_cAKVZlw1FhWzmVgese7p201R01sJD8tNRB2Ead_QmFZZQYsoE2tiMUpuj-KOONkCqNiUMfNdTW3uWScw/file [following]\n",
            "--2023-11-13 13:29:25--  https://uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com/cd/0/inline2/CHdzsc4P6IyUgVDNsGfYkBIfJtbMCE6It9qP7jv_pIB6Dbub9yzBcZFUzFXTSRTokUuoCT_WraPRKp-OswRA5JjQYiltJT0dHRb15NUNb8dAI8vnhH3qT-MdMm3bseO-j2c7XRWv6YdAdRO6q-Zdd546-76evOLMz-I1V2_aAAV3Mqfj3X-9sxpzCQZ9_xE8MZszvjuBcUeZzKKuh9DV_bwaYCpo0H62Xz8VJGFGcP24h9hxqKrkL6aiqj5DVlmiiY41rsCljARgq89wNNs6lhSrMjqQWtDMQIkF-muMgmgba7QKY4_cAKVZlw1FhWzmVgese7p201R01sJD8tNRB2Ead_QmFZZQYsoE2tiMUpuj-KOONkCqNiUMfNdTW3uWScw/file\n",
            "Reusing existing connection to uc0e83b24936f0cd8e1df420cc77.dl.dropboxusercontent.com:443.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 233577865 (223M) [application/zip]\n",
            "Saving to: ‘/content/training-data.zip’\n",
            "\n",
            "/content/training-d 100%[===================>] 222.76M  69.9MB/s    in 3.6s    \n",
            "\n",
            "2023-11-13 13:29:29 (61.4 MB/s) - ‘/content/training-data.zip’ saved [233577865/233577865]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CHjOhoSGYwT7"
      },
      "source": [
        "## 3.2 Split images into train, validation, and test folders\n",
        "At this point, whether you used Option 1, 2, or 3, you should be able to click the folder icon on the left and see your \"images.zip\" file in the list of files. Now that the dataset is uploaded, let's unzip it and create some folders to hold the images. These directories are created in the /content folder in this instance's filesystem. You can browse the filesystem by clicking the \"Files\" icon on the left."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "id": "mGvoHH-unSVO"
      },
      "outputs": [],
      "source": [
        "!mkdir /content/images\n",
        "!unzip -q /content/training-data.zip -d /content/images/all\n",
        "!mkdir /content/images/train; mkdir /content/images/validation; mkdir /content/images/test"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n-6RIcrwbQMh"
      },
      "source": [
        "Next, we'll split the images into train, validation, and test sets. Here's what each set is used for:\n",
        "\n",
        "\n",
        "\n",
        "*   **Train**: These are the actual images used to train the model. In each step of training, a batch of images from the \"train\" set is passed into the neural network. The network predicts classes and locations of objects in the images. The training algorithm calculates the loss (i.e. how \"wrong\" the predictions were) and adjusts the network weights through backpropagation.\n",
        "\n",
        "\n",
        "*   **Validation**: Images from the \"validation\" set can be used by the training algorithm to check the progress of training and adjust hyperparameters (like learning rate). Unlike \"train\" images, these images are only used periodically during training (i.e. once every certain number of training steps).\n",
        "\n",
        "\n",
        "* **Test**: These images are never seen by the neural network during training. They are intended to be used by a human to perform final testing of the model to check how accurate the model is.\n",
        "\n",
        "I wrote a Python script to randomly move 80% of the images to the \"train\" folder, 10% to the \"validation\" folder, and 10% to the \"test\" folder. Click play on the following block to download the script and execute it."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!wget https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/util_scripts/train_val_test_split.py\n",
        "!python train_val_test_split.py"
      ],
      "metadata": {
        "id": "PfuZpmdBLjh-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1214d466-bb8d-4b9c-f935-1182d5d0ae00"
      },
      "execution_count": 34,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-11-14 00:12:26--  https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/util_scripts/train_val_test_split.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 2803 (2.7K) [text/plain]\n",
            "Saving to: ‘train_val_test_split.py.1’\n",
            "\n",
            "\r          train_val   0%[                    ]       0  --.-KB/s               \rtrain_val_test_spli 100%[===================>]   2.74K  --.-KB/s    in 0s      \n",
            "\n",
            "2023-11-14 00:12:27 (41.7 MB/s) - ‘train_val_test_split.py.1’ saved [2803/2803]\n",
            "\n",
            "Total images: 2396\n",
            "Images moving to train: 1916\n",
            "Images moving to validation: 239\n",
            "Images moving to test: 241\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p--K1PJXEgNo"
      },
      "source": [
        "## 3.3 Create Labelmap and TFRecords\n",
        "Finally, we need to create a labelmap for the detector and convert the images into a data file format called TFRecords, which are used by TensorFlow for training. We'll use Python scripts to automatically convert the data into TFRecord format. Before running them, we need to define a labelmap for our classes.\n",
        "\n",
        "The code section below will create a \"labelmap.txt\" file that contains a list of classes. Replace the `class1`, `class2`, `class3` text with your own classes (for example, `penny`, `nickel`, `dime`, `quarter`), adding a new line for each class. Then, click play to execute the code."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "id": "_DE_r4MKY7ln"
      },
      "outputs": [],
      "source": [
        "### This creates a a \"labelmap.txt\" file with a list of classes the object detection model will detect.\n",
        "%%bash\n",
        "cat <<EOF >> /content/labelmap.txt\n",
        "fly\n",
        "EOF"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5pa2VYhTIT1l"
      },
      "source": [
        "Download and run the data conversion scripts from the [GitHub repository](https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi) by clicking play on the following three sections of code. They will create TFRecord files for the train and validation datasets, as well as a `labelmap.pbtxt` file which contains the labelmap in a different format."
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Download data conversion scripts\n",
        "! wget https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/util_scripts/create_csv.py\n",
        "! wget https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/util_scripts/create_tfrecord.py"
      ],
      "metadata": {
        "id": "laZZE0TlEeUF",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "810bd159-f389-4383-eba0-da08d0fc9060"
      },
      "execution_count": 35,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2023-11-14 00:13:06--  https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/util_scripts/create_csv.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1348 (1.3K) [text/plain]\n",
            "Saving to: ‘create_csv.py.2’\n",
            "\n",
            "\rcreate_csv.py.2       0%[                    ]       0  --.-KB/s               \rcreate_csv.py.2     100%[===================>]   1.32K  --.-KB/s    in 0s      \n",
            "\n",
            "2023-11-14 00:13:06 (97.9 MB/s) - ‘create_csv.py.2’ saved [1348/1348]\n",
            "\n",
            "--2023-11-14 00:13:06--  https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/util_scripts/create_tfrecord.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 4414 (4.3K) [text/plain]\n",
            "Saving to: ‘create_tfrecord.py.2’\n",
            "\n",
            "create_tfrecord.py. 100%[===================>]   4.31K  --.-KB/s    in 0s      \n",
            "\n",
            "2023-11-14 00:13:07 (45.9 MB/s) - ‘create_tfrecord.py.2’ saved [4414/4414]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 41,
      "metadata": {
        "id": "5tdDbTmHYwu-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "676c5bed-7e3e-4068-c24f-23f7b60624e1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Traceback (most recent call last):\n",
            "  File \"/content/models/mymodel/create_csv.py\", line 36, in <module>\n",
            "    main()\n",
            "  File \"/content/models/mymodel/create_csv.py\", line 33, in main\n",
            "    xml_df.to_csv(('images/' + folder + '_labels.csv'), index=None)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py\", line 211, in wrapper\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/core/generic.py\", line 3720, in to_csv\n",
            "    return DataFrameRenderer(formatter).to_csv(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py\", line 211, in wrapper\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/formats/format.py\", line 1189, in to_csv\n",
            "    csv_formatter.save()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/formats/csvs.py\", line 241, in save\n",
            "    with get_handle(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/common.py\", line 734, in get_handle\n",
            "    check_parent_directory(str(handle))\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/common.py\", line 597, in check_parent_directory\n",
            "    raise OSError(rf\"Cannot save file into a non-existent directory: '{parent}'\")\n",
            "OSError: Cannot save file into a non-existent directory: 'images'\n",
            "Traceback (most recent call last):\n",
            "  File \"/content/models/mymodel/create_tfrecord.py\", line 120, in <module>\n",
            "    tf.app.run()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/tensorflow/python/platform/app.py\", line 36, in run\n",
            "    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/absl/app.py\", line 308, in run\n",
            "    _run_main(main, args)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/absl/app.py\", line 254, in _run_main\n",
            "    sys.exit(main(argv))\n",
            "  File \"/content/models/mymodel/create_tfrecord.py\", line 93, in main\n",
            "    examples = pd.read_csv(FLAGS.csv_input)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py\", line 211, in wrapper\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py\", line 331, in wrapper\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 950, in read_csv\n",
            "    return _read(filepath_or_buffer, kwds)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 605, in _read\n",
            "    parser = TextFileReader(filepath_or_buffer, **kwds)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 1442, in __init__\n",
            "    self._engine = self._make_engine(f, self.engine)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 1735, in _make_engine\n",
            "    self.handles = get_handle(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/common.py\", line 856, in get_handle\n",
            "    handle = open(\n",
            "FileNotFoundError: [Errno 2] No such file or directory: '/content/images/train_labels.csv'\n",
            "Traceback (most recent call last):\n",
            "  File \"/content/models/mymodel/create_tfrecord.py\", line 120, in <module>\n",
            "    tf.app.run()\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/tensorflow/python/platform/app.py\", line 36, in run\n",
            "    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/absl/app.py\", line 308, in run\n",
            "    _run_main(main, args)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/absl/app.py\", line 254, in _run_main\n",
            "    sys.exit(main(argv))\n",
            "  File \"/content/models/mymodel/create_tfrecord.py\", line 93, in main\n",
            "    examples = pd.read_csv(FLAGS.csv_input)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py\", line 211, in wrapper\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/util/_decorators.py\", line 331, in wrapper\n",
            "    return func(*args, **kwargs)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 950, in read_csv\n",
            "    return _read(filepath_or_buffer, kwds)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 605, in _read\n",
            "    parser = TextFileReader(filepath_or_buffer, **kwds)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 1442, in __init__\n",
            "    self._engine = self._make_engine(f, self.engine)\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/parsers/readers.py\", line 1735, in _make_engine\n",
            "    self.handles = get_handle(\n",
            "  File \"/usr/local/lib/python3.10/dist-packages/pandas/io/common.py\", line 856, in get_handle\n",
            "    handle = open(\n",
            "FileNotFoundError: [Errno 2] No such file or directory: '/content/images/validation_labels.csv'\n"
          ]
        }
      ],
      "source": [
        "# Create CSV data files and TFRecord files\n",
        "!python3 create_csv.py\n",
        "!python3 create_tfrecord.py --csv_input=/content/images/train_labels.csv --labelmap=labelmap.txt --image_dir=images/train --output_path=train.tfrecord\n",
        "!python3 create_tfrecord.py --csv_input=/content/images/validation_labels.csv --labelmap=labelmap.txt --image_dir=images/validation --output_path=val.tfrecord"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RNyv_YyDXwMs"
      },
      "source": [
        "We'll store the locations of the TFRecord and labelmap files as variables so we can reference them later in this Colab session."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "YUd2wtfrqedy"
      },
      "outputs": [],
      "source": [
        "train_record_fname = '/content/train.tfrecord'\n",
        "val_record_fname = '/content/val.tfrecord'\n",
        "label_map_pbtxt_fname = '/content/labelmap.pbtxt'"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 4.&nbsp;Set Up Training Configuration"
      ],
      "metadata": {
        "id": "eGEUZYAMEZ6f"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I2MAcgJ53STW"
      },
      "source": [
        "In this section, we'll set up the model and training configuration. We'll specifiy which pretrained TensorFlow model we want to use from the [TensorFlow 2 Object Detection Model Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md). Each model also comes with a configuration file that points to file locations, sets training parameters (such as learning rate and total number of training steps), and more. We'll modify the configuration file for our custom training job.\n",
        "\n",
        "The first section of code lists out some models availabe in the TF2 Model Zoo and defines some filenames that will be used later to download the model and config file. This makes it easy to manage which model you're using and to add other models to the list later.\n",
        "\n",
        "Set the \"chosen_model\" variable to match the name of the model you'd like to train with. It's currently set to use the popular \"ssd-mobilenet-v2\" model. Click play on the next block once the chosen model has been set.\n",
        "\n",
        "Not sure which model to pick? [Check out my blog post comparing each model's speed and accuracy.](https://ejtech.io/learn/tflite-object-detection-model-comparison)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "gN0EUEa3e5Un"
      },
      "outputs": [],
      "source": [
        "# Change the chosen_model variable to deploy different models available in the TF2 object detection zoo\n",
        "chosen_model = 'ssd-mobilenet-v2-fpnlite-320'\n",
        "\n",
        "MODELS_CONFIG = {\n",
        "    'ssd-mobilenet-v2': {\n",
        "        'model_name': 'ssd_mobilenet_v2_320x320_coco17_tpu-8',\n",
        "        'base_pipeline_file': 'ssd_mobilenet_v2_320x320_coco17_tpu-8.config',\n",
        "        'pretrained_checkpoint': 'ssd_mobilenet_v2_320x320_coco17_tpu-8.tar.gz',\n",
        "    },\n",
        "    'efficientdet-d0': {\n",
        "        'model_name': 'efficientdet_d0_coco17_tpu-32',\n",
        "        'base_pipeline_file': 'ssd_efficientdet_d0_512x512_coco17_tpu-8.config',\n",
        "        'pretrained_checkpoint': 'efficientdet_d0_coco17_tpu-32.tar.gz',\n",
        "    },\n",
        "    'ssd-mobilenet-v2-fpnlite-320': {\n",
        "        'model_name': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8',\n",
        "        'base_pipeline_file': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config',\n",
        "        'pretrained_checkpoint': 'ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz',\n",
        "    },\n",
        "    # The centernet model isn't working as of 9/10/22\n",
        "    #'centernet-mobilenet-v2': {\n",
        "    #    'model_name': 'centernet_mobilenetv2fpn_512x512_coco17_od',\n",
        "    #    'base_pipeline_file': 'pipeline.config',\n",
        "    #    'pretrained_checkpoint': 'centernet_mobilenetv2fpn_512x512_coco17_od.tar.gz',\n",
        "    #}\n",
        "}\n",
        "\n",
        "model_name = MODELS_CONFIG[chosen_model]['model_name']\n",
        "pretrained_checkpoint = MODELS_CONFIG[chosen_model]['pretrained_checkpoint']\n",
        "base_pipeline_file = MODELS_CONFIG[chosen_model]['base_pipeline_file']"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Download the pretrained model file and configuration file by clicking Play on the following section."
      ],
      "metadata": {
        "id": "JMG3EEPqPggV"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "kG4TmJUVrYQ7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "21a880f7-16bc-4a7b-838b-e3ff57841585"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/content/models/mymodel\n",
            "--2023-11-13 23:13:15--  http://download.tensorflow.org/models/object_detection/tf2/20200711/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz\n",
            "Resolving download.tensorflow.org (download.tensorflow.org)... 108.177.11.207, 108.177.12.207, 108.177.13.207, ...\n",
            "Connecting to download.tensorflow.org (download.tensorflow.org)|108.177.11.207|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 20515344 (20M) [application/x-tar]\n",
            "Saving to: ‘ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz’\n",
            "\n",
            "ssd_mobilenet_v2_fp 100%[===================>]  19.56M   114MB/s    in 0.2s    \n",
            "\n",
            "2023-11-13 23:13:15 (114 MB/s) - ‘ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz’ saved [20515344/20515344]\n",
            "\n",
            "--2023-11-13 23:13:15--  https://raw.githubusercontent.com/tensorflow/models/master/research/object_detection/configs/tf2/ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 4684 (4.6K) [text/plain]\n",
            "Saving to: ‘ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config’\n",
            "\n",
            "ssd_mobilenet_v2_fp 100%[===================>]   4.57K  --.-KB/s    in 0s      \n",
            "\n",
            "2023-11-13 23:13:15 (36.3 MB/s) - ‘ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.config’ saved [4684/4684]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "# Create \"mymodel\" folder for holding pre-trained weights and configuration files\n",
        "%mkdir /content/models/mymodel/\n",
        "%cd /content/models/mymodel/\n",
        "\n",
        "# Download pre-trained model weights\n",
        "import tarfile\n",
        "download_tar = 'http://download.tensorflow.org/models/object_detection/tf2/20200711/' + pretrained_checkpoint\n",
        "!wget {download_tar}\n",
        "tar = tarfile.open(pretrained_checkpoint)\n",
        "tar.extractall()\n",
        "tar.close()\n",
        "\n",
        "# Download training configuration file for model\n",
        "download_config = 'https://raw.githubusercontent.com/tensorflow/models/master/research/object_detection/configs/tf2/' + base_pipeline_file\n",
        "!wget {download_config}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BFAlqNrPn5y3"
      },
      "source": [
        "Now that we've downloaded our model and config file, we need to modify the configuration file with some high-level training parameters. The following variables are used to control training steps:\n",
        "\n",
        "* **num_steps**: The total amount of steps to use for training the model. A good number to start with is 40,000 steps. You can use more steps if you notice the loss metrics are still decreasing by the time training finishes. The more steps, the longer training will take. Training can also be stopped early if loss flattens out before reaching the specified number of steps.\n",
        "* **batch_size**: The number of images to use per training step. A larger batch size allows a model to be trained in fewer steps, but the size is limited by the GPU memory available for training. With the GPUs used in Colab instances, 16 is a good number for SSD models and 4 is good for EfficientDet models.\n",
        "\n",
        "Other training information, like the location of the pretrained model file, the config file, and total number of classes are also assigned in this step. To learn more about training configuration with the TensorFlow Object Detection API, read this [article from Neptune](https://neptune.ai/blog/tensorflow-object-detection-api-best-practices-to-training-evaluation-deployment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "1lYDvJN-n69v"
      },
      "outputs": [],
      "source": [
        "# Set training parameters for the model\n",
        "num_steps = 40000\n",
        "\n",
        "if chosen_model == 'efficientdet-d0':\n",
        "  batch_size = 4\n",
        "else:\n",
        "  batch_size = 16"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "b_ki9jOqxn7V",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 513
        },
        "outputId": "3653f48f-9a76-42dc-ae32-e2d7798f0c1c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/content/labelmap.txt\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mParseError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/object_detection/utils/label_map_util.py\u001b[0m in \u001b[0;36mload_labelmap\u001b[0;34m(path, validator)\u001b[0m\n\u001b[1;32m    174\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 175\u001b[0;31m       \u001b[0mtext_format\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mMerge\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabel_map_string\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel_map\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    176\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mtext_format\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mParseError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/protobuf/text_format.py\u001b[0m in \u001b[0;36mMerge\u001b[0;34m(text, message, allow_unknown_extension, allow_field_number, descriptor_pool, allow_unknown_field)\u001b[0m\n\u001b[1;32m    718\u001b[0m   \"\"\"\n\u001b[0;32m--> 719\u001b[0;31m   return MergeLines(\n\u001b[0m\u001b[1;32m    720\u001b[0m       \u001b[0mtext\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msplit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34mb'\\n'\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtext\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbytes\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34mu'\\n'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/protobuf/text_format.py\u001b[0m in \u001b[0;36mMergeLines\u001b[0;34m(lines, message, allow_unknown_extension, allow_field_number, descriptor_pool, allow_unknown_field)\u001b[0m\n\u001b[1;32m    792\u001b[0m                    allow_unknown_field=allow_unknown_field)\n\u001b[0;32m--> 793\u001b[0;31m   \u001b[0;32mreturn\u001b[0m \u001b[0mparser\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mMergeLines\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlines\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    794\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/protobuf/text_format.py\u001b[0m in \u001b[0;36mMergeLines\u001b[0;34m(self, lines, message)\u001b[0m\n\u001b[1;32m    817\u001b[0m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_allow_multiple_scalars\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 818\u001b[0;31m     \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_ParseOrMerge\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlines\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    819\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/protobuf/text_format.py\u001b[0m in \u001b[0;36m_ParseOrMerge\u001b[0;34m(self, lines, message)\u001b[0m\n\u001b[1;32m    836\u001b[0m     \u001b[0;32mwhile\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mtokenizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mAtEnd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 837\u001b[0;31m       \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_MergeField\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtokenizer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    838\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/google/protobuf/text_format.py\u001b[0m in \u001b[0;36m_MergeField\u001b[0;34m(self, tokenizer, message)\u001b[0m\n\u001b[1;32m    931\u001b[0m       \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mfield\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mallow_unknown_field\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 932\u001b[0;31m         raise tokenizer.ParseErrorPreviousToken(\n\u001b[0m\u001b[1;32m    933\u001b[0m             \u001b[0;34m'Message type \"%s\" has no field named \"%s\".'\u001b[0m \u001b[0;34m%\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mParseError\u001b[0m: 1:1 : Message type \"object_detection.protos.StringIntLabelMap\" has no field named \"fly\".",
            "\nDuring handling of the above exception, another exception occurred:\n",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-18-3ac7eea8609f>\u001b[0m in \u001b[0;36m<cell line: 13>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     11\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcategory_index\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeys\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabel_map_pbtxt_fname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mnum_classes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_num_classes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabel_map_pbtxt_fname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Total classes:'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnum_classes\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-18-3ac7eea8609f>\u001b[0m in \u001b[0;36mget_num_classes\u001b[0;34m(pbtxt_fname)\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mget_num_classes\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpbtxt_fname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m     \u001b[0;32mfrom\u001b[0m \u001b[0mobject_detection\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mutils\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mlabel_map_util\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m     \u001b[0mlabel_map\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mlabel_map_util\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mload_labelmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpbtxt_fname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m     categories = label_map_util.convert_label_map_to_categories(\n\u001b[1;32m      9\u001b[0m         label_map, max_num_classes=90, use_display_name=True)\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/object_detection/utils/label_map_util.py\u001b[0m in \u001b[0;36mload_labelmap\u001b[0;34m(path, validator)\u001b[0m\n\u001b[1;32m    175\u001b[0m       \u001b[0mtext_format\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mMerge\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabel_map_string\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel_map\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    176\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0mtext_format\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mParseError\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 177\u001b[0;31m       \u001b[0mlabel_map\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mParseFromString\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlabel_map_string\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    178\u001b[0m   \u001b[0;32mif\u001b[0m \u001b[0mvalidator\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    179\u001b[0m     \u001b[0mvalidator\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_validate_label_map\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: a bytes-like object is required, not 'str'"
          ]
        }
      ],
      "source": [
        "# Set file locations and get number of classes for config file\n",
        "pipeline_fname = '/content/models/mymodel/' + base_pipeline_file\n",
        "fine_tune_checkpoint = '/content/models/mymodel/' + model_name + '/checkpoint/ckpt-0'\n",
        "\n",
        "def get_num_classes(pbtxt_fname):\n",
        "    from object_detection.utils import label_map_util\n",
        "    label_map = label_map_util.load_labelmap(pbtxt_fname)\n",
        "    categories = label_map_util.convert_label_map_to_categories(\n",
        "        label_map, max_num_classes=90, use_display_name=True)\n",
        "    category_index = label_map_util.create_category_index(categories)\n",
        "    return len(category_index.keys())\n",
        "print(label_map_pbtxt_fname)\n",
        "num_classes = get_num_classes(label_map_pbtxt_fname)\n",
        "print('Total classes:', num_classes)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cwPyaIAXxyKu"
      },
      "source": [
        "Next, we'll rewrite the config file to use the training parameters we just specified. The following section of code will automatically replace the necessary parameters in the downloaded .config file and save it as our custom \"pipeline_file.config\" file."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5eA5ht3_yukT"
      },
      "outputs": [],
      "source": [
        "# Create custom configuration file by writing the dataset, model checkpoint, and training parameters into the base pipeline file\n",
        "import re\n",
        "\n",
        "%cd /content/models/mymodel\n",
        "print('writing custom configuration file')\n",
        "\n",
        "with open(pipeline_fname) as f:\n",
        "    s = f.read()\n",
        "with open('pipeline_file.config', 'w') as f:\n",
        "\n",
        "    # Set fine_tune_checkpoint path\n",
        "    s = re.sub('fine_tune_checkpoint: \".*?\"',\n",
        "               'fine_tune_checkpoint: \"{}\"'.format(fine_tune_checkpoint), s)\n",
        "\n",
        "    # Set tfrecord files for train and test datasets\n",
        "    s = re.sub(\n",
        "        '(input_path: \".*?)(PATH_TO_BE_CONFIGURED/train)(.*?\")', 'input_path: \"{}\"'.format(train_record_fname), s)\n",
        "    s = re.sub(\n",
        "        '(input_path: \".*?)(PATH_TO_BE_CONFIGURED/val)(.*?\")', 'input_path: \"{}\"'.format(val_record_fname), s)\n",
        "\n",
        "    # Set label_map_path\n",
        "    s = re.sub(\n",
        "        'label_map_path: \".*?\"', 'label_map_path: \"{}\"'.format(label_map_pbtxt_fname), s)\n",
        "\n",
        "    # Set batch_size\n",
        "    s = re.sub('batch_size: [0-9]+',\n",
        "               'batch_size: {}'.format(batch_size), s)\n",
        "\n",
        "    # Set training steps, num_steps\n",
        "    s = re.sub('num_steps: [0-9]+',\n",
        "               'num_steps: {}'.format(num_steps), s)\n",
        "\n",
        "    # Set number of classes num_classes\n",
        "    s = re.sub('num_classes: [0-9]+',\n",
        "               'num_classes: {}'.format(num_classes), s)\n",
        "\n",
        "    # Change fine-tune checkpoint type from \"classification\" to \"detection\"\n",
        "    s = re.sub(\n",
        "        'fine_tune_checkpoint_type: \"classification\"', 'fine_tune_checkpoint_type: \"{}\"'.format('detection'), s)\n",
        "\n",
        "    # If using ssd-mobilenet-v2, reduce learning rate (because it's too high in the default config file)\n",
        "    if chosen_model == 'ssd-mobilenet-v2':\n",
        "      s = re.sub('learning_rate_base: .8',\n",
        "                 'learning_rate_base: .08', s)\n",
        "\n",
        "      s = re.sub('warmup_learning_rate: 0.13333',\n",
        "                 'warmup_learning_rate: .026666', s)\n",
        "\n",
        "    # If using efficientdet-d0, use fixed_shape_resizer instead of keep_aspect_ratio_resizer (because it isn't supported by TFLite)\n",
        "    if chosen_model == 'efficientdet-d0':\n",
        "      s = re.sub('keep_aspect_ratio_resizer', 'fixed_shape_resizer', s)\n",
        "      s = re.sub('pad_to_max_dimension: true', '', s)\n",
        "      s = re.sub('min_dimension', 'height', s)\n",
        "      s = re.sub('max_dimension', 'width', s)\n",
        "\n",
        "    f.write(s)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GDySP7TLzdCM"
      },
      "source": [
        "(Optional) If you're curious, you can display the configuration file's contents here in the browser by running the line of code below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "HEsOLOMHzBqF"
      },
      "outputs": [],
      "source": [
        "# (Optional) Display the custom configuration file's contents\n",
        "!cat /content/models/mymodel/pipeline_file.config"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UXpnXYC908Zl"
      },
      "source": [
        "Finally, let's set the locations of the configuration file and model output directory as variables so we can reference them when we call the training command."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GMlaN3rs3zLe"
      },
      "outputs": [],
      "source": [
        "# Set the path to the custom config file and the directory to store training checkpoints in\n",
        "pipeline_file = '/content/models/mymodel/pipeline_file.config'\n",
        "model_dir = '/content/training/'"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 5.&nbsp;Train Custom TFLite Detection Model"
      ],
      "metadata": {
        "id": "-19zML6oEO7l"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XxPj_QV43qD5"
      },
      "source": [
        "We're ready to train our object detection model! Before we start training, let's load up a TensorBoard session to monitor training progress. Run the following section of code, and a TensorBoard session will appear in the browser. It won't show anything yet, because we haven't started training. Once training starts, come back and click the refresh button to see the model's overall loss.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TI9iCCxoNlAL"
      },
      "outputs": [],
      "source": [
        "%load_ext tensorboard\n",
        "%tensorboard --logdir '/content/training/train'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5cuQpPJL2pUq"
      },
      "source": [
        "Model training is performed using the \"model_main_tf2.py\" script from the TF Object Detection API. Training will take anywhere from 2 to 6 hours, depending on the model, batch size, and number of training steps. We've already defined all the parameters and arguments used by `model_main_tf2.py` in previous sections of this Colab. Just click Play on the following block to begin training!\n",
        "\n",
        "\n",
        "\n",
        "> *Note: It takes a few minutes for the program to display any training messages, because it only displays logs once every 100 steps. If it seems like nothing is happening, just wait a couple minutes.*"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tQTfZChVzzpZ"
      },
      "outputs": [],
      "source": [
        "# Run training!\n",
        "!python /content/models/research/object_detection/model_main_tf2.py \\\n",
        "    --pipeline_config_path={pipeline_file} \\\n",
        "    --model_dir={model_dir} \\\n",
        "    --alsologtostderr \\\n",
        "    --num_train_steps={num_steps} \\\n",
        "    --sample_1_of_n_eval_examples=1"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you want to stop training early, just click Stop a couple times or right-click on the code block and select \"Interrupt Execution\". Otherwise, training will stop by itself once it reaches the specified number of training steps.\n"
      ],
      "metadata": {
        "id": "WHxbX4ZpzXIv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 6.&nbsp;Convert Model to TensorFlow Lite"
      ],
      "metadata": {
        "id": "kPg8oMnQDYKl"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "spQXdq8Y63pj"
      },
      "source": [
        "Alright! Our model is all trained up and ready to be used for detecting objects. First, we need to export the model graph (a file that contains information about the architecture and weights) to a TensorFlow Lite-compatible format. We'll do this using the `export_tflite_graph_tf2.py` script."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "RaUU8tBlHifd"
      },
      "outputs": [],
      "source": [
        "# Make a directory to store the trained TFLite model\n",
        "!mkdir /content/custom_model_lite\n",
        "output_directory = '/content/custom_model_lite'\n",
        "\n",
        "# Path to training directory (the conversion script automatically chooses the highest checkpoint file)\n",
        "last_model_path = '/content/training'\n",
        "\n",
        "!python /content/models/research/object_detection/export_tflite_graph_tf2.py \\\n",
        "    --trained_checkpoint_dir {last_model_path} \\\n",
        "    --output_directory {output_directory} \\\n",
        "    --pipeline_config_path {pipeline_file}\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z_NuapO2VROu"
      },
      "source": [
        "Next, we'll take the exported graph and use the `TFLiteConverter` module to convert it to `.tflite` FlatBuffer format."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TsE_uVjlsz3u"
      },
      "outputs": [],
      "source": [
        "# Convert exported graph file into TFLite model file\n",
        "import tensorflow as tf\n",
        "\n",
        "converter = tf.lite.TFLiteConverter.from_saved_model('/content/custom_model_lite/saved_model')\n",
        "tflite_model = converter.convert()\n",
        "\n",
        "with open('/content/custom_model_lite/detect.tflite', 'wb') as f:\n",
        "  f.write(tflite_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 7.&nbsp;Test TensorFlow Lite Model and Calculate mAP"
      ],
      "metadata": {
        "id": "RDQrtQhvC3oG"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vtSmUZcxIAvt"
      },
      "source": [
        "We've trained our custom model and converted it to TFLite format. But how well does it actually perform at detecting objects in images? This is where the images we set aside in the **test** folder come in. The model never saw any test images during training, so its performance on these images should be representative of how it will perform on new images from the field.\n",
        "\n",
        "### 7.1 Inference test images\n",
        "The following code defines a function to run inference on test images. It loads the images, loads the model and labelmap, runs the model on each image, and displays the result. It also optionally saves detection results as text files so we can use them to calculate model mAP score.\n",
        "\n",
        "This code is based off the [TFLite_detection_image.py](https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/TFLite_detection_image.py) script from my [TensorFlow Lite Object Detection repository on GitHub](https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi); feel free to use it as a starting point for your own application."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "e4WtI8i5K96w"
      },
      "outputs": [],
      "source": [
        "# Script to run custom TFLite model on test images to detect objects\n",
        "# Source: https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/TFLite_detection_image.py\n",
        "\n",
        "# Import packages\n",
        "import os\n",
        "import cv2\n",
        "import numpy as np\n",
        "import sys\n",
        "import glob\n",
        "import random\n",
        "import importlib.util\n",
        "from tensorflow.lite.python.interpreter import Interpreter\n",
        "\n",
        "import matplotlib\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "%matplotlib inline\n",
        "\n",
        "### Define function for inferencing with TFLite model and displaying results\n",
        "\n",
        "def tflite_detect_images(modelpath, imgpath, lblpath, min_conf=0.5, num_test_images=10, savepath='/content/results', txt_only=False):\n",
        "\n",
        "  # Grab filenames of all images in test folder\n",
        "  images = glob.glob(imgpath + '/*.jpg') + glob.glob(imgpath + '/*.JPG') + glob.glob(imgpath + '/*.png') + glob.glob(imgpath + '/*.bmp')\n",
        "\n",
        "  # Load the label map into memory\n",
        "  with open(lblpath, 'r') as f:\n",
        "      labels = [line.strip() for line in f.readlines()]\n",
        "\n",
        "  # Load the Tensorflow Lite model into memory\n",
        "  interpreter = Interpreter(model_path=modelpath)\n",
        "  interpreter.allocate_tensors()\n",
        "\n",
        "  # Get model details\n",
        "  input_details = interpreter.get_input_details()\n",
        "  output_details = interpreter.get_output_details()\n",
        "  height = input_details[0]['shape'][1]\n",
        "  width = input_details[0]['shape'][2]\n",
        "\n",
        "  float_input = (input_details[0]['dtype'] == np.float32)\n",
        "\n",
        "  input_mean = 127.5\n",
        "  input_std = 127.5\n",
        "\n",
        "  # Randomly select test images\n",
        "  images_to_test = random.sample(images, num_test_images)\n",
        "\n",
        "  # Loop over every image and perform detection\n",
        "  for image_path in images_to_test:\n",
        "\n",
        "      # Load image and resize to expected shape [1xHxWx3]\n",
        "      image = cv2.imread(image_path)\n",
        "      image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
        "      imH, imW, _ = image.shape\n",
        "      image_resized = cv2.resize(image_rgb, (width, height))\n",
        "      input_data = np.expand_dims(image_resized, axis=0)\n",
        "\n",
        "      # Normalize pixel values if using a floating model (i.e. if model is non-quantized)\n",
        "      if float_input:\n",
        "          input_data = (np.float32(input_data) - input_mean) / input_std\n",
        "\n",
        "      # Perform the actual detection by running the model with the image as input\n",
        "      interpreter.set_tensor(input_details[0]['index'],input_data)\n",
        "      interpreter.invoke()\n",
        "\n",
        "      # Retrieve detection results\n",
        "      boxes = interpreter.get_tensor(output_details[1]['index'])[0] # Bounding box coordinates of detected objects\n",
        "      classes = interpreter.get_tensor(output_details[3]['index'])[0] # Class index of detected objects\n",
        "      scores = interpreter.get_tensor(output_details[0]['index'])[0] # Confidence of detected objects\n",
        "\n",
        "      detections = []\n",
        "\n",
        "      # Loop over all detections and draw detection box if confidence is above minimum threshold\n",
        "      for i in range(len(scores)):\n",
        "          if ((scores[i] > min_conf) and (scores[i] <= 1.0)):\n",
        "\n",
        "              # Get bounding box coordinates and draw box\n",
        "              # Interpreter can return coordinates that are outside of image dimensions, need to force them to be within image using max() and min()\n",
        "              ymin = int(max(1,(boxes[i][0] * imH)))\n",
        "              xmin = int(max(1,(boxes[i][1] * imW)))\n",
        "              ymax = int(min(imH,(boxes[i][2] * imH)))\n",
        "              xmax = int(min(imW,(boxes[i][3] * imW)))\n",
        "\n",
        "              cv2.rectangle(image, (xmin,ymin), (xmax,ymax), (10, 255, 0), 2)\n",
        "\n",
        "              # Draw label\n",
        "              object_name = labels[int(classes[i])] # Look up object name from \"labels\" array using class index\n",
        "              label = '%s: %d%%' % (object_name, int(scores[i]*100)) # Example: 'person: 72%'\n",
        "              labelSize, baseLine = cv2.getTextSize(label, cv2.FONT_HERSHEY_SIMPLEX, 0.7, 2) # Get font size\n",
        "              label_ymin = max(ymin, labelSize[1] + 10) # Make sure not to draw label too close to top of window\n",
        "              cv2.rectangle(image, (xmin, label_ymin-labelSize[1]-10), (xmin+labelSize[0], label_ymin+baseLine-10), (255, 255, 255), cv2.FILLED) # Draw white box to put label text in\n",
        "              cv2.putText(image, label, (xmin, label_ymin-7), cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 0), 2) # Draw label text\n",
        "\n",
        "              detections.append([object_name, scores[i], xmin, ymin, xmax, ymax])\n",
        "\n",
        "\n",
        "      # All the results have been drawn on the image, now display the image\n",
        "      if txt_only == False: # \"text_only\" controls whether we want to display the image results or just save them in .txt files\n",
        "        image = cv2.cvtColor(image,cv2.COLOR_BGR2RGB)\n",
        "        plt.figure(figsize=(12,16))\n",
        "        plt.imshow(image)\n",
        "        plt.show()\n",
        "\n",
        "      # Save detection results in .txt files (for calculating mAP)\n",
        "      elif txt_only == True:\n",
        "\n",
        "        # Get filenames and paths\n",
        "        image_fn = os.path.basename(image_path)\n",
        "        base_fn, ext = os.path.splitext(image_fn)\n",
        "        txt_result_fn = base_fn +'.txt'\n",
        "        txt_savepath = os.path.join(savepath, txt_result_fn)\n",
        "\n",
        "        # Write results to text file\n",
        "        # (Using format defined by https://github.com/Cartucho/mAP, which will make it easy to calculate mAP)\n",
        "        with open(txt_savepath,'w') as f:\n",
        "            for detection in detections:\n",
        "                f.write('%s %.4f %d %d %d %d\\n' % (detection[0], detection[1], detection[2], detection[3], detection[4], detection[5]))\n",
        "\n",
        "  return"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The next block sets the paths to the test images and models and then runs the inferencing function. If you want to use more than 10 images, change the `images_to_test` variable. Click play to run inferencing!"
      ],
      "metadata": {
        "id": "-CJI4A0f_zqz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set up variables for running user's model\n",
        "PATH_TO_IMAGES='/content/images/test'   # Path to test images folder\n",
        "PATH_TO_MODEL='/content/custom_model_lite/detect.tflite'   # Path to .tflite model file\n",
        "PATH_TO_LABELS='/content/labelmap.txt'   # Path to labelmap.txt file\n",
        "min_conf_threshold=0.5   # Confidence threshold (try changing this to 0.01 if you don't see any detection results)\n",
        "images_to_test = 10   # Number of images to run detection on\n",
        "\n",
        "# Run inferencing function!\n",
        "tflite_detect_images(PATH_TO_MODEL, PATH_TO_IMAGES, PATH_TO_LABELS, min_conf_threshold, images_to_test)"
      ],
      "metadata": {
        "id": "6t8CMarqBqP9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### 7.2 Calculate mAP\n",
        "Now we have a visual sense of how our model performs on test images, but how can we quantitatively measure its accuracy?\n",
        "\n",
        "One popular methord for measuring object detection model accuracy is \"mean average precision\" (mAP). Basically, the higher the mAP score, the better your model is at detecting objects in images. To learn more about mAP, read through this [article from Roboflow](https://blog.roboflow.com/mean-average-precision/).\n",
        "\n",
        "We'll use the mAP calculator tool at https://github.com/Cartucho/mAP to determine our model's mAP score. First, we need to clone the repository and remove its existing example data. We'll also download a script I wrote for interfacing with the calculator."
      ],
      "metadata": {
        "id": "N_ckqeWqBF0P"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%bash\n",
        "git clone https://github.com/Cartucho/mAP /content/mAP\n",
        "cd /content/mAP\n",
        "rm input/detection-results/*\n",
        "rm input/ground-truth/*\n",
        "rm input/images-optional/*\n",
        "wget https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/util_scripts/calculate_map_cartucho.py"
      ],
      "metadata": {
        "id": "JlWarXEZDUqS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we'll copy the images and annotation data from the **test** folder to the appropriate folders inside the cloned repository. These will be used as the \"ground truth data\" that our model's detection results will be compared to.\n"
      ],
      "metadata": {
        "id": "qn22nGGqH5T6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cp /content/images/test/* /content/mAP/input/images-optional # Copy images and xml files\n",
        "!mv /content/mAP/input/images-optional/*.xml /content/mAP/input/ground-truth/  # Move xml files to the appropriate folder"
      ],
      "metadata": {
        "id": "5szFfVxwI3wT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The calculator tool expects annotation data in a format that's different from the Pascal VOC .xml file format we're using. Fortunately, it provides an easy script, `convert_gt_xml.py`, for converting to the expected .txt format.\n",
        "\n"
      ],
      "metadata": {
        "id": "u6aro817DGzx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!python /content/mAP/scripts/extra/convert_gt_xml.py"
      ],
      "metadata": {
        "id": "qdjtOUDnK2AA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Okay, we've set up the ground truth data, but now we need actual detection results from our model. The detection results will be compared to the ground truth data to calculate the model's accuracy in mAP.\n",
        "\n",
        "The inference function we defined in Step 7.1 can be used to generate detection data for all the images in the **test** folder. We'll use it the same as before, except this time we'll tell it to save detection results into the `detection-results` folder.\n",
        "\n",
        "Click Play to run the following code block!"
      ],
      "metadata": {
        "id": "mnIUacAlLP0B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set up variables for running inference, this time to get detection results saved as .txt files\n",
        "PATH_TO_IMAGES='/content/images/test'   # Path to test images folder\n",
        "PATH_TO_MODEL='/content/custom_model_lite/detect.tflite'   # Path to .tflite model file\n",
        "PATH_TO_LABELS='/content/labelmap.txt'   # Path to labelmap.txt file\n",
        "PATH_TO_RESULTS='/content/mAP/input/detection-results' # Folder to save detection results in\n",
        "min_conf_threshold=0.1   # Confidence threshold\n",
        "\n",
        "# Use all the images in the test folder\n",
        "image_list = glob.glob(PATH_TO_IMAGES + '/*.jpg') + glob.glob(PATH_TO_IMAGES + '/*.JPG') + glob.glob(PATH_TO_IMAGES + '/*.png') + glob.glob(PATH_TO_IMAGES + '/*.bmp')\n",
        "images_to_test = min(500, len(image_list)) # If there are more than 500 images in the folder, just use 500\n",
        "\n",
        "# Tell function to just save results and not display images\n",
        "txt_only = True\n",
        "\n",
        "# Run inferencing function!\n",
        "print('Starting inference on %d images...' % images_to_test)\n",
        "tflite_detect_images(PATH_TO_MODEL, PATH_TO_IMAGES, PATH_TO_LABELS, min_conf_threshold, images_to_test, PATH_TO_RESULTS, txt_only)\n",
        "print('Finished inferencing!')"
      ],
      "metadata": {
        "id": "szzHFAhsMNFF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, let's calculate mAP! One popular style for reporting mAP is the COCO metric for mAP @ 0.50:0.95. Basically, this means that mAP is calculated at several IoU thresholds between 0.50 and 0.95, and then the result from each threshold is averaged to get a final mAP score. [Learn more here!](https://blog.roboflow.com/mean-average-precision/)\n",
        "\n",
        "I wrote a script to run the calculator tool at each IoU threshold, average the results, and report the final accuracy score. It reports mAP for each class and overall mAP. Click Play on the following two blocks to calculate mAP!"
      ],
      "metadata": {
        "id": "e_QRnTqNPX4z"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%cd /content/mAP\n",
        "!python calculate_map_cartucho.py --labels=/content/labelmap.txt"
      ],
      "metadata": {
        "id": "3DkjpIBARTQ7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "The score reported at the end is your model's overall mAP score. Ideally, it should be above 50% (0.50). If it isn't, you can increase your model's accuracy by adding more images to your dataset. See my [dataset video](https://www.youtube.com/watch?v=v0ssiOY6cfg) for tips on how to capture good training images and improve accuracy."
      ],
      "metadata": {
        "id": "R9HPoOBVKvxU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 8.&nbsp;Deploy TensorFlow Lite Model"
      ],
      "metadata": {
        "id": "5i40ve0SCLaE"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "phT8vvzriqQp"
      },
      "source": [
        "Now that your custom model has been trained and converted to TFLite format, it's ready to be downloaded and deployed in an application! This section shows how to download the model and provides links to instructions for deploying it on the Raspberry Pi, your PC, or other edge devices."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 8.1. Download TFLite model\n",
        "\n",
        "Run the two following cells to copy the labelmap files into the model folder, compress it into a zip folder, and then download it. The zip folder contains the `detect.tflite` model and `labelmap.txt` labelmap files that are needed to run the model in your application."
      ],
      "metadata": {
        "id": "zq3L2IoP4VHp"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "awZMQGVqMpVL"
      },
      "outputs": [],
      "source": [
        "# Move labelmap and pipeline config files into TFLite model folder and zip it up\n",
        "!cp /content/labelmap.txt /content/custom_model_lite\n",
        "!cp /content/labelmap.pbtxt /content/custom_model_lite\n",
        "!cp /content/models/mymodel/pipeline_file.config /content/custom_model_lite\n",
        "\n",
        "%cd /content\n",
        "!zip -r custom_model_lite.zip custom_model_lite"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FVPfAGbNPV56"
      },
      "outputs": [],
      "source": [
        "from google.colab import files\n",
        "\n",
        "files.download('/content/custom_model_lite.zip')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The `custom_model_lite.zip` file containing the model will download into your Downloads folder. It's ready to be deployed on your device!"
      ],
      "metadata": {
        "id": "9Kb3ZBsMq95l"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GSJ2wgGCixy2"
      },
      "source": [
        "## 8.2. Deploy model\n",
        "TensorFlow Lite models can run on a wide variety of hardware, including PCs, embedded systems, and phones. This section provides instructions showing how to deploy your TFLite model on various devices.\n",
        "\n",
        "### 8.2.1. Deploy on Raspberry Pi\n",
        "TFLite models are great for running on the Raspberry Pi, because they require less processing power than regular TensorFlow vision models. The Pi can run TFLite models in near real-time.\n",
        "\n",
        "To run your new model on the Raspberry Pi, you'll have to install TensorFlow Lite and prepare a Python environment for your application. I provide step-by-step instructions on how to set up TFLite on the Pi in my video, [How To Run TensorFlow Lite on Raspberry Pi for Object Detection](https://youtu.be/aimSGOAUI8Y).\n",
        "\n",
        "[![Link to my YouTube video!](https://raw.githubusercontent.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/master/doc/YouTube_video1.JPG)](https://www.youtube.com/watch?v=aimSGOAUI8Y)\n",
        "\n",
        "Once you've completed all the steps in the video, move the `custom_model_lite.zip` file downloaded from this Colab session over to your Raspberry Pi into the `~/tflite1` folder. Move into the folder and unzip it by issuing:\n",
        "\n",
        "```\n",
        "cd ~/tflite1\n",
        "unzip custom_model_lite.zip\n",
        "```\n",
        "\n",
        "Then, run the image, video, or webcam TFLite detection program with the `--modeldir=fine_tuned_model_lite` argument. For example, to run the webcam detection program, issue:\n",
        "\n",
        "```\n",
        "python TFLite_detection_webcam.py --modeldir=custom_model_lite\n",
        "```\n",
        "\n",
        "A window will appear showing a live feed from your webcam with boxes drawn around detected objects in each frame.\n",
        "\n",
        "### 8.2.2. Deploy on Windows, Linux, or macOS\n",
        "Follow the instructions linked below to quickly set up your Windows, Linux, or macOS computer to run TFLite models. It only takes a few minutes! Running a model on your PC is good for quickly testing your model with a webcam. However, keep in mind that the TFLite Runtime is optimized for lower-power processors, and it won't utilize the full capability of your PC's processor.\n",
        "\n",
        "Here are links to the deployment guides for Windows, Linux, and macOS:\n",
        "* [How to Run TensorFlow Lite Models on Windows](https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/deploy_guides/Windows_TFLite_Guide.md)\n",
        "* *link to Linux guide to be added (but really it's the same as Raspberry Pi)*\n",
        "* *link to macOS guide to be added*\n",
        "\n",
        "### 8.2.3. Deploy on other Linux-based edge devices\n",
        "Instructions to be added! 🐧\n",
        "\n",
        "### 8.2.4. Deploy on Android\n",
        "Instructions to be added! 🤖\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 9.&nbsp;(Optional) Post-Training Quantization"
      ],
      "metadata": {
        "id": "WoptFnAhCSrR"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I54paUm8dUCr"
      },
      "source": [
        "Want to make your TFLite model run even faster? Quantize it! Quantizating a model converts its weights from 32-bit floating-point values to 8-bit integer values. This allows the quantized model to run faster and occupy less memory without too much reduction in accuracy.\n",
        "\n",
        "> Note: If you observe an obvious decrease in detection accuracy when quantizing your model with TF2, I recommend using TensorFlow 1 to quantize your model instead. TF1 supports quantization-aware training, which helps improve the accuracy of quantized models. The ssd-mobilenet-v2-quantized model from the TF1 Model Zoo has fast and accurate performance when trained with a custom dataset. Visit my [TFLite v1 Colab notebook](https://colab.research.google.com/github/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/Train_TFLite1_Object_Detection_Model.ipynb) for step-by-step instructions on how to train and quantize a model with TensorFlow 1."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 9.1. Quantize model\n",
        "We'll use the \"TFLiteConverter\" module to perform [post-training quantization](https://www.tensorflow.org/lite/performance/post_training_quantization) on the model. To quantize the model, we need to provide a representative dataset, which is a set of images that represent what the model will see when deployed in the field. First, we'll create a list of images to include in the representative dataset (we'll just use the images in the `train` folder).\n"
      ],
      "metadata": {
        "id": "VTyqlXFTJ0Uv"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XSNZtfj_k3NP"
      },
      "outputs": [],
      "source": [
        "# Get list of all images in train directory\n",
        "image_path = '/content/images/train'\n",
        "\n",
        "jpg_file_list = glob.glob(image_path + '/*.jpg')\n",
        "JPG_file_list = glob.glob(image_path + '/*.JPG')\n",
        "png_file_list = glob.glob(image_path + '/*.png')\n",
        "bmp_file_list = glob.glob(image_path + '/*.bmp')\n",
        "\n",
        "quant_image_list = jpg_file_list + JPG_file_list + png_file_list + bmp_file_list"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next, we'll define a function to yield images from our representative dataset. Refer to [TensorFlow's sample quantization code](https://colab.research.google.com/github/google-coral/tutorials/blob/master/retrain_classification_ptq_tf2.ipynb#scrollTo=kRDabW_u1wnv) to get a better understanding of what this is doing!"
      ],
      "metadata": {
        "id": "cqbH1VlEgiuy"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ORzx0XRErSLV"
      },
      "outputs": [],
      "source": [
        "# A generator that provides a representative dataset\n",
        "# Code modified from https://colab.research.google.com/github/google-coral/tutorials/blob/master/retrain_classification_ptq_tf2.ipynb\n",
        "\n",
        "# First, get input details for model so we know how to preprocess images\n",
        "interpreter = Interpreter(model_path=PATH_TO_MODEL) # PATH_TO_MODEL is defined in Step 7 above\n",
        "interpreter.allocate_tensors()\n",
        "input_details = interpreter.get_input_details()\n",
        "output_details = interpreter.get_output_details()\n",
        "height = input_details[0]['shape'][1]\n",
        "width = input_details[0]['shape'][2]\n",
        "\n",
        "import random\n",
        "\n",
        "def representative_data_gen():\n",
        "  dataset_list = quant_image_list\n",
        "  quant_num = 300\n",
        "  for i in range(quant_num):\n",
        "    pick_me = random.choice(dataset_list)\n",
        "    image = tf.io.read_file(pick_me)\n",
        "\n",
        "    if pick_me.endswith('.jpg') or pick_me.endswith('.JPG'):\n",
        "      image = tf.io.decode_jpeg(image, channels=3)\n",
        "    elif pick_me.endswith('.png'):\n",
        "      image = tf.io.decode_png(image, channels=3)\n",
        "    elif pick_me.endswith('.bmp'):\n",
        "      image = tf.io.decode_bmp(image, channels=3)\n",
        "\n",
        "    image = tf.image.resize(image, [width, height])  # TO DO: Replace 300s with an automatic way of reading network input size\n",
        "    image = tf.cast(image / 255., tf.float32)\n",
        "    image = tf.expand_dims(image, 0)\n",
        "    yield [image]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wqtu98mzebEj"
      },
      "source": [
        "Finally, we'll initialize the TFLiteConverter module, point it at the TFLite graph we generated in Step 6, and provide it with the representative dataset generator function we created in the previous code block. We'll configure the converter to quantize the model's weight values to INT8 format."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ox0bGDWds_Ce"
      },
      "outputs": [],
      "source": [
        "# Initialize converter module\n",
        "converter = tf.lite.TFLiteConverter.from_saved_model('/content/custom_model_lite/saved_model')\n",
        "\n",
        "# This enables quantization\n",
        "converter.optimizations = [tf.lite.Optimize.DEFAULT]\n",
        "# This sets the representative dataset for quantization\n",
        "converter.representative_dataset = representative_data_gen\n",
        "# This ensures that if any ops can't be quantized, the converter throws an error\n",
        "converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.TFLITE_BUILTINS_INT8]\n",
        "# For full integer quantization, though supported types defaults to int8 only, we explicitly declare it for clarity.\n",
        "converter.target_spec.supported_types = [tf.int8]\n",
        "# These set the input tensors to uint8 and output tensors to float32\n",
        "converter.inference_input_type = tf.uint8\n",
        "converter.inference_output_type = tf.float32\n",
        "tflite_model = converter.convert()\n",
        "\n",
        "with open('/content/custom_model_lite/detect_quant.tflite', 'wb') as f:\n",
        "  f.write(tflite_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 9.2. Test quantized model\n",
        "The model has been quantized and exported as `detect_quant.tflite`. Let's test it out! We'll re-use the function from Section 7 for running the model on test images and display the results, except this time we'll point it at the quantized model.\n",
        "\n",
        "Click Play on the code block below to test the `detect_quant.tflite` model."
      ],
      "metadata": {
        "id": "dYVVlv5QUUZF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set up parameters for inferencing function (using detect_quant.tflite instead of detect.tflite)\n",
        "PATH_TO_IMAGES='/content/images/test'   #Path to test images folder\n",
        "PATH_TO_MODEL='/content/custom_model_lite/detect_quant.tflite'   #Path to .tflite model file\n",
        "PATH_TO_LABELS='/content/labelmap.txt'   #Path to labelmap.txt file\n",
        "min_conf_threshold=0.5   #Confidence threshold (try changing this to 0.01 if you don't see any detection results)\n",
        "images_to_test = 10   #Number of images to run detection on\n",
        "\n",
        "# Run inferencing function!\n",
        "tflite_detect_images(PATH_TO_MODEL, PATH_TO_IMAGES, PATH_TO_LABELS, min_conf_threshold, images_to_test)"
      ],
      "metadata": {
        "id": "6OoirJuOtdOG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "If your quantized model isn't performing very well, try using my TensorFlow Lite 1 notebook *(link to be added)* to train a SSD-MobileNet model with your dataset. In my experience, the `ssd-mobilenet-v2-quantized` model from the [TF1 Model Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1_detection_zoo.md) has the best quantized performance out of any other TensorFlow Lite model.\n",
        "\n",
        "TFLite models created with TensorFlow 1 are still compatible with the TensorFlow Lite 2 runtime, so your TFLite 1 model will still work with my [TensorFlow setup guide for the Raspberry Pi](https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/Raspberry_Pi_Guide.md)."
      ],
      "metadata": {
        "id": "cKo7ZtfOyoxG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 9.3 Calculate quantized model mAP\n",
        "\n",
        "Let's calculate the quantize model's mAP using the calculator tool we set up in Step 7.2. We just need to perform inference with our quantized model (`detect_quant.tflite`) to get a new set of detection results.\n",
        "\n",
        "Run the following block to run inference on the test images and save the detection results."
      ],
      "metadata": {
        "id": "vWdVxs6LUjbR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Need to remove existing detection results first\n",
        "!rm /content/mAP/input/detection-results/*\n",
        "\n",
        "# Set up variables for running inference, this time to get detection results saved as .txt files\n",
        "PATH_TO_IMAGES='/content/images/test'   # Path to test images folder\n",
        "PATH_TO_MODEL='/content/custom_model_lite/detect_quant.tflite'   # Path to quantized .tflite model file\n",
        "PATH_TO_LABELS='/content/labelmap.txt'   # Path to labelmap.txt file\n",
        "PATH_TO_RESULTS='/content/mAP/input/detection-results' # Folder to save detection results in\n",
        "min_conf_threshold=0.1   # Confidence threshold\n",
        "\n",
        "# Use all the images in the test folder\n",
        "image_list = glob.glob(PATH_TO_IMAGES + '/*.jpg') + glob.glob(PATH_TO_IMAGES + '/*.JPG') + glob.glob(PATH_TO_IMAGES + '/*.png') + glob.glob(PATH_TO_IMAGES + '/*.bmp')\n",
        "images_to_test = min(500, len(image_list)) # If there are more than 500 images in the folder, just use 500\n",
        "\n",
        "# Tell function to just save results and not display images\n",
        "txt_only = True\n",
        "\n",
        "# Run inferencing function!\n",
        "print('Starting inference on %d images...' % images_to_test)\n",
        "tflite_detect_images(PATH_TO_MODEL, PATH_TO_IMAGES, PATH_TO_LABELS, min_conf_threshold, images_to_test, PATH_TO_RESULTS, txt_only)\n",
        "print('Finished inferencing!')"
      ],
      "metadata": {
        "id": "ZMaumV-11Et0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we can run the mAP calculation script to determine our quantized model's mAP."
      ],
      "metadata": {
        "id": "QgcmdLQf1Et1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "cd /content/mAP"
      ],
      "metadata": {
        "id": "ZIRNp0Af1Et1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!python calculate_map_cartucho.py --labels=/content/labelmap.txt"
      ],
      "metadata": {
        "id": "4TDgMBw_1Et1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XFsuasvxFHo8"
      },
      "source": [
        "## 9.4. Compile model for Edge TPU\n",
        "\n",
        "Now that the model has been converted to TFLite and quantized, we can compile it to run on Edge TPU devices like the [Coral USB Accelerator](https://coral.ai/products/accelerator/) or the [Coral Dev Board](https://coral.ai/products/dev-board/). This allows the model to run much faster! For information on how to set up the USB Accelerator, my [TensorFlow Lite repository on GitHub](https://github.com/EdjeElectronics/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi/blob/master/deploy_guides/Raspberry_Pi_Guide.md#section-2---run-edge-tpu-object-detection-models-on-the-raspberry-pi-using-the-coral-usb-accelerator).\n",
        "\n",
        "First, install the Edge TPU Compiler package inside this Colab instance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mUd_SNC0JSq0"
      },
      "outputs": [],
      "source": [
        "! curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -\n",
        "! echo \"deb https://packages.cloud.google.com/apt coral-edgetpu-stable main\" | sudo tee /etc/apt/sources.list.d/coral-edgetpu.list\n",
        "! sudo apt-get update\n",
        "! sudo apt-get install edgetpu-compiler"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "usfmdtSiJuuC"
      },
      "source": [
        "Next, compile the quantize TFLite model. (If your model has a different filename than \"detect_quant.tflite\", use that instead.)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mULCY0nb0ahH"
      },
      "outputs": [],
      "source": [
        "%cd /content/custom_model_lite\n",
        "!edgetpu_compiler detect_quant.tflite\n",
        "!mv detect_quant_edgetpu.tflite edgetpu.tflite\n",
        "!rm detect_quant_edgetpu.log"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oqGy2FgzKomN"
      },
      "source": [
        "The compiled model will be output in the `custom_model_lite` folder as \"detect__quant_edgetpu.tflite\". It gets renamed to \"edgetpu.tflite\" to be consistent with my code. Zip the `custom_model_lite` folder and download it by running the two code blocks below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8nCdUouYJjQM"
      },
      "outputs": [],
      "source": [
        "%cd /content\n",
        "!zip -r custom_model_lite.zip custom_model_lite"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AmjqvKuuK8ZR"
      },
      "outputs": [],
      "source": [
        "from google.colab import files\n",
        "\n",
        "files.download('custom_model_lite.zip')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ptwpBBEWLfuJ"
      },
      "source": [
        "Now you're all set to use the Coral model! For instructions on how to run an object detection model on the Raspberry Pi using the Coral USB Acclerator, please see my video, [\"How to Use the Coral USB Accelerator with the Raspberry Pi\"](https://www.youtube.com/watch?v=qJMwNHQNOVU)."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Appendix: Common Errors"
      ],
      "metadata": {
        "id": "5VI_Gh5dCd7w"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sEbd9cO7I_o3"
      },
      "source": [
        "Here are solutions to common errors that can occur while stepping through this notebook.\n",
        "\n",
        "**1. Training suddenly stops with ^C output**\n",
        "\n",
        "If your training randomly stops without any error messages except a `^C`, that means the virtual machine has run out of memory. To resolve the issue, try reducing the `batch_size` variable in Step 4 to a lower value like `batch_size = 4`. The value must be a power of 2. (e.g. 2, 4, 8 ...)\n",
        "\n",
        "Source: https://stackoverflow.com/questions/75901898/why-my-model-training-automatically-stopped-during-training"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "4VAvZo8qE4u5",
        "-19zML6oEO7l",
        "kPg8oMnQDYKl",
        "RDQrtQhvC3oG",
        "5i40ve0SCLaE",
        "WoptFnAhCSrR",
        "5VI_Gh5dCd7w"
      ],
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}