Home » Tutorials » TensorFlow » Transfer Learning on EfficientDet

Transfer Learning on EfficientDet

Published:

Transfer Learning is an effiicent way to train an existing model on a new and unique dataset with equivalent accuracy and significantly less training time.

View source on GitHub

Introduction

This tutorial demonstrates how to use transfer learning to detect road signs from a publicly available dataset using a pre-trained EfficientDet-D0 model on Habana Gaudi. The EfficientDet-D0 model was pre-trained with MS COCO dataset. In order to use the general features learned by the pre-trained model, it adds new class/box prediction networks on the top of pre-trained model. The weights of the pre-trained model remain frozen and are not updated. Only new prediction networks are trained from scratch.

Setup

This notebook runs on a Gaudi system with SynapseAI and Gaudi drivers installed. Please refer to installation guide for more information. Ensure that the five associated .py files are also included the ./root folder

Install Dependencies

Install the required packages as below.

%pip install pyyaml pycocotools opencv-python lxml scikit-learn
Requirement already satisfied: pyyaml in /usr/local/lib/python3.8/dist-packages (6.0)
Collecting pycocotools
  Downloading pycocotools-2.0.6.tar.gz (24 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting opencv-python
  Downloading opencv_python-4.7.0.68-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (61.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.8/61.8 MB 202.6 MB/s eta 0:00:00a 0:00:01
Collecting lxml
  Downloading lxml-4.9.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (7.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.1/7.1 MB 217.2 MB/s eta 0:00:00
Collecting scikit-learn
  Downloading scikit_learn-1.2.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (9.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.7/9.7 MB 225.0 MB/s eta 0:00:00a 0:00:01
Collecting matplotlib>=2.1.0
  Downloading matplotlib-3.6.2-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (9.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.4/9.4 MB 249.2 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from pycocotools) (1.23.5)
Collecting scipy>=1.3.2
  Downloading scipy-1.10.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (34.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 34.5/34.5 MB 135.3 MB/s eta 0:00:0000:0100:01
Collecting joblib>=1.1.1
  Downloading joblib-1.2.0-py3-none-any.whl (297 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 298.0/298.0 kB 382.1 MB/s eta 0:00:00
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=2.1.0->pycocotools) (3.0.9)
Collecting fonttools>=4.22.0
  Downloading fonttools-4.38.0-py3-none-any.whl (965 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 965.4/965.4 kB 368.9 MB/s eta 0:00:00
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.4.4-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 259.5 MB/s eta 0:00:00
Collecting contourpy>=1.0.1
  Downloading contourpy-1.0.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (295 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 296.0/296.0 kB 385.2 MB/s eta 0:00:00
Collecting cycler>=0.10
  Downloading cycler-0.11.0-py3-none-any.whl (6.4 kB)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=2.1.0->pycocotools) (2.8.2)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=2.1.0->pycocotools) (9.3.0)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=2.1.0->pycocotools) (21.3)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.7->matplotlib>=2.1.0->pycocotools) (1.16.0)
Building wheels for collected packages: pycocotools
  Building wheel for pycocotools (pyproject.toml) ... done
  Created wheel for pycocotools: filename=pycocotools-2.0.6-cp38-cp38-linux_x86_64.whl size=423715 sha256=ddcd82be74f49b46aa60e89443248adf95030366750d2954aba19e6a93cf6cbe
  Stored in directory: /tmp/pip-ephem-wheel-cache-pqkj8h3s/wheels/07/42/0a/26d14b3a7343223fcee09c101e73d82fff8bdd1541d85fe033
Successfully built pycocotools
Installing collected packages: threadpoolctl, scipy, opencv-python, lxml, kiwisolver, joblib, fonttools, cycler, contourpy, scikit-learn, matplotlib, pycocotools
Successfully installed contourpy-1.0.6 cycler-0.11.0 fonttools-4.38.0 joblib-1.2.0 kiwisolver-1.4.4 lxml-4.9.2 matplotlib-3.6.2 opencv-python-4.7.0.68 pycocotools-2.0.6 scikit-learn-1.2.0 scipy-1.10.0 threadpoolctl-3.1.0
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Note: you may need to restart the kernel to use updated packages.
Code language: JavaScript (javascript)

Clone the Model References repository and add the repository path to your PYTHONPATH environment variable. The transfer learning task depends on Habana’s EfficientDet modules in Model References repository.

!git clone -b 1.7.0 https://github.com/HabanaAI/Model-References.git
!export PYTHONPATH=/root/Model-References/TensorFlow/computer_vision/efficientdet:$PYTHONPATH
%cd /root/Model-References/TensorFlow/computer_vision/efficientdet
# move the associated .py files for Transfer learning into the efficientDet folder
%cp ~/*.py ~/Model-References/TensorFlow/computer_vision/efficientdet/
Cloning into 'Model-References'...
remote: Enumerating objects: 14504, done.
remote: Counting objects: 100% (14503/14503), done.
remote: Compressing objects: 100% (6228/6228), done.
remote: Total 14504 (delta 7903), reused 14437 (delta 7868), pack-reused 1
Receiving objects: 100% (14504/14504), 100.74 MiB | 33.55 MiB/s, done.
Resolving deltas: 100% (7903/7903), done.
/root/Model-References/TensorFlow/computer_vision/efficientdet
Code language: JavaScript (javascript)
import sys
sys.path.append('/root/Model-References/TensorFlow/computer_vision/efficientdet')
sys.path.append('/root/Model-References')

Data Preparation – Download Kaggle Dataset

Please download the road sign detection dataset from kaggle and place the annotations and images in the ./road_sign_data/annotations and ./road_sign_data/images folders.

Users bear sole liability and responsibility to follow and comply with any third party licenses, and Habana Labs disclaims and will bear no liability with respect to users’ use or compliance with third party licenses.

%mkdir road_sign_data
%cd road_sign_data
%mkdir annotations
%mkdir images
/root/Model-References/TensorFlow/computer_vision/efficientdet/road_sign_data

Now that the images are loaded from Kaggle, we need to Copy a handful of the images to be used for inference at the end of the tutorial

%cd images
%mkdir roadsign_infer_input_dir
%cp road415.png road418.png road536.png road560.png road7.png road854.png ./roadsign_infer_input_dir
/root/Model-References/TensorFlow/computer_vision/efficientdet/road_sign_data/images

Data Preparation – Convert to TFRecord

Convert the data into TFRecord format using create_roadsign_tfrecord.py. If you have downloaded the road sign dataset in ./road_sign_data, the following command will create TFRecord files under ./road_sign_tfrecord:

%cd /root/Model-References/TensorFlow/computer_vision/efficientdet
%mkdir road_sign_tfrecord
!python3 create_roadsign_tfrecord.py --data_dir ./road_sign_data --output_path ./road_sign_tfrecord/train --set train --num_shards 8
!python3 create_roadsign_tfrecord.py --data_dir ./road_sign_data --output_path ./road_sign_tfrecord/test --set test --num_shards 1
/root/Model-References/TensorFlow/computer_vision/efficientdet
2023-01-09 20:30:12.644509: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-09 20:30:12.754485: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
I0109 20:30:14.193033 140555461715776 create_roadsign_tfrecord.py:204] writing to output path: ./road_sign_tfrecord/train
I0109 20:30:14.193703 140555461715776 create_roadsign_tfrecord.py:228] Reading from road sign dataset.
I0109 20:30:14.194633 140555461715776 create_roadsign_tfrecord.py:240] On image 0 of 790
/root/Model-References/TensorFlow/computer_vision/efficientdet/dataset/tfrecord_util.py:78: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
  if not xml:
I0109 20:30:14.312039 140555461715776 create_roadsign_tfrecord.py:240] On image 100 of 790
I0109 20:30:14.436049 140555461715776 create_roadsign_tfrecord.py:240] On image 200 of 790
I0109 20:30:14.558966 140555461715776 create_roadsign_tfrecord.py:240] On image 300 of 790
I0109 20:30:14.685188 140555461715776 create_roadsign_tfrecord.py:240] On image 400 of 790
Traceback (most recent call last):
  File "create_roadsign_tfrecord.py", line 283, in <module>
    tf.app.run()
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/platform/app.py", line 36, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
  File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/usr/local/lib/python3.8/dist-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "create_roadsign_tfrecord.py", line 244, in main
    xml_str = fid.read()
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/lib/io/file_io.py", line 114, in read
    self._preread_check()
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/lib/io/file_io.py", line 76, in _preread_check
    self._read_buf = _pywrap_file_io.BufferedInputStream(
tensorflow.python.framework.errors_impl.NotFoundError: ./road_sign_data/annotations/roadsign_infer_input_dir.xml; No such file or directory
2023-01-09 20:30:15.711045: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-09 20:30:15.822879: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
I0109 20:30:17.239722 139850909525824 create_roadsign_tfrecord.py:204] writing to output path: ./road_sign_tfrecord/test
I0109 20:30:17.240286 139850909525824 create_roadsign_tfrecord.py:228] Reading from road sign dataset.
I0109 20:30:17.241228 139850909525824 create_roadsign_tfrecord.py:258] On image 0 of 790
/root/Model-References/TensorFlow/computer_vision/efficientdet/dataset/tfrecord_util.py:78: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead.
  if not xml:
I0109 20:30:17.306591 139850909525824 create_roadsign_tfrecord.py:258] On image 50 of 790
Code language: JavaScript (javascript)

In the following functions for transfer learning, assume that the dataset is already in TFRecord format.

Download Pre-trained Checkpoint

To run the transfer learning example, you need to have a pre-trained checkpoint. Habana provides EfficientDet-D0 checkpoint pre-trained on Habana Gaudi. Please download the checkpoint from Habana Vault.

%mkdir pretrained_ckpt 
%cd pretrained_ckpt
!wget https://vault.habana.ai/artifactory/pretrained-models/checkpoints/1.7.0/TF/EfficientDet/EfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz
!tar -xvf EfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz
%cd ~/Model-References/TensorFlow/computer_vision/efficientdet
/root/Model-References/TensorFlow/computer_vision/efficientdet/pretrained_ckpt
--2023-01-09 20:30:25--  https://vault.habana.ai/artifactory/pretrained-models/checkpoints/1.7.0/TF/EfficientDet/EfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz
Resolving vault.habana.ai (vault.habana.ai)... 35.164.198.2, 54.148.44.92, 54.186.236.167, ...
Connecting to vault.habana.ai (vault.habana.ai)|35.164.198.2|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://jfrog-prod-usw2-shared-oregon-main.s3.amazonaws.com/aol-hbnartuswest/filestore/d9/d9ebf85cd07f5ed164770795103eeec8a5806aa5?X-Artifactory-username=anonymous&X-Artifactory-repositoryKey=pretrained-models&X-Artifactory-artifactPath=checkpoints%2F1.7.0%2FTF%2FEfficientDet%2FEfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz&X-Artifactory-projectKey=default&x-jf-traceId=f456073efb8e346e&response-content-disposition=attachment%3Bfilename%3D%22EfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz%22&response-content-type=application%2Fx-gzip&X-Amz-Security-Token=FwoGZXIvYXdzEDYaDGPmQwQGK9A94B%2B5kCKlBDtHC5dlA3NIKhAUmdYFMw1A6oEgc8mNZ4j4f1igPuRjidmlkzqkRRYMq7t0OZrj9UjlcxPcwIZ5xZlWveEvD6VgxMxg7vhNYvzC6egHOdeq9xt6%2FJhVAEy%2BUq9%2Fcsl7Lyr1slS88bQCmUC2WitPxWuG2Ncr%2Fmnq0q17mN2h2FQ%2Bvqv6P8nYY28cNml2eN3DrY%2Fn9CyxpLcO%2Bw%2BKAb32anGqxWfCtfhoEtOz%2BWrlvLrLQmjKm8GGCKFsrnsdSE0e5K5NiufCs6zujuIbV4h14eOYNdUO%2Bo4U3WrcuVfefAtugNarVa7XIHUtuGDW4U1OJ7igBxppf%2FaeClcnJXAZS%2BoxqgeFQLQ9x7rsse1AF5ADEHKhmTSWnpfV%2B1FhRJs83W3Z%2BbejHrouR%2FffufngHcOonr9oJ1qCBDIE1D9Hmv1DCLGAnRFPZlgSuS2o8bzC%2FxaET6UkqEzmDItTBtFLwNjuOVpshy%2F2Izl6NWm2g56ew2sec0wz01OIllKPBiPcB9wPSZSdb7Ixv%2FxJ7AfALmhm7hk0Lsulq1gKTCJ8oAFOAOTSjN3MYWAhy%2BzDby0PMEEIeRQ0cB4NPlBQW1QFJmqot83oNW3c9BebBioRLkJYfOFfOh5q%2FCrbxcYDho%2FVGlIf2Ommmkv%2FfyU2ahQgGHiUBdX2Dpf0aGBdDIFVS%2BwwE21kwinBSMegRVg2VKiXde15c7bOaNC5%2BU6i2sI7L%2Fhx%2BlmXFSjq8PGdBjIrYgV3BZppNgR3SAuG%2FZZgBUoPEQSkiKp5atYBAUcGG57qhStmepMnlgEeVA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230109T203025Z&X-Amz-SignedHeaders=host&X-Amz-Expires=60&X-Amz-Credential=ASIASG3IHPL6UC2CWPEQ%2F20230109%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=16620fb66d941b5ca1d5e1a1ecdfb544f03aec4bfb9c6ae1c513d0d3dbb98fb5 [following]
--2023-01-09 20:30:25--  https://jfrog-prod-usw2-shared-oregon-main.s3.amazonaws.com/aol-hbnartuswest/filestore/d9/d9ebf85cd07f5ed164770795103eeec8a5806aa5?X-Artifactory-username=anonymous&X-Artifactory-repositoryKey=pretrained-models&X-Artifactory-artifactPath=checkpoints%2F1.7.0%2FTF%2FEfficientDet%2FEfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz&X-Artifactory-projectKey=default&x-jf-traceId=f456073efb8e346e&response-content-disposition=attachment%3Bfilename%3D%22EfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz%22&response-content-type=application%2Fx-gzip&X-Amz-Security-Token=FwoGZXIvYXdzEDYaDGPmQwQGK9A94B%2B5kCKlBDtHC5dlA3NIKhAUmdYFMw1A6oEgc8mNZ4j4f1igPuRjidmlkzqkRRYMq7t0OZrj9UjlcxPcwIZ5xZlWveEvD6VgxMxg7vhNYvzC6egHOdeq9xt6%2FJhVAEy%2BUq9%2Fcsl7Lyr1slS88bQCmUC2WitPxWuG2Ncr%2Fmnq0q17mN2h2FQ%2Bvqv6P8nYY28cNml2eN3DrY%2Fn9CyxpLcO%2Bw%2BKAb32anGqxWfCtfhoEtOz%2BWrlvLrLQmjKm8GGCKFsrnsdSE0e5K5NiufCs6zujuIbV4h14eOYNdUO%2Bo4U3WrcuVfefAtugNarVa7XIHUtuGDW4U1OJ7igBxppf%2FaeClcnJXAZS%2BoxqgeFQLQ9x7rsse1AF5ADEHKhmTSWnpfV%2B1FhRJs83W3Z%2BbejHrouR%2FffufngHcOonr9oJ1qCBDIE1D9Hmv1DCLGAnRFPZlgSuS2o8bzC%2FxaET6UkqEzmDItTBtFLwNjuOVpshy%2F2Izl6NWm2g56ew2sec0wz01OIllKPBiPcB9wPSZSdb7Ixv%2FxJ7AfALmhm7hk0Lsulq1gKTCJ8oAFOAOTSjN3MYWAhy%2BzDby0PMEEIeRQ0cB4NPlBQW1QFJmqot83oNW3c9BebBioRLkJYfOFfOh5q%2FCrbxcYDho%2FVGlIf2Ommmkv%2FfyU2ahQgGHiUBdX2Dpf0aGBdDIFVS%2BwwE21kwinBSMegRVg2VKiXde15c7bOaNC5%2BU6i2sI7L%2Fhx%2BlmXFSjq8PGdBjIrYgV3BZppNgR3SAuG%2FZZgBUoPEQSkiKp5atYBAUcGG57qhStmepMnlgEeVA%3D%3D&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230109T203025Z&X-Amz-SignedHeaders=host&X-Amz-Expires=60&X-Amz-Credential=ASIASG3IHPL6UC2CWPEQ%2F20230109%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Signature=16620fb66d941b5ca1d5e1a1ecdfb544f03aec4bfb9c6ae1c513d0d3dbb98fb5
Resolving jfrog-prod-usw2-shared-oregon-main.s3.amazonaws.com (jfrog-prod-usw2-shared-oregon-main.s3.amazonaws.com)... 52.92.193.17, 52.218.250.83, 52.92.181.65, ...
Connecting to jfrog-prod-usw2-shared-oregon-main.s3.amazonaws.com (jfrog-prod-usw2-shared-oregon-main.s3.amazonaws.com)|52.92.193.17|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 46061694 (44M) [application/x-gzip]
Saving to: ‘EfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz’

EfficientDet-Tensor 100%[===================>]  43.93M   106MB/s    in 0.4s    

2023-01-09 20:30:26 (106 MB/s) - ‘EfficientDet-TensorFlow-2.8.3-1.7.0-665-8n-checkpoint-1.tar.gz’ saved [46061694/46061694]

model.ckpt-0.data-00000-of-00001
graph.pbtxt
model.ckpt-0.meta
model.ckpt-0.index
checkpoint
/root/Model-References/TensorFlow/computer_vision/efficientdet
Code language: PHP (php)

Transfer Learning on Road Sign Detection Dataset

Import Dependent Modules

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import multiprocessing
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = "3"

import numpy as np
import tensorflow.compat.v1 as tf

from absl import flags
from absl import logging

from PIL import Image
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import copy

import det_model_fn
import efficientdet_arch

import dataloader_tl
import det_model_fn_tl
import hparams_config_tl
import utils
from inference import InferenceDriver
from inference import restore_ckpt, det_post_process, visualize_image, image_preprocess

from TensorFlow.common.debug import dump_callback
from TensorFlow.common.tb_utils import write_hparams_v1
from pathlib import Path
Using custom Horovod, path: /usr/local/lib/python3.8/dist-packages/horovod/tensorflow/mpi_lib.cpython-38-x86_64-linux-gnu.so
Code language: JavaScript (javascript)

Load Habana Modules

from habana_frameworks.tensorflow import load_habana_module
load_habana_module()

Prepare for Transfer Learning

Set up the configurations to prepare for transfer learning. Please make sure to update the training_file_pattern and validation_file_pattern to point to the preprocessed TFRecord files.

train_batch_size = 2
eval_batch_size = 2
eval_samples = 88
training_file_pattern = './road_sign_tfrecord/train-*'
validation_file_pattern = './road_sign_tfrecord/test-*'
num_examples_per_epoch = 789
num_epochs = 10
model_name = 'efficientdet-d0'
log_every_n_steps = 37
cp_every_n_steps = 1000
keep_checkpoint_max = 5

Define the functions for transfer learning and evaluation steps.

tf.get_logger().propagate = False

tf.disable_v2_behavior()
tf.compat.v1.enable_resource_variables()

def set_env():
    os.environ['TF_ADJUST_HUE_FUSED'] = '1'
    os.environ['TF_ADJUST_SATURATION_FUSED'] = '1'
    os.environ['TF_ENABLE_WINOGRAD_NONFUSED'] = '1'
    os.environ['TF_SYNC_ON_FINISH'] = '0'
    os.environ['TF_AUTOTUNE_THRESHOLD'] = '2'

def get_session_config(use_xla):
    config = tf.ConfigProto()

    config.allow_soft_placement = True
    config.log_device_placement = False
    
    config.intra_op_parallelism_threads = 1
    config.inter_op_parallelism_threads = 4

    return config

def build_estimator_params(mode, config, model_dir, ckpt):
  assert mode in ('train', 'eval')
  params = dict(
      config.as_dict(),
      model_name=model_name,
      num_epochs=num_epochs,
      iterations_per_loop=200,
      model_dir=model_dir,
      num_shards=1,
      num_examples_per_epoch=num_examples_per_epoch,
      use_tpu=False,
      backbone_ckpt=None,
      ckpt=ckpt,
      val_json_file=None,
      testdev_dir=None,
      mode=mode,
      dropout_rate=None
  )
  params['every_n_iter'] = log_every_n_steps
  params['dump_all_ranks'] = 0

  if mode == 'train':
    params['batch_size'] = train_batch_size
  else: # 'eval' mode
    params = dict(
      params,
      use_tpu=False,
      input_rand_hflip=False,
      is_training_bn=False,
    )
  return params


def build_estimator_config(mode, config, model_dir):
  assert mode in ('train', 'eval')
  if mode == 'train':
    run_config = tf.estimator.RunConfig(
      session_config=get_session_config(use_xla=False),
      save_checkpoints_steps=cp_every_n_steps,
      log_step_count_steps=log_every_n_steps,
      keep_checkpoint_max=keep_checkpoint_max)
  else: # 'eval' mode
    config_proto = tf.ConfigProto(allow_soft_placement=True, log_device_placement=False)

    tpu_config = tf.estimator.tpu.TPUConfig(
      iterations_per_loop=200,
      num_shards=1,
      per_host_input_for_training=tf.estimator.tpu.InputPipelineConfig
        .PER_HOST_V2)

    run_config = tf.estimator.tpu.RunConfig(
      model_dir=model_dir,
      log_step_count_steps=200,
      session_config=config_proto,
      tpu_config=tpu_config,
    )
  return run_config
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow/python/compat/v2_compat.py:107: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
Code language: JavaScript (javascript)

Function for Transfer Learning and Evaluation

After setting up the configurations such as input image files, model architectures, etc. the following run function will be used to perform transfer learning and evaluation tasks.

def run(mode, model_dir, ckpt, hparams):
  tf.logging.set_verbosity(tf.logging.FATAL)
  set_env()
    
  config = hparams_config_tl.get_detection_config(model_name)
  config.override(hparams)
    
  model_fn_instance = det_model_fn_tl.get_model_fn(model_name)

    
  if mode == 'train':
    params = build_estimator_params(mode, config, model_dir, ckpt)
    
    run_config = build_estimator_config(mode, config, model_dir)
    
    train_estimator = tf.estimator.Estimator(
            model_fn=model_fn_instance,
            model_dir=model_dir,
            config=run_config,
            params=params)
        
    input_fn = dataloader_tl.InputReaderTL(training_file_pattern,
                                      is_training=True,
                                      params=params,
                                      use_fake_data=False,
                                      is_deterministic=False)
    max_steps = int((num_epochs * num_examples_per_epoch) / (train_batch_size)) + 1
    
    tf.logging.set_verbosity(tf.logging.INFO)
    train_estimator.train(input_fn=input_fn, steps=max_steps)

  elif mode == 'eval':
    eval_params = build_estimator_params(mode, config, model_dir, ckpt)
    eval_config = build_estimator_config(mode, config, model_dir)

    eval_estimator = tf.estimator.tpu.TPUEstimator(
            model_fn=model_fn_instance,
            use_tpu=False,
            train_batch_size=train_batch_size,
            eval_batch_size=eval_batch_size,
            config=eval_config,
            params=eval_params)
    
    for ckpt in tf.train.checkpoints_iterator(
                                            model_dir,
                                            min_interval_secs=0,
                                            timeout=None,
                                            timeout_fn=None):

      logging.info('Starting to evaluate.')

      tf.logging.set_verbosity(tf.logging.INFO)
      eval_results = eval_estimator.evaluate(
                input_fn=dataloader_tl.InputReaderTL(validation_file_pattern,
                is_training=False),
                steps=eval_samples//eval_batch_size)
      logging.info('Eval results: %s', eval_results)

      try:
        current_step = int(os.path.basename(ckpt).split('-')[1])
      except IndexError:
        logging.info('%s has no global step info: stop!', ckpt)
        break

      utils.archive_ckpt(eval_results, eval_results['AP'], ckpt)
      total_step = int((num_epochs * num_examples_per_epoch) / train_batch_size)

      if current_step >= total_step:
        logging.info('Evaluation finished after training step %d', current_step)
        break
  else:
    logging.info('Mode not found.')

In the next step, we perform transfer learning using the pre-trained model as a starting point.

Transfer Learning

Now we are ready to perform transfer learning for the road sign detection dataset by using the pre-trained model’s weights. Transfer learning is to train the last few prediction layers of the EfficientDet-D0 model on the road sign detection dataset while freezing the base networks. The run function is called with the parameters defined by users. Set pretrained_ckpt to the path of the pre-trained checkpoint downloaded in the previous step. The trained model will be saved in model_dir_tl. Users can pass hyperparameters using hparams_tl:

  • num_classes is the number of object categories in the dataset. It is set to 5 because the road sign detection dataset has 5 classes. {0: ‘background’, 1:’traffic light’, 2:’stop’, 3:’speed limit’, 4:’crosswalk’}
  • train_scale_min and train_scale_max are for scale jittering. Each input image is resized with a randomly sampled scale factor between train_scale_min and train_scale_max.
  • use_bfloat16 enables Habana mixed precision training.
  • var_freeze_expr is to specify layers to freeze. By setting var_freeze_expr to “(efficientnet|fpn_cells|resample_p6)”, all previous layers before the class/box prediction networks are frozen.
  • lr_warmup_epoch skips learning rate warm-up.
pretrained_ckpt = './pretrained_ckpt/'
model_dir_tl = './roadsign_transfer_learning'
hparams_tl = 'num_classes=5,train_scale_min=0.8,train_scale_max=1.2,use_bfloat16=True,var_freeze_expr="(efficientnet|resample_p6|fpn_cells)",lr_warmup_epoch=0.0'

run('train', model_dir_tl, pretrained_ckpt, hparams_tl)
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow/python/training/training_util.py:396: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.
Instructions for updating:
Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.
WARNING:tensorflow:From /root/dataloader_tl.py:179: parallel_interleave (from tensorflow.python.data.experimental.ops.interleave_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Dataset.interleave(map_func, cycle_length, block_length, num_parallel_calls=tf.data.AUTOTUNE)` instead. If sloppy execution is desired, use `tf.data.Options.deterministic`.
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1176: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1176: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.cast` instead.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:ip-172-31-30-151: {
    "name": "efficientdet-d0",
    "image_size": 512,
    "input_rand_hflip": true,
    "train_scale_min": 0.8,
    "train_scale_max": 1.2,
    "autoaugment_policy": null,
    "num_classes": 5,
    "skip_crowd_during_training": true,
    "min_level": 3,
    "max_level": 7,
    "num_scales": 3,
    "aspect_ratios": [
        [
            1.0,
            1.0
        ],
        [
            1.4,
            0.7
        ],
        [
            0.7,
            1.4
        ]
    ],
    "anchor_scale": 4.0,
    "is_training_bn": true,
    "momentum": 0.9,
    "learning_rate": 0.08,
    "lr_warmup_init": 0.008,
    "lr_warmup_epoch": 0.0,
    "first_lr_drop_epoch": 200.0,
    "second_lr_drop_epoch": 250.0,
    "clip_gradients_norm": 10.0,
    "num_epochs": 10,
    "alpha": 0.25,
    "gamma": 1.5,
    "delta": 0.1,
    "box_loss_weight": 50.0,
    "weight_decay": 4e-05,
    "use_bfloat16": true,
    "box_class_repeats": 3,
    "fpn_cell_repeats": 3,
    "fpn_num_filters": 64,
    "separable_conv": true,
    "apply_bn_for_resampling": true,
    "conv_after_downsample": false,
    "conv_bn_relu_pattern": false,
    "use_native_resize_op": false,
    "pooling_type": null,
    "fpn_name": null,
    "fpn_config": null,
    "survival_prob": null,
    "lr_decay_method": "cosine",
    "moving_average_decay": 0.9998,
    "ckpt_var_scope": null,
    "var_exclude_expr": ".*/class-predict/.*",
    "backbone_name": "efficientnet-b0",
    "backbone_config": null,
    "var_freeze_expr": "(efficientnet|resample_p6|fpn_cells)",
    "resnet_depth": 50,
    "model_name": "efficientdet-d0",
    "iterations_per_loop": 200,
    "model_dir": "./roadsign_transfer_learning",
    "num_shards": 1,
    "num_examples_per_epoch": 789,
    "use_tpu": false,
    "backbone_ckpt": null,
    "ckpt": "./pretrained_ckpt/",
    "val_json_file": null,
    "testdev_dir": null,
    "mode": "train",
    "dropout_rate": null,
    "every_n_iter": 37,
    "dump_all_ranks": 0,
    "batch_size": 2
}
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/keras/layers/normalization/batch_normalization.py:562: _colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/tensorflow/python/profiler/internal/flops_registry.py:138: tensor_shape_from_node_def_name (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.tensor_shape_from_node_def_name`
INFO:tensorflow:ip-172-31-30-151: backbone params/flops = 3.595388M, 4.067005741B
INFO:tensorflow:building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
/root/Model-References/TensorFlow/computer_vision/efficientdet/efficientdet_arch.py:96: UserWarning: `tf.layers.conv2d` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.Conv2D` instead.
  feat = tf.layers.conv2d(
/root/Model-References/TensorFlow/computer_vision/efficientdet/efficientdet_arch.py:119: UserWarning: `tf.layers.max_pooling2d` is deprecated and will be removed in a future version. Please use `tf.keras.layers.MaxPooling2D` instead.
  feat = tf.layers.max_pooling2d(
/root/Model-References/TensorFlow/computer_vision/efficientdet/efficientdet_arch.py:516: UserWarning: `tf.layers.separable_conv2d` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.SeparableConv2D` instead.
  new_node = conv_op(
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: backbone+fpn params/flops = 3.791669M, 4.363357977B
/root/Model-References/TensorFlow/computer_vision/efficientdet/efficientdet_arch.py:180: UserWarning: `tf.layers.separable_conv2d` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.SeparableConv2D` instead.
  images = conv_op(
/root/Model-References/TensorFlow/computer_vision/efficientdet/efficientdet_arch.py:199: UserWarning: `tf.layers.separable_conv2d` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.SeparableConv2D` instead.
  classes = conv_op(
/root/Model-References/TensorFlow/computer_vision/efficientdet/efficientdet_arch.py:224: UserWarning: `tf.layers.separable_conv2d` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.SeparableConv2D` instead.
  images = conv_op(
/root/Model-References/TensorFlow/computer_vision/efficientdet/efficientdet_arch.py:243: UserWarning: `tf.layers.separable_conv2d` is deprecated and will be removed in a future version. Please Use `tf.keras.layers.SeparableConv2D` instead.
  boxes = conv_op(
INFO:tensorflow:ip-172-31-30-151: backbone+fpn+box params/flops = 3.830342M, 4.755610185B
global_step:0 - []
efficientnet-b0/stem/conv2d/kernel:0 - [3, 3, 3, 32]
                    /tpu_batch_normalization/gamma:0 - [32]
                                            /beta:0 - [32]
                                            /moving_mean:0 - [32]
                                            /moving_variance:0 - [32]
---
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...
INFO:tensorflow:Saving checkpoints for 0 into ./roadsign_transfer_learning/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...
INFO:tensorflow:loss = 1.3171757, step = 0
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.0025, trainloss/cls_loss = 1.1528778, trainloss/box_loss = 0.0014949199, trainloss/det_loss = 1.2276238, trainloss/l2_loss = 0.089551896, trainloss/loss = 1.3171757, loss = 1.3171757, gnorm = 0.9507865, step = 0, step_time = 56.17737889289856
INFO:tensorflow:global_step/sec: 0.620444
INFO:tensorflow:loss = 1.365207, step = 37 (59.636 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.0024994574, trainloss/cls_loss = 1.08917, trainloss/box_loss = 0.0037296268, trainloss/det_loss = 1.2756513, trainloss/l2_loss = 0.089555606, trainloss/loss = 1.365207, loss = 1.365207, gnorm = 1.3043177, step = 37, step_time = 29.238378286361694
INFO:tensorflow:global_step/sec: 53.6772
INFO:tensorflow:loss = 1.2420802, step = 74 (0.688 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.0024978302, trainloss/cls_loss = 0.99094343, trainloss/box_loss = 0.0032314616, trainloss/det_loss = 1.1525165, trainloss/l2_loss = 0.08956372, trainloss/loss = 1.2420802, loss = 1.2420802, gnorm = 1.6397538, step = 74, step_time = 0.02148723602294922
INFO:tensorflow:global_step/sec: 1.84119
INFO:tensorflow:loss = 1.095116, step = 111 (20.097 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.0024951196, trainloss/cls_loss = 0.8337222, trainloss/box_loss = 0.0034364709, trainloss/det_loss = 1.0055457, trainloss/l2_loss = 0.08957032, trainloss/loss = 1.095116, loss = 1.095116, gnorm = 1.6060473, step = 111, step_time = 0.022833824157714844
INFO:tensorflow:global_step/sec: 54.1106
INFO:tensorflow:loss = 0.73057675, step = 148 (0.682 sec)
---
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.0003652372, trainloss/cls_loss = 0.30853012, trainloss/box_loss = 0.0011691295, trainloss/det_loss = 0.3669866, trainloss/l2_loss = 0.0897358, trainloss/loss = 0.4567224, loss = 0.4567224, gnorm = 1.6575912, step = 2960, step_time = 0.02270364761352539
INFO:tensorflow:global_step/sec: 51.1702
INFO:tensorflow:loss = 0.38455752, step = 2997 (0.723 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00033960736, trainloss/cls_loss = 0.23391595, trainloss/box_loss = 0.0012181017, trainloss/det_loss = 0.29482102, trainloss/l2_loss = 0.0897365, trainloss/loss = 0.38455752, loss = 0.38455752, gnorm = 1.2247577, step = 2997, step_time = 0.022186279296875
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 3000...
INFO:tensorflow:Saving checkpoints for 3000 into ./roadsign_transfer_learning/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 3000...
INFO:tensorflow:global_step/sec: 12.449
INFO:tensorflow:loss = 0.54368114, step = 3034 (2.972 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.0003147678, trainloss/cls_loss = 0.36714843, trainloss/box_loss = 0.0017359259, trainloss/det_loss = 0.45394474, trainloss/l2_loss = 0.089736395, trainloss/loss = 0.54368114, loss = 0.54368114, gnorm = 2.5524006, step = 3034, step_time = 0.02264857292175293
INFO:tensorflow:global_step/sec: 50.4167
INFO:tensorflow:loss = 0.35340744, step = 3071 (0.734 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00029074028, trainloss/cls_loss = 0.24375314, trainloss/box_loss = 0.0003983433, trainloss/det_loss = 0.2636703, trainloss/l2_loss = 0.08973715, trainloss/loss = 0.35340744, loss = 0.35340744, gnorm = 1.7057644, step = 3071, step_time = 0.02365565299987793
INFO:tensorflow:global_step/sec: 50.1099
INFO:tensorflow:loss = 0.4142894, step = 3108 (0.738 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00026754543, trainloss/cls_loss = 0.2769215, trainloss/box_loss = 0.00095261, trainloss/det_loss = 0.324552, trainloss/l2_loss = 0.08973743, trainloss/loss = 0.4142894, loss = 0.4142894, gnorm = 2.3052468, step = 3108, step_time = 0.022006511688232422
INFO:tensorflow:global_step/sec: 49.9603
INFO:tensorflow:loss = 0.36595383, step = 3145 (0.741 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00024520344, trainloss/cls_loss = 0.23737893, trainloss/box_loss = 0.0007767526, trainloss/det_loss = 0.27621657, trainloss/l2_loss = 0.08973727, trainloss/loss = 0.36595383, loss = 0.36595383, gnorm = 1.8468841, step = 3145, step_time = 0.022057056427001953
INFO:tensorflow:global_step/sec: 50.2301
INFO:tensorflow:loss = 0.6034251, step = 3182 (0.737 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.000223734, trainloss/cls_loss = 0.45375285, trainloss/box_loss = 0.0011987037, trainloss/det_loss = 0.513688, trainloss/l2_loss = 0.089737065, trainloss/loss = 0.6034251, loss = 0.6034251, gnorm = 4.113144, step = 3182, step_time = 0.021676301956176758
INFO:tensorflow:global_step/sec: 49.8141
INFO:tensorflow:loss = 0.2933924, step = 3219 (0.743 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00020315513, trainloss/cls_loss = 0.17689353, trainloss/box_loss = 0.00053523295, trainloss/det_loss = 0.20365518, trainloss/l2_loss = 0.08973721, trainloss/loss = 0.2933924, loss = 0.2933924, gnorm = 1.2557589, step = 3219, step_time = 0.022655725479125977
INFO:tensorflow:global_step/sec: 50.0401
INFO:tensorflow:loss = 0.4638117, step = 3256 (0.739 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00018348522, trainloss/cls_loss = 0.31669247, trainloss/box_loss = 0.0011476396, trainloss/det_loss = 0.37407446, trainloss/l2_loss = 0.08973725, trainloss/loss = 0.4638117, loss = 0.4638117, gnorm = 3.5361226, step = 3256, step_time = 0.022051572799682617
INFO:tensorflow:global_step/sec: 50.0615
INFO:tensorflow:loss = 0.40125006, step = 3293 (0.739 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.0001647412, trainloss/cls_loss = 0.2582486, trainloss/box_loss = 0.0010652812, trainloss/det_loss = 0.31151265, trainloss/l2_loss = 0.08973742, trainloss/loss = 0.40125006, loss = 0.40125006, gnorm = 2.2921462, step = 3293, step_time = 0.02237677574157715
INFO:tensorflow:global_step/sec: 49.8302
INFO:tensorflow:loss = 0.31020695, step = 3330 (0.742 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00014693923, trainloss/cls_loss = 0.20007302, trainloss/box_loss = 0.0004079268, trainloss/det_loss = 0.22046936, trainloss/l2_loss = 0.0897376, trainloss/loss = 0.31020695, loss = 0.31020695, gnorm = 1.5963246, step = 3330, step_time = 0.02227926254272461
INFO:tensorflow:global_step/sec: 49.9879
INFO:tensorflow:loss = 0.48326403, step = 3367 (0.740 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00013009466, trainloss/cls_loss = 0.36006755, trainloss/box_loss = 0.0006691792, trainloss/det_loss = 0.3935265, trainloss/l2_loss = 0.08973752, trainloss/loss = 0.48326403, loss = 0.48326403, gnorm = 2.0124254, step = 3367, step_time = 0.022183656692504883
INFO:tensorflow:global_step/sec: 50.4199
INFO:tensorflow:loss = 0.4442764, step = 3404 (0.734 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 0.00011422269, trainloss/cls_loss = 0.26125792, trainloss/box_loss = 0.0018656175, trainloss/det_loss = 0.3545388, trainloss/l2_loss = 0.0897376, trainloss/loss = 0.4442764, loss = 0.4442764, gnorm = 3.0906272, step = 3404, step_time = 0.022417545318603516
INFO:tensorflow:global_step/sec: 50.1484
INFO:tensorflow:loss = 0.27552527, step = 3441 (0.738 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 9.933643e-05, trainloss/cls_loss = 0.16401397, trainloss/box_loss = 0.00043547424, trainloss/det_loss = 0.18578768, trainloss/l2_loss = 0.089737594, trainloss/loss = 0.27552527, loss = 0.27552527, gnorm = 1.2029936, step = 3441, step_time = 0.02284526824951172
INFO:tensorflow:global_step/sec: 49.678
INFO:tensorflow:loss = 0.41166908, step = 3478 (0.746 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 8.544929e-05, trainloss/cls_loss = 0.28208044, trainloss/box_loss = 0.0007970235, trainloss/det_loss = 0.3219316, trainloss/l2_loss = 0.089737475, trainloss/loss = 0.41166908, loss = 0.41166908, gnorm = 3.3671312, step = 3478, step_time = 0.022664308547973633
INFO:tensorflow:global_step/sec: 49.7911
INFO:tensorflow:loss = 1.0225842, step = 3515 (0.743 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 7.257298e-05, trainloss/cls_loss = 0.73870814, trainloss/box_loss = 0.0038827718, trainloss/det_loss = 0.9328467, trainloss/l2_loss = 0.089737505, trainloss/loss = 1.0225842, loss = 1.0225842, gnorm = 3.1646774, step = 3515, step_time = 0.022260189056396484
INFO:tensorflow:global_step/sec: 50.4506
INFO:tensorflow:loss = 0.5086609, step = 3552 (0.733 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 6.0718656e-05, trainloss/cls_loss = 0.26030445, trainloss/box_loss = 0.00317238, trainloss/det_loss = 0.41892344, trainloss/l2_loss = 0.089737505, trainloss/loss = 0.5086609, loss = 0.5086609, gnorm = 2.086274, step = 3552, step_time = 0.02291274070739746
INFO:tensorflow:global_step/sec: 50.8777
INFO:tensorflow:loss = 0.30778006, step = 3589 (0.727 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 4.9897135e-05, trainloss/cls_loss = 0.19322756, trainloss/box_loss = 0.00049630034, trainloss/det_loss = 0.21804258, trainloss/l2_loss = 0.089737475, trainloss/loss = 0.30778006, loss = 0.30778006, gnorm = 1.1423111, step = 3589, step_time = 0.0222012996673584
INFO:tensorflow:global_step/sec: 50.2484
INFO:tensorflow:loss = 0.5591459, step = 3626 (0.736 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 4.0117055e-05, trainloss/cls_loss = 0.3903717, trainloss/box_loss = 0.0015807328, trainloss/det_loss = 0.46940833, trainloss/l2_loss = 0.089737594, trainloss/loss = 0.5591459, loss = 0.5591459, gnorm = 4.5914807, step = 3626, step_time = 0.022908926010131836
INFO:tensorflow:global_step/sec: 50.4464
INFO:tensorflow:loss = 0.4445104, step = 3663 (0.733 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 3.1387583e-05, trainloss/cls_loss = 0.2991038, trainloss/box_loss = 0.0011133794, trainloss/det_loss = 0.35477278, trainloss/l2_loss = 0.08973763, trainloss/loss = 0.4445104, loss = 0.4445104, gnorm = 2.3967123, step = 3663, step_time = 0.02254796028137207
INFO:tensorflow:global_step/sec: 49.8922
INFO:tensorflow:loss = 0.38806683, step = 3700 (0.742 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 2.3715867e-05, trainloss/cls_loss = 0.2500372, trainloss/box_loss = 0.0009658404, trainloss/det_loss = 0.2983292, trainloss/l2_loss = 0.08973762, trainloss/loss = 0.38806683, loss = 0.38806683, gnorm = 2.021242, step = 3700, step_time = 0.025778532028198242
INFO:tensorflow:global_step/sec: 50.568
INFO:tensorflow:loss = 0.5523442, step = 3737 (0.731 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 1.7108769e-05, trainloss/cls_loss = 0.2514748, trainloss/box_loss = 0.0042226356, trainloss/det_loss = 0.46260658, trainloss/l2_loss = 0.089737624, trainloss/loss = 0.5523442, loss = 0.5523442, gnorm = 1.8378862, step = 3737, step_time = 0.021888017654418945
INFO:tensorflow:global_step/sec: 49.9676
INFO:tensorflow:loss = 0.38872015, step = 3774 (0.740 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 1.1571943e-05, trainloss/cls_loss = 0.27719787, trainloss/box_loss = 0.00043569325, trainloss/det_loss = 0.29898253, trainloss/l2_loss = 0.08973762, trainloss/loss = 0.38872015, loss = 0.38872015, gnorm = 3.5920844, step = 3774, step_time = 0.022457599639892578
INFO:tensorflow:global_step/sec: 49.7899
INFO:tensorflow:loss = 0.38265115, step = 3811 (0.743 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 7.110238e-06, trainloss/cls_loss = 0.22550912, trainloss/box_loss = 0.0013480883, trainloss/det_loss = 0.29291353, trainloss/l2_loss = 0.08973762, trainloss/loss = 0.38265115, loss = 0.38265115, gnorm = 2.6317253, step = 3811, step_time = 0.023236513137817383
INFO:tensorflow:global_step/sec: 49.9752
INFO:tensorflow:loss = 0.35137, step = 3848 (0.740 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 3.7275254e-06, trainloss/cls_loss = 0.22654226, trainloss/box_loss = 0.00070180267, trainloss/det_loss = 0.26163238, trainloss/l2_loss = 0.08973762, trainloss/loss = 0.35137, loss = 0.35137, gnorm = 2.1286929, step = 3848, step_time = 0.023079395294189453
INFO:tensorflow:global_step/sec: 50.1127
INFO:tensorflow:loss = 0.44619188, step = 3885 (0.738 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 1.4265627e-06, trainloss/cls_loss = 0.26969928, trainloss/box_loss = 0.0017350997, trainloss/det_loss = 0.35645425, trainloss/l2_loss = 0.08973762, trainloss/loss = 0.44619188, loss = 0.44619188, gnorm = 1.7004699, step = 3885, step_time = 0.021839380264282227
INFO:tensorflow:global_step/sec: 50.6964
INFO:tensorflow:loss = 0.41729945, step = 3922 (0.730 sec)
INFO:tensorflow:ip-172-31-30-151: logging tensors: lrn_rate = 2.0965933e-07, trainloss/cls_loss = 0.31155214, trainloss/box_loss = 0.00032019374, trainloss/det_loss = 0.32756183, trainloss/l2_loss = 0.08973762, trainloss/loss = 0.41729945, loss = 0.41729945, gnorm = 2.423685, step = 3922, step_time = 0.02083277702331543
INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 3946...
INFO:tensorflow:Saving checkpoints for 3946 into ./roadsign_transfer_learning/model.ckpt.
INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 3946...
INFO:tensorflow:Loss for final step: 0.2902587.
Code language: JavaScript (javascript)

Evaluate the Model

After the transfer learning process completes, now we are ready to evalute the model. To check the accuracy, call the previously defined run function in ‘eval’ mode.

run('eval', model_dir_tl, model_dir_tl, hparams_tl)
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Running eval on CPU/GPU
INFO:tensorflow:ip-172-31-30-151: {
    "name": "efficientdet-d0",
    "image_size": 512,
    "input_rand_hflip": false,
    "train_scale_min": 0.8,
    "train_scale_max": 1.2,
    "autoaugment_policy": null,
    "num_classes": 5,
    "skip_crowd_during_training": true,
    "min_level": 3,
    "max_level": 7,
    "num_scales": 3,
    "aspect_ratios": [
        [
            1.0,
            1.0
        ],
        [
            1.4,
            0.7
        ],
        [
            0.7,
            1.4
        ]
    ],
    "anchor_scale": 4.0,
    "is_training_bn": false,
    "momentum": 0.9,
    "learning_rate": 0.08,
    "lr_warmup_init": 0.008,
    "lr_warmup_epoch": 0.0,
    "first_lr_drop_epoch": 200.0,
    "second_lr_drop_epoch": 250.0,
    "clip_gradients_norm": 10.0,
    "num_epochs": 10,
    "alpha": 0.25,
    "gamma": 1.5,
    "delta": 0.1,
    "box_loss_weight": 50.0,
    "weight_decay": 4e-05,
    "use_bfloat16": true,
    "box_class_repeats": 3,
    "fpn_cell_repeats": 3,
    "fpn_num_filters": 64,
    "separable_conv": true,
    "apply_bn_for_resampling": true,
    "conv_after_downsample": false,
    "conv_bn_relu_pattern": false,
    "use_native_resize_op": false,
    "pooling_type": null,
    "fpn_name": null,
    "fpn_config": null,
    "survival_prob": null,
    "lr_decay_method": "cosine",
    "moving_average_decay": 0.9998,
    "ckpt_var_scope": null,
    "var_exclude_expr": ".*/class-predict/.*",
    "backbone_name": "efficientnet-b0",
    "backbone_config": null,
    "var_freeze_expr": "(efficientnet|resample_p6|fpn_cells)",
    "resnet_depth": 50,
    "model_name": "efficientdet-d0",
    "iterations_per_loop": 200,
    "model_dir": "./roadsign_transfer_learning",
    "num_shards": 1,
    "num_examples_per_epoch": 789,
    "use_tpu": false,
    "backbone_ckpt": null,
    "ckpt": "./roadsign_transfer_learning",
    "val_json_file": null,
    "testdev_dir": null,
    "mode": "eval",
    "dropout_rate": null,
    "every_n_iter": 37,
    "dump_all_ranks": 0,
    "batch_size": 2
}
INFO:tensorflow:ip-172-31-30-151: backbone params/flops = 3.595388M, 3.892550245B
INFO:tensorflow:building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: backbone+fpn params/flops = 3.791669M, 4.180154000B
INFO:tensorflow:ip-172-31-30-151: backbone+fpn+box params/flops = 3.830342M, 4.561922048B
global_step:0 - []
efficientnet-b0/stem/conv2d/kernel:0 - [3, 3, 3, 32]
                    /tpu_batch_normalization/gamma:0 - [32]
                                            /beta:0 - [32]
                                            /moving_mean:0 - [32]
                                            /moving_variance:0 - [32]
---
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2023-01-09T20:38:01
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from ./roadsign_transfer_learning/model.ckpt-3946
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Evaluation [4/44]
INFO:tensorflow:Evaluation [8/44]
INFO:tensorflow:Evaluation [12/44]
INFO:tensorflow:Evaluation [16/44]
INFO:tensorflow:Evaluation [20/44]
INFO:tensorflow:Evaluation [24/44]
INFO:tensorflow:Evaluation [28/44]
INFO:tensorflow:Evaluation [32/44]
INFO:tensorflow:Evaluation [36/44]
INFO:tensorflow:Evaluation [40/44]
INFO:tensorflow:Evaluation [44/44]
creating index...
index created!
Loading and preparing results...
Converting ndarray to lists...
(8800, 7)
0/8800
DONE (t=0.02s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=0.24s).
Accumulating evaluation results...
DONE (t=0.06s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.322
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.492
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.390
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.531
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.549
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.675
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.682
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.682
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000
INFO:tensorflow:Inference Time : 16.06437s
INFO:tensorflow:Finished evaluation at 2023-01-09-20:38:17
INFO:tensorflow:Saving dict for global step 3946: AP = 0.32231948, AP50 = 0.49190423, AP75 = 0.38954064, APl = -1.0, APm = -1.0, APs = 0.5311195, ARl = -1.0, ARm = -1.0, ARmax1 = 0.54910135, ARmax10 = 0.6750149, ARmax100 = 0.68208057, ARs = 0.68208057, box_loss = 0.001763712, cls_loss = 1.9835176, global_step = 3946, loss = 2.1614404
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 3946: ./roadsign_transfer_learning/model.ckpt-3946
INFO:tensorflow:evaluation_loop marked as finished
Code language: PHP (php)

Inference Examples

Now, let’s visualize our transfer learning results by predicting object detection from the test dataset. The following source code loads the model architecture, restore weights from the finetuned model and then perform the inference. The inference results are visualized with predicted probabilities and bounding boxes.

%matplotlib inline

def build_inputs(image_path_pattern, image_size):
  """Read and preprocess input images.

  Args:
    image_path_pattern: a path to indicate a single or multiple files.
    image_size: a single integer for image width and height.

  Returns:
    (raw_images, images, scales): raw images, processed images, and scales.

  Raises:
    ValueError if image_path_pattern doesn't match any file.
  """
  raw_images, images, scales = [], [], []
  for f in tf.io.gfile.glob(image_path_pattern):
    image = Image.open(f)
    image = image.convert('RGB')
    raw_images.append(image)
    image, scale = image_preprocess(image, image_size)
    images.append(image)
    scales.append(scale)
  if not images:
    raise ValueError(
        'Cannot find any images for pattern {}'.format(image_path_pattern))
  return raw_images, tf.stack(images), tf.stack(scales)

def build_model(model_name, inputs, config):
    model_arch = det_model_fn.get_model_arch(model_name)
    class_outputs, box_outputs = model_arch(inputs, model_name=model_name, config=config)
    return class_outputs, box_outputs

tf.reset_default_graph()
tf.disable_v2_behavior()
tf.compat.v1.disable_eager_execution()
tf.compat.v1.enable_resource_variables()

image_path_pattern = "./road_sign_data/images/roadsign_infer_input_dir/*.png"
label_id_mapping = {1:'traffic light', 2:'stop', 3:'speed limit', 4:'crosswalk'}

config = hparams_config_tl.get_detection_config(model_name)
config.update(dict(dropout_rate=None, batch_size=1, is_training_bn=False, use_bfloat16=False, num_classes=5))

sess = tf.Session()
raw_images, images, scales = build_inputs(image_path_pattern, 512)
class_outputs, box_outputs = build_model(model_name, images, config)

restore_ckpt(sess, model_dir_tl, enable_ema=False, export_ckpt=None)
config.update(dict(batch_size=len(raw_images),disable_pyfun=False))
detections_batch = det_post_process(config.as_dict(), class_outputs, box_outputs, scales)
outputs_np = sess.run(detections_batch) # [image_id, y, x, height, width, score, class]


fig, ax = plt.subplots(2,3,figsize=(8,8),layout="compressed")
ax = ax.flatten()
for i, output_np in enumerate(outputs_np):
    boxes = output_np[:, 1:5]
    classes = output_np[:, 6].astype(int)
    scores = output_np[:, 5]
    
    boxes[:, [0, 1, 2, 3]] = boxes[:, [1, 0, 3, 2]] # [x, y, width, height] to [ymin, xmin, ymax, xmax]

    boxes[:, 2:4] += boxes[:, 0:2]
    img = visualize_image(raw_images[i], boxes, classes, scores,
                        id_mapping=label_id_mapping, min_score_thresh=0.4, line_thickness=7)
    ax[i].imshow(img)
    ax[i].axis('off')
  
plt.show()
INFO:tensorflow:ip-172-31-30-151: {
    "name": "efficientdet-d0",
    "image_size": 512,
    "input_rand_hflip": true,
    "train_scale_min": 0.1,
    "train_scale_max": 2.0,
    "autoaugment_policy": null,
    "num_classes": 5,
    "skip_crowd_during_training": true,
    "min_level": 3,
    "max_level": 7,
    "num_scales": 3,
    "aspect_ratios": [
        [
            1.0,
            1.0
        ],
        [
            1.4,
            0.7
        ],
        [
            0.7,
            1.4
        ]
    ],
    "anchor_scale": 4.0,
    "is_training_bn": false,
    "momentum": 0.9,
    "learning_rate": 0.08,
    "lr_warmup_init": 0.008,
    "lr_warmup_epoch": 1.0,
    "first_lr_drop_epoch": 200.0,
    "second_lr_drop_epoch": 250.0,
    "clip_gradients_norm": 10.0,
    "num_epochs": 300,
    "alpha": 0.25,
    "gamma": 1.5,
    "delta": 0.1,
    "box_loss_weight": 50.0,
    "weight_decay": 4e-05,
    "use_bfloat16": false,
    "box_class_repeats": 3,
    "fpn_cell_repeats": 3,
    "fpn_num_filters": 64,
    "separable_conv": true,
    "apply_bn_for_resampling": true,
    "conv_after_downsample": false,
    "conv_bn_relu_pattern": false,
    "use_native_resize_op": false,
    "pooling_type": null,
    "fpn_name": null,
    "fpn_config": null,
    "survival_prob": null,
    "lr_decay_method": "cosine",
    "moving_average_decay": 0.9998,
    "ckpt_var_scope": null,
    "var_exclude_expr": ".*/class-predict/.*",
    "backbone_name": "efficientnet-b0",
    "backbone_config": null,
    "var_freeze_expr": null,
    "resnet_depth": 50,
    "dropout_rate": null,
    "batch_size": 1
}
INFO:tensorflow:ip-172-31-30-151: backbone params/flops = 3.595388M, 11.677380155B
INFO:tensorflow:building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: building cell %d
INFO:tensorflow:ip-172-31-30-151: fnode 0 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 4]}
INFO:tensorflow:ip-172-31-30-151: fnode 1 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 5]}
INFO:tensorflow:ip-172-31-30-151: fnode 2 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 6]}
INFO:tensorflow:ip-172-31-30-151: fnode 3 : {'width_ratio': 0.125, 'inputs_offsets': [0, 7]}
INFO:tensorflow:ip-172-31-30-151: fnode 4 : {'width_ratio': 0.0625, 'inputs_offsets': [1, 7, 8]}
INFO:tensorflow:ip-172-31-30-151: fnode 5 : {'width_ratio': 0.03125, 'inputs_offsets': [2, 6, 9]}
INFO:tensorflow:ip-172-31-30-151: fnode 6 : {'width_ratio': 0.015625, 'inputs_offsets': [3, 5, 10]}
INFO:tensorflow:ip-172-31-30-151: fnode 7 : {'width_ratio': 0.0078125, 'inputs_offsets': [4, 11]}
INFO:tensorflow:ip-172-31-30-151: backbone+fpn params/flops = 3.791669M, 12.540000870B
INFO:tensorflow:ip-172-31-30-151: backbone+fpn+box params/flops = 3.830342M, 13.685270646B
INFO:tensorflow:Restoring parameters from ./roadsign_transfer_learning/model.ckpt-3946
Code language: JavaScript (javascript)
Transfer Learning

Copyright(C) 2023 Habana Labs, Ltd. an Intel Company.

Licensed under the Apache License, Version 2.0

You may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Share this article:

You might also be interested in:

trend-of-sizes-nlp-models

Memory-Efficient Training on Habana® Gaudi® with DeepSpeed

Fine tuning GPT2 with Hugging Face and Habana Gaudi

synapseAI

The Habana team is happy to announce the release of SynapseAI® version 1.7.0.  

Back to tutorials

Sign up for the latest Habana developer news, events, training, and updates.
Sign up