Waiting for applications to start in docker

The following script waits for applications to start in docker. These can be web applications and non web services.

This necessary for programmatically proceeding before especially in the scope of build / test automation, where-in you want to run your tests once the services are available.

The following bash script does two things

Wait for web applications to be available in docker by checking the availability of the application on the localhost URL using wget spider which essentially hits the URL without downloading it to check the availability

wget --spider localhost:$WEB_APPLICATION_PORT

Wait for non-web applications to be available

We do this by using docker ps and that a grep with the port number returns a line

docker-compose ps | grep $APPLICATION_PORT | wc -l == 1

Full script

Below is the script reference implementing the above, extrapolating it to a waiting for a number of application (web and non-web) to start.

#!/usr/bin/env bash
# Run this script to prepare the Docker cluster for running the cdfpay Imports job chains

set -o pipefail  # trace ERR through pipes
set -o errtrace  # trace ERR through 'time command' and other functions
set -o nounset   # set -u : exit the script if you try to use an uninitialised variable
set -o errexit   # set -e : exit the script if any statement returns a non-true return value

# Cleanup
docker-compose down -v

# Run this script to run the docker cluster
docker-compose up -d --build

declare -A application_ports
declare -A application_web_ports

application_web_ports["impala_web_port"]=$(docker-compose port data-cluster 25000 | sed 's/.*://')
application_web_ports["hive_web_port"]=$(docker-compose port data-cluster 10002 | sed 's/.*://')

application_ports["impala_client_port"]=$(docker-compose port data-cluster 21050 | sed 's/.*://')
application_ports["impala_shell_port"]=$(docker-compose port data-cluster 21000 | sed 's/.*://')
application_ports["hive_port"]=$(docker-compose port data-cluster 10000 | sed 's/.*://')
application_ports["mysql_port"]=$(docker-compose port mysql 3306 | sed 's/.*://')

all_available=false
timeout=900 # seconds

while ! $all_available && [ $timeout -gt 0 ]; do  
  all_available=true
  
  for app in "${!application_web_ports[@]}"; do
      if v 2>/dev/null; then
            echo "$app is open (port ${application_web_ports[$app]})"
      else
          echo "Waiting for $app (port ${application_web_ports[$app]}) ..."
          all_available=false
      fi
  done
  
  for app in "${!application_ports[@]}"; do
      if [[ $(docker-compose ps | grep ${application_ports[$app]} | wc -l) == 1 ]]; then
            echo "$app is open (port ${application_ports[$app]})"
      else
          echo "Waiting for $app (port ${application_ports[$app]}) ..."
          all_available=false
      fi
  done
  
  sleep 30
  timeout=$((timeout - 30))
done 

if [ $timeout -eq 0 ]; then
  echo "Timed out waiting for all applications to become available."
  exit 1
fi

docker-compose ps
echo "DOCKER is ready."

There is a part in this is script that is specific to finding the port mapped to a service

docker-compose port mysql 3306 | sed 's/.*://'

This can be specified plainly too, if you don’t alter (forward) the port in your docker-compose

- 3306:3306 # Unchanged then use application_ports["mysql_port"]=3306

But that is if you have assigned a port to the application in docker, the docker-compose port helps find the actual assigned port

- 7700:3306 #use application_ports["mysql_port"]=7700
- ${MYSQL_PORT:-0}:3306 #MYSQL_PORT variable, or Randomly allocated if MYSQL_PORT is not defined. Use application_ports["mysql_port"]=$(docker-compose port mysql 3306 | sed 's/.*://')