Improve verify auto pause step and Update README.md for auto-pause benchmark

Author: govargo

hack/benchmark/cpu_usage/auto_pause/README.md

@@ -5,17 +5,22 @@ These scripts are for the benchmark of cpu usage, minikube vs kind vs k3d.
  * `update_summary.sh`: create one summary csv file of each drivers and products
  * `chart.go`: create bar chart graph as a png file
 
-In `benchmark_local_k8s.sh`, we compare minikube drivers(hyperkit, virtualbox, docker, docker with auto-pause) and kind, k3d, Docker for Mac Kubernetes in case of macOS.   
-In `benchmark_local_k8s.sh`, we compare minikube drivers(kvm2, virtualbox, docker, docker with auto-pause) and kind, k3d in case of Linux.   
+In `benchmark_local_k8s.sh`, we compare minikube drivers(hyperkit, virtualbox, docker) and kind, k3d, Docker for Mac Kubernetes in case of macOS.   
+In `benchmark_local_k8s.sh`, we compare minikube drivers(kvm2, virtualbox, docker) and kind, k3d in case of Linux.   
 `benchmark_local_k8s.sh` take these steps to measure `auto-pause` vs. `non auto-pause`.   
 
  1. start each local kubernetes tool
- 2. deploy sample application(nginx deployment)
+ 2. deploy sample application(nginx deployment) to each tool
  3. wait 1 minute without anything
  4. measure No.3 idle CPU usage with [cstat](https://github.com/tstromberg/cstat)
- 5. enable auto-pause addons(only if tool is minikube)
- 6. wait 3 minute without anything
- 7. measure No.6 idle CPU usage with [cstat](https://github.com/tstromberg/cstat)
+ 5. if tool is minikube, enable auto-pause addon which pause control plane
+ 6. if tool is minikube, wait 1 minute so that control plane will become Paused status(It takes 1 minute to become Pause status from Stopped status)  
+ 7. if tool is minikube, verify if minikube control plane is paused
+ 8. if tool is minikube, wait 3 minute without anything
+ 9. if tool is minikube, measure No.8 idle CPU usage with [cstat](https://github.com/tstromberg/cstat)
+
+No.1-4: Initial start CPU usage with sample(nginx) deployment   
+No.5-9: Auto Paused CPU usage with sample(nginx) deployment    
 
 # How to use these scripts
 
@@ -31,4 +36,4 @@ If you update the benchmark results to [our website](https://minikube.sigs.k8s.i
 git status
 git add <Changed png file>
 git commit
-```
+```

hack/benchmark/cpu_usage/auto_pause/benchmark_local_k8s.sh

@@ -271,21 +271,25 @@ main() {
       echo "-> enable auto-pause to control plane"
       out/minikube addons enable auto-pause
 
-      # 6. wait 1 minute so that control plane will become Paused status(It takes 1 minute)
-      sleep 65
-      # 7. verify if minikube control plane is paused
-      PAUSE=$(out/minikube status)
-      echo $PAUSE | grep "apiserver: Paused"
-      # 8. wait 3 minute without anything
-      # 9. measure No.6 idle CPU usage
-      if [[ "$?" == 0 ]]; then
-        echo "kube-apiserver is paused"
-        measure "minikube.${driver}.autopause" $i "3m" || fail "minikube.${driver}.autopause" $i
-      else
-        echo "failed to auto pause"
-        fail "minikube.${driver}.autopause" $i
-      fi
-
+      # 6. wait 1 minute so that control plane will become Paused status
+      pause=0
+      while [ "${pause}" = 0 ]
+      do
+	# It takes 1 minute to become Pause status from Stopped status. 70s is a number with a margin
+	sleep 70
+        # 7. verify if minikube control plane is paused
+        PAUSE=$(out/minikube status)
+        echo $PAUSE | grep "apiserver: Paused"
+        if [[ "$?" == 0 ]]; then
+          echo "kube-apiserver is paused"
+	  pause=1
+	else
+	  echo "...status is not Paused. wait for becoming Pause..."
+        fi
+      done
+
+      # 8. wait 3 minute without anything and 9. measure No.8 idle CPU usage
+      measure "minikube.${driver}.autopause" $i "3m" || fail "minikube.${driver}.autopause" $i
       cleanup
 
       # We won't be needing docker for the remaining tests this iteration