Updating a Service
By the end of this exercise, you should be able to:
- Update a swarm service's underlying image, controlling update parallelism, speed, and rollback contingencies
Creating Rolling Updates
First, let's change one of our services a bit: open
orchestration-workshop/dockercoins/worker/worker.pyin your favorite text editor, and find the following section:def work_once(): log.debug("Doing one unit of work") time.sleep(0.1)Change the
0.1to a0.01. Save the file, exit the text editor.Rebuild the worker image with a tag of
<Docker ID>/dockercoins_worker:1.1, and push it to Docker Hub.Start the update, and wait for it to converge:
[centos@node-0 ~]$ docker service update dc_worker \ --image <Docker ID>/dockercoins_worker:1.1Tasks are updated to our new 1.1 image one at a time.
Parallelizing Updates
We can also set our updates to run in batches by configuring some options associated with each service. Change the update parallelism to 2 and the delay to 5 seconds on the
workerservice by editing its definition in thedocker-compose.yml:worker: image: <Docker ID>/dockercoins_worker:1.0 networks: - dockercoins deploy: replicas: 10 update_config: parallelism: 2 delay: 5sRoll back the worker service to 1.0:
[centos@node-0 ~]$ docker stack deploy -c=docker-compose.yml dcOn
node-1, watch your updates:[centos@node-1 ~]$ watch -n1 "docker service ps dc_worker \ | grep -v Shutdown.*Shutdown"You should see two tasks get shutdown and restarted with the
1.0image every five seconds.
Auto-Rollback Failed Updates
In the event of an application or container failure on deployment, we'd like to automatically roll the update back to the previous version.
Update the
workerservice with some parameters to define rollback:[centos@node-0 ~]$ docker service update \ --update-failure-action=rollback \ --update-max-failure-ratio=0.2 \ --update-monitor=20s \ dc_workerThese parameters will trigger a rollback if more than 20% of services tasks fail in the first 20 seconds after an update.
Make a broken version of the
workerservice to trigger a rollback with; try removing all theimportcommands at the top ofworker.py, for example. Then rebuild the worker image with a tag<Docker ID>/dockercoins_worker:bugged, push it to Docker Hub, and attempt to update your service:[centos@node-0 ~]$ docker image build -t <Docker ID>/dockercoins_worker:bugged . [centos@node-0 ~]$ docker image push <Docker ID>/dockercoins_worker:bugged [centos@node-0 ~]$ docker service update \ dc_worker --image <Docker ID>/dockercoins_worker:buggedThe connection to
node-1runningwatchshould show the:buggedtag getting deployed, failing, and rolling back to:1.0automatically over the next minute or two.
Shutting Down a Stack
To shut down a running stack:
[centos@node-0 ~]$ docker stack rm <stack name>Where the stack name can be found in the output of
docker stack ls.
Conclusion
In this exercise, we explored deploying and redeploying an application as stacks and services. Note that relaunching a running stack updates all the objects it manages in the most non-disruptive way possible; there is usually no need to remove a stack before updating it. In production, rollback contingencies should always be used to cautiously upgrade images, cutting off potential damage before an entire service is taken down.