Rollbacks
Rolling Back to Rancher v2.2-v2.4
For Rancher installed on Kubernetes, follow the procedure detailed here: Restoring Backups for Kubernetes installs. Restoring a snapshot of the Rancher Server cluster will revert Rancher to the version and state at the time of the snapshot.
For information on how to roll back Rancher installed with Docker, refer to this page.
Managed clusters are authoritative for their state. This means restoring the rancher server will not revert workload deployments or changes made on managed clusters after the snapshot was taken.
Rolling Back to v2.0.0-v2.1.5
If you are rolling back to versions in either of these scenarios, you must follow some extra instructions in order to get your clusters working.
- Rolling back from v2.1.6+ to any version between v2.1.0 - v2.1.5 or v2.0.0 - v2.0.10.
- Rolling back from v2.0.11+ to any version between v2.0.0 - v2.0.10.
Because of the changes necessary to address CVE-2018-20321, special steps are necessary if the user wants to roll back to a previous version of Rancher where this vulnerability exists. The steps are as follows:
Record the
serviceAccountToken
for each cluster. To do this, save the following script on a machine withkubectl
access to the Rancher management plane and execute it. You will need to run these commands on the machine where the rancher container is running. Ensure JQ is installed before running the command. The commands will vary depending on how you installed Rancher.Rancher Installed with Docker
docker exec <NAME OF RANCHER CONTAINER> kubectl get clusters -o json | jq '[.items[] | select(any(.status.conditions[]; .type == "ServiceAccountMigrated")) | {name: .metadata.name, token: .status.serviceAccountToken}]' > tokens.json
Rancher Installed on a Kubernetes Cluster
kubectl get clusters -o json | jq '[.items[] | select(any(.status.conditions[]; .type == "ServiceAccountMigrated")) | {name: .metadata.name, token: .status.serviceAccountToken}]' > tokens.json
After executing the command a
tokens.json
file will be created. Important! Back up this file in a safe place. You will need it to restore functionality to your clusters after rolling back Rancher. If you lose this file, you may lose access to your clusters.**Rollback Rancher following the normal instructions.
Once Rancher comes back up, every cluster managed by Rancher (except for Imported clusters) will be in an
Unavailable
state.Apply the backed up tokens based on how you installed Rancher.
Rancher Installed with Docker
Save the following script as
apply_tokens.sh
to the machine where the Rancher docker container is running. Also copy thetokens.json
file created previously to the same directory as the script.set -e
tokens=$(jq .[] -c tokens.json)
for token in $tokens; do
name=$(echo $token | jq -r .name)
value=$(echo $token | jq -r .token)
docker exec $1 kubectl patch --type=merge clusters $name -p "{\"status\": {\"serviceAccountToken\": \"$value\"}}"
donethe script to allow execution (
chmod +x apply_tokens.sh
) and execute the script as follows:./apply_tokens.sh <DOCKER CONTAINER NAME>
After a few moments the clusters will go from Unavailable back to Available.
Rancher Installed on a Kubernetes Cluster
Save the following script as
apply_tokens.sh
to a machine with kubectl access to the Rancher management plane. Also copy thetokens.json
file created previously to the same directory as the script.set -e
tokens=$(jq .[] -c tokens.json)
for token in $tokens; do
name=$(echo $token | jq -r .name)
value=$(echo $token | jq -r .token)
kubectl patch --type=merge clusters $name -p "{\"status\": {\"serviceAccountToken\": \"$value\"}}"
doneSet the script to allow execution (
chmod +x apply_tokens.sh
) and execute the script as follows:./apply_tokens.sh
After a few moments the clusters will go from
Unavailable
back toAvailable
.Continue using Rancher as normal.