Rest-Proxy Migration Poc
Description
The Rest proxy is linked to the Confluent cluster via the kubernetes Secret key (the secret has the cluster bootstrap and as well as secret name and secret).
Prerequisites
- You must have a rest proxy deployed and connected to the source cluster (Example :- cluster id:lkc-gqjvpv name:test-cluster).
- You must have a destion clsuter ready where you want the rest-proxy to be migrate(Example:- cluster id:lkc-8nqr0).
Step-1: Create a new **Secret** with the Same **SA** for the destination cluster
Step-2: with the new secret using the kafka-rest.properties file create a new kubernetes secret (this secret will have the destination cluster information).
For Example : kafka-rest.properties
ssl.endpoint.identification.algorithm=https
client.sasl.mechanism=PLAIN
request.timeout.ms=20000
bootstrap.servers=xx.yyy.zzz.gcp.confluent.cloud:9092
retry.backoff.ms=500
client.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username={API_KEY_HERE} password={API_SECRET_HERE}
client.security.protocol=SASL_SSL
Create kubernetes Secret
kubectl create secret generic kafka-rest-config --from-file=kafka-rest.properties
Step-3: Now that the newly created secret has all the information we needed to connect with the destination cluster.
# We can migrate the **Rest-Proxy** in 2 ways.
1. We can delete the old proxy and create a new one with the destination cluster information and we can use kubectl rollout restart to restart the pod.Rolling out restart is the ideal approach to restarting your pods because your application will not be affected or go down.
For rolling out a restart, use the following command:
kubectl rollout restart deployment -n
it will restart the pod and after few min it will be connected to the destination cluster.
Testing Connectivity Process
Create a port forwarding to the the service
**Verify that we can see the previously created topic**
curl http://localhost:8082/topics
You will see all the topic you have created using the Secret.
["tes-proxy-1","tes-proxy-2"]
If you are unable to confirm that the proxy is operational, the following items should be checked:
1. Verify the API Key and Secret are correct or generate a new pair
2. Check the pod logs for errors
2. We can create a new secret with destination cluster information like :Secretnew and we can just Edit the pod deployment and replace the old secret with the new one and save it, that way the pod will restart autometically and will be connected to the destination cluster.
For Example:
kubectl get deployment
kubectl edit deployment
## Rest-Proxy Version Migration from 5.x.x to 7.x.x
to change the rest-proxy version from 5.x.x to 7.x.x just edit the deployment file and change its image to 7.2.2 (this is the latest version)
also if the volumes.secret.defaultMode is not 444 make it defaultMode: 444. becasue rest-proxy version 5.x.x can be deployed using 440 but version 7.x.x required the permission .
For Example: change these lines
image: confluentinc/cp-kafka-rest:5.1.3
volumes:
- name: kafka-rest-config7
secret:
defaultMode: 444
secretName: kafka-rest-config7
items:
- key: kafka-rest.properties
path: kafka-rest.deployment.properties
Rahib
Rahib