Configure Canary Traffic Shifting

Canary traffic shifting enables you to safely test new upstreams with a small percentage of your traffic while keeping most traffic to the baseline upstream.

info

Canary traffic shifting differs from a canary release as the API version is unchanged. A canary traffic shifting rule applies to all routes in a service and cannot be applied to individual routes.

Prerequisites

1. Install API7 Enterprise and complete the Publish APIs by Service tutorial.

Shift Traffic by Percentage

In this example, you will direct 10% of the traffic to a canary upstream. The remaining 90% will continue to be forwarded to the baseline upstream.

After the new canary upstream is tested, all traffic can be routed to the canary upstream and it becomes the new baseline upstream. The older baseline upstream can then be removed.

Dashboard

1. Select Published Services from the side navigation bar and then select httpbin API.

2. Under the service, select Upstreams from the side navigation bar.

3. In the Canary Rules field, click Start Canary.

4. In the Set Up Canary Rules field, do the following:

   1. In the Conditions field, keep the switch open.
   2. In the Weight field, enter 10.

5. Click Next.

6. In the Choose or Create Canary Upstream field, choose to create a new upstream.

   1. Modify the name of the new upstream to newupstream.
   2. Adjust the host of the node to point to the new backend. For this tutorial, use mock.api7.ai as the host and 80 as the port.
   3. Keep the other properties the same as the baseline upstream.

7. Click Next.

8. Confirm the displayed information and click Start. The canary rules will start working immediately.

9. Validate the canary rules by sending 10 requests:

   for i in {1..10}; do "curl 127.0.0.1:9080/headers";  done

   9 requests will be sent to httpbin.org and will receive the following response:

   {
     "headers": {
       "Accept": "*/*",
       "Host": "httpbin.org",
       "User-Agent": "curl/7.74.0",
       "X-Amzn-Trace-Id": "Root=1-6650ab7e-32c90eba787abbeb4e3dbb0c",
       "X-Forwarded-Host": "127.0.0.1"
     }
   }

   A single request will be sent to mock.api7.ai:

   {
     "headers": {
       "accept": "*/*",
       "accept-encoding": "gzip, br",
       "cf-connecting-ip": "159.89.160.194",
       "cf-ipcountry": "IN",
       "cf-ray": "888e28733f9604aa",
       "cf-visitor": "{\"scheme\":\"https\"}",
       "connection": "Keep-Alive",
       "content-type": "application/json",
       "host": "mock.api7.ai",
       "user-agent": "curl/7.74.0",
       "x-application-owner": "API7.ai",
       "x-forwarded-for": "127.0.0.1",
       "x-forwarded-host": "127.0.0.1",
       "x-forwarded-port": "9080",
       "x-forwarded-proto": "https",
       "x-real-ip": "159.89.160.194",
       "X-Application-Owner": "API7.ai",
       "Content-Type": "application/json"
     }
   }

10. In the Canary Rules field, click Edit.

    • From the dialog box, adjust the weight to 50%.
    • Click Edit.

11. Make more requests to test the canary upstream.

12. In the Canary Rules field, click Finish. From the dialog box, do the following:

    • In the Baseline Upstream field, choose canary upstream: newupstream.
    • In the Delete Unselected Upstream field, close the switch.

13. Click Finish.

ADC

Update your ADC configuration file (adc.yaml) to include the canary upstream. The complete configuration is given below:

adc.yaml

services:
  - name: httpbin API
    upstream:
      name: Test Group
      scheme: https
      nodes:
        - host: httpbin.org
          port: 443
          weight: 100
    plugins:
      api7-traffic-split:
        rules:
          - canary_upstreams:
              - upstream_name: newupstream
                weight: 10
              - weight: 90
        upstreams:
          - name: newupstream
            nodes:
              - host: mock.api7.ai
                port: 443
                weight: 100
            scheme: https
    routes:
      - uris:
          - /headers
        name: getting-started-headers
        methods:
          - GET

Synchronize the configuration to API7 Enterprise:

adc sync -f adc.yaml

Validate the canary rules by sending 10 requests:

for i in {1..10}; do curl "127.0.0.1:9080/headers";  done

9 requests will be sent to httpbin.org and will receive the following response:

{
 "headers": {
   "Accept": "*/*",
   "Host": "httpbin.org",
   "User-Agent": "curl/7.74.0",
   "X-Amzn-Trace-Id": "Root=1-6650ab7e-32c90eba787abbeb4e3dbb0c",
   "X-Forwarded-Host": "127.0.0.1"
 }
}

A single request will be sent to mock.api7.ai:

{
 "headers": {
   "accept": "*/*",
   "accept-encoding": "gzip, br",
   "cf-connecting-ip": "159.89.160.194",
   "cf-ipcountry": "IN",
   "cf-ray": "888e28733f9604aa",
   "cf-visitor": "{\"scheme\":\"https\"}",
   "connection": "Keep-Alive",
   "content-type": "application/json",
   "host": "mock.api7.ai",
   "user-agent": "curl/7.74.0",
   "x-application-owner": "API7.ai",
   "x-forwarded-for": "127.0.0.1",
   "x-forwarded-host": "127.0.0.1",
   "x-forwarded-port": "9080",
   "x-forwarded-proto": "https",
   "x-real-ip": "159.89.160.194",
   "X-Application-Owner": "API7.ai",
   "Content-Type": "application/json"
 }
}

Now, update the weights to 50:50 to allow half of the traffic to be routed to the canary upstream:

adc.yaml

services:
  - name: httpbin API
    upstream:
      name: Test Group
      scheme: https
      nodes:
        - host: httpbin.org
          port: 443
          weight: 100
    plugins:
      api7-traffic-split:
        rules:
          - canary_upstreams:
              - upstream_name: newupstream
                weight: 50
              - weight: 50
        upstreams:
          - name: newupstream
            nodes:
              - host: mock.api7.ai
                port: 443
                weight: 100
            scheme: https
    routes:
      - uris:
          - /headers
        name: getting-started-headers
        methods:
          - GET

Send more requests to test the canary upstream. Finally, update the old upstream with the current upstream to finish the canary traffic shifting:

adc.yaml

services:
  - name: httpbin API
    upstream:
      name: Test Group
      scheme: https
      nodes:
        - host: mock.api7.ai
          port: 443
          weight: 100
    routes:
      - uris:
          - /headers
        name: getting-started-headers
        methods:
          - GET

You can also keep the old upstream as a canary upstream with 0 weight to roll back to if there are issues with the new upstream.

Shift Traffic by Request Header

In this example, you will direct requests with the header version = test to the canary upstream, while the remaining traffic will continue to the baseline upstream.

Dashboard

1. Select Published Services from the side navigation bar and then select httpbin API.

2. In the Canary Rules field, click Start Canary.

3. In the Set Up Canary Rules field, do the following:

   • In the Conditions field, close the switch.
   • Fill in the header requirement with header version == test.
   • In the Weight field, enter 100.
   • Click Next.

4. In the Choose or Create Canary Upstream field, choose to "create a new upstream".

   • Modify the name of the new upstream to newupstream.
   • Adjust the host of the node to point to the new backend. For this tutorial, use mock.api7.ai as the host and 80 as the port.
   • Keep the other properties the same as the baseline upstream.
   • Click Next.

5. Confirm the displayed information and click Start. The canary rules will start working immediately.

6. Validate the canary rules by sending requests:

   Send a request with the version:test header:

   curl 127.0.0.1:9080/headers -H "version:test"

   {
     "headers": {
       "accept": "*/*",
       "accept-encoding": "gzip, br",
       "cf-connecting-ip": "159.89.160.194",
       "cf-ipcountry": "IN",
       "cf-ray": "888e28733f9604aa",
       "cf-visitor": "{\"scheme\":\"https\"}",
       "connection": "Keep-Alive",
       "content-type": "application/json",
       "host": "mock.api7.ai",
       "user-agent": "curl/7.74.0",
       "x-application-owner": "API7.ai",
       "x-forwarded-for": "127.0.0.1",
       "x-forwarded-host": "127.0.0.1",
       "x-forwarded-port": "9080",
       "x-forwarded-proto": "https",
       "x-real-ip": "159.89.160.194",
       "X-Application-Owner": "API7.ai",
       "Content-Type": "application/json"
     }
   }

   Send a request with the wrong header:

   curl 127.0.0.1:9080/headers -H "version:new"

   {
     "headers": {
       "Accept": "*/*",
       "Host": "httpbin.org",
       "User-Agent": "curl/7.74.0",
       "X-Amzn-Trace-Id": "Root=1-6650ab7e-32c90eba787abbeb4e3dbb0c",
       "X-Forwarded-Host": "127.0.0.1"
     }
   }

   Send a request with no header:

   curl 127.0.0.1:9080/headers

   {
     "headers": {
       "Accept": "*/*",
       "Host": "httpbin.org",
       "User-Agent": "curl/7.74.0",
       "X-Amzn-Trace-Id": "Root=1-6650ab7e-32c90eba787abbeb4e3dbb0c",
       "X-Forwarded-Host": "127.0.0.1"
     }
   }

7. In the Canary Rules field, click Finish. From the dialog box, do the following:

   • In the Baseline Upstream field, choose canary upstream: newupstream.
   • In the Delete Unselected Upstream field, close the switch.

8. Click Finish.

ADC

Update your ADC configuration file (adc.yaml) to include the canary upstream. The complete configuration is given below:

adc.yaml

services:
  - name: httpbin API
    upstream:
      name: default
      scheme: https
      nodes:
        - host: httpbin.org
          port: 443
          weight: 100
    plugins:
      api7-traffic-split:
        rules:
          - canary_upstreams:
              - upstream_name: newupstream
                weight: 100
            match:
              - exprs:
                  - - http_version
                    - ==
                    - test
        upstreams:
          - name: newupstream
            nodes:
              - host: mock.api7.ai
                port: 443
                weight: 100
            scheme: https
    routes:
      - uris:
          - /headers
        name: getting-started-headers
        methods:
          - GET

Synchronize the configuration to API7 Enterprise:

adc sync -f adc.yaml

Validate the canary rules by sending a request with the version:test header:

curl 127.0.0.1:9080/headers  

You will get back a response from the canary upstream:

{
 "headers": {
   "accept": "*/*",
   "accept-encoding": "gzip, br",
   "cf-connecting-ip": "159.89.160.194",
   "cf-ipcountry": "IN",
   "cf-ray": "888e28733f9604aa",
   "cf-visitor": "{\"scheme\":\"https\"}",
   "connection": "Keep-Alive",
   "content-type": "application/json",
   "host": "mock.api7.ai",
   "user-agent": "curl/7.74.0",
   "x-application-owner": "API7.ai",
   "x-forwarded-for": "127.0.0.1",
   "x-forwarded-host": "127.0.0.1",
   "x-forwarded-port": "9080",
   "x-forwarded-proto": "https",
   "x-real-ip": "159.89.160.194",
   "X-Application-Owner": "API7.ai",
   "Content-Type": "application/json"
 }
}

Send a request with the wrong header:

curl 127.0.0.1:9080/headers -H "version:new"

{
 "headers": {
   "Accept": "*/*",
   "Host": "httpbin.org",
   "User-Agent": "curl/7.74.0",
   "X-Amzn-Trace-Id": "Root=1-6650ab7e-32c90eba787abbeb4e3dbb0c",
   "X-Forwarded-Host": "127.0.0.1"
 }
}

Send a request with no header:

curl 127.0.0.1:9080/headers

{
 "headers": {
   "Accept": "*/*",
   "Host": "httpbin.org",
   "User-Agent": "curl/7.74.0",
   "X-Amzn-Trace-Id": "Root=1-6650ab7e-32c90eba787abbeb4e3dbb0c",
   "X-Forwarded-Host": "127.0.0.1"
 }
}

Update the old upstream with the current upstream to finish the canary traffic shifting:

adc.yaml

services:
  - name: httpbin API
    upstream:
      name: Test Group
      scheme: https
      nodes:
        - host: mock.api7.ai
          port: 443
          weight: 100
    routes:
      - uris:
          - /headers
        name: getting-started-headers
        methods:
          - GET

You can also keep the old upstream as a canary upstream with 0 weight to roll back to if there are issues with the new upstream.