Extend Chaos Daemon Interface

note

It’s recommended to read Chaos Mesh architecture before you go forward.

This document covers:

• Implement the gRPC interface
• Next steps

In api/v1alpha1/helloworldchaos_type.go, you have defined HelloWorldSpec, which includes ContainerSelector:

In Chaos Mesh, Selector is used to define the scope of a chaos experiment, the target namespace, annotation, label, etc. Selector can also be some more specific values (for example, AWSSelector in AWSChaos). Usually, each chaos experiment requires only one Selector, with exceptions such as NetworkChaos because it sometimes needs two Selectors as two objects for network partition.

To allow Chaos Daemon to accept the requests from Chaos Controller Manager, you need to implement a new gRPC interface.

1. Add the RPC in pkg/chaosdaemon/pb/chaosdaemon.proto:

   service chaosDaemon {
      ...
      rpc ExecHelloWorldChaos(ExecHelloWorldRequest) returns (google.protobuf.Empty) {}
   }
   message ExecHelloWorldRequest {
      string container_id = 1;
   }

   You need to update the Golang code generated by this proto file:

   make proto

2. Implement gRPC services in Chaos Daemon.

   package chaosdaemon
   import (
      "context"
      "fmt"
      "github.com/golang/protobuf/ptypes/empty"
      "github.com/chaos-mesh/chaos-mesh/pkg/bpm"
      "github.com/chaos-mesh/chaos-mesh/pkg/chaosdaemon/pb"
   func (s *DaemonServer) ExecHelloWorldChaos(ctx context.Context, req *pb.ExecHelloWorldRequest) (*empty.Empty, error) {
      log := s.getLoggerFromContext(ctx)
      log.Info("ExecHelloWorldChaos", "request", req)
      pid, err := s.crClient.GetPidFromContainerID(ctx, req.ContainerId)
      if err != nil {
         return nil, err
      }
      cmd := bpm.DefaultProcessBuilder("sh", "-c", fmt.Sprintf("ps aux")).
         SetNS(pid, bpm.MountNS).
         SetContext(ctx).
         Build(ctx)
      out, err := cmd.Output()
      if err != nil {
         return nil, err
      }
      if len(out) != 0 {
         log.Info("cmd output", "output", string(out))
      }
      return &empty.Empty{}, nil
   }

   After chaos-daemon receives the ExecHelloWorldChaos request, you can see a list of processes in the current container.

To verify the experiment, perform the following steps.

1. Build the Docker image and push it to your local Registry. If the Kubernetes cluster is deployed using kind, you need to load the image to kind:

   make image
   make docker-push
   kind load docker-image localhost:5000/pingcap/chaos-mesh:latest
   kind load docker-image localhost:5000/pingcap/chaos-daemon:latest
   kind load docker-image localhost:5000/pingcap/chaos-dashboard:latest

2. Update Chaos Mesh:

3. kubectl apply -f https://raw.githubusercontent.com/chaos-mesh/apps/master/ping/busybox-statefulset.yaml

4. Create a new YAML file with the following content:

   apiVersion: chaos-mesh.org/v1alpha1
   kind: HelloWorldChaos
   metadata:
     name: busybox-helloworld-chaos
   spec:
     selector:
       namespaces:
         - busybox
     mode: all
     duration: 1h

5. Apply the chaos experiment:

6. Verify the results. You can check several logs:

   • Check the logs of Chaos Controller Manager:

   kubectl logs chaos-controller-manager-{pod-post-fix} -n chaos-mesh

   Example output:

   2021-06-25T06:02:12.754Z        INFO    records apply chaos     {"id": "busybox/busybox-1/busybox"}
   2021-06-25T06:02:12.754Z        INFO    helloworldchaos Apply helloworld chaos

   • Check the logs of Chaos Daemon:

   Example output:

   You can see ps aux in two separate lines, which are corresponding to two different Pods.

   note

   If your cluster has multiple nodes, you will find more than one Chaos Daemon Pod. Try to check logs of every Chaos Daemon Pods and find which Pod is being called.

If you encounter any problems in this process, create an in the Chaos Mesh repository.

Now you are ready to become a Chaos Mesh developer! You are welcome to visit the Chaos Mesh repository to find a and get started!