Simulate Disk Faults

    This section describes how to create disk fault experiments using the command-line mode.

    Before creating an experiment, you can run the following command to check the types of disk faults that are supported by Chaosd:

    The result is as follows:

    3. Usage:
    4.   chaosd attack disk [command]
    6. Available Commands:
    7.   add-payload add disk payload
    8.   fill        fill disk
    10. Flags:
    11.   -h, --help   help for disk
    13. Global Flags:
    15. Use "chaosd attack disk [command] --help" for more information about a command.

    Currently, Chaosd supports creating disk read load experiments, disk write load experiments, and disk fill experiments.

    Commands for simulating disk read load

    1. chaosd attack disk add-payload read -h

    The result is as follows:

    1. read payload
    3. Usage:
    4.   chaosd attack disk add-payload read [flags]
    6. Flags:
    7.   -h, --help                help for read
    8.   -p, --path string         'path' specifies the location to read data.If path not provided, payload will read from disk mount on "/"
    9.   -n, --process-num uint8   'process-num' specifies the number of process work on reading , default 1, only 1-255 is valid value (default 1)
    10.   -s, --size string         'size' specifies how many units of data will read from the file path.'unit' specifies the unit of data, support c=1, w=2, b=512, kB=1000, K=1024, MB=1000*1000,M=1024*1024, , GB=1000*1000*1000, G=1024*1024*1024 BYTESexample : 1M | 512kB
    12. Global Flags:
    13.       --log-level string   the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error'

    Configuration description for simulating disk read load

    Example for simulating disk read load

    1. chaosd attack disk add-payload read -s 1000G -n 7 -p /dev/zero

    The result is as follows:

    1. andrew@LAPTOP-NUS30NQD:~/chaosd/bin$ ./chaosd attack disk add-payload read -s 1000G -n 7 -p /dev/zero
    2. [2021/05/20 13:54:31.323 +08:00] [INFO] [disk.go:128] ["5242880+0 records in\n5242880+0 records out\n5242880 bytes (5.2 MB, 5.0 MiB) copied, 4.13252 s, 1.3 MB/s\n"]
    3. [2021/05/20 13:54:46.977 +08:00] [INFO] [disk.go:147] ["146285+0 records in\n146285+0 records out\n153390940160 bytes (153 GB, 143 GiB) copied, 15.6513 s, 9.8 GB/s\n"]
    4. [2021/05/20 13:54:47.002 +08:00] [INFO] [disk.go:147] ["146285+0 records in\n146285+0 records out\n153390940160 bytes (153 GB, 143 GiB) copied, 15.6762 s, 9.8 GB/s\n"]
    5. [2021/05/20 13:54:47.004 +08:00] [INFO] [disk.go:147] ["146285+0 records in\n146285+0 records out\n153390940160 bytes (153 GB, 143 GiB) copied, 15.6777 s, 9.8 GB/s\n"]
    6. [2021/05/20 13:54:47.015 +08:00] [INFO] [disk.go:147] ["146285+0 records in\n146285+0 records out\n153390940160 bytes (153 GB, 143 GiB) copied, 15.6899 s, 9.8 GB/s\n"]
    7. [2021/05/20 13:54:47.018 +08:00] [INFO] [disk.go:147] ["146285+0 records in\n146285+0 records out\n153390940160 bytes (153 GB, 143 GiB) copied, 15.6914 s, 9.8 GB/s\n"]
    8. [2021/05/20 13:54:47.051 +08:00] [INFO] [disk.go:147] ["146285+0 records in\n146285+0 records out\n153390940160 bytes (153 GB, 143 GiB) copied, 15.7254 s, 9.8 GB/s\n"]
    9. [2021/05/20 13:54:47.074 +08:00] [INFO] [disk.go:147] ["146285+0 records in\n146285+0 records out\n153390940160 bytes (153 GB, 143 GiB) copied, 15.7487 s, 9.7 GB/s\n"]
    10. Read file /dev/zero successfully, uid: 4bc9b74a-5fe2-4038-b4f2-09ae95b57694

    Simulate disk write load using the command-line mode

    Command for simulating disk write load

    The result is as follows:

    1. write payload
    3. Usage:
    4.   chaosd attack disk add-payload write [flags]
    6. Flags:
    8.   -p, --path string         'path' specifies the location to fill data in.If path not provided, payload will write into a temp file, temp file will be deleted after writing
    9.   -n, --process-num uint8   'process-num' specifies the number of process work on writing , default 1, only 1-255 is valid value (default 1)
    11. Global Flags:
    12.       --log-level string   the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error'

    Configuration description for simulating disk write load

    Configuration itemAbbreviationDescriptionValue
    pathpSpecifies the file path to write the data. If this parameter is not specified, or the parameter value is set to an empty string, a temporary file will be created in the program execution directory. Depending on the permissions to write the files, you might be required to run this program using certain permissions.type: string; default: “”
    process-numnSpecifies the number of concurrently running dd programs to be used.type: uint8; default: 1; range: 1 to 255
    sizesSpecifies the volume of data to be written. It is the total size of data that <0>dd</0> writes.type: string; default: “”; required; legal form: the combination of an integer and a unit. For example, 1M, 512kB. Supported units are c=1, w=2, b=512, kB=1000, K=1024, MB=10001000, M=10241024, GB=100010001000, G=102410241024*1024 BYTE and so on.

    Example for simulating disk write load

    1. chaosd attack disk add-payload write -s 2G -n 8

    The result is as follows:

    1. [2021/05/20 14:28:14.452 +08:00] [INFO] [disk.go:128] ["0+0 records in\n0+0 records out\n0 bytes copied, 4.3e-05 s, 0.0 kB/s\n"]
    2. [2021/05/20 14:28:16.793 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.32841 s, 115 MB/s\n"]
    3. [2021/05/20 14:28:16.793 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.3344 s, 115 MB/s\n"]
    4. [2021/05/20 14:28:16.793 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.33312 s, 115 MB/s\n"]
    5. [2021/05/20 14:28:16.793 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.33466 s, 115 MB/s\n"]
    6. [2021/05/20 14:28:16.793 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.33189 s, 115 MB/s\n"]
    7. [2021/05/20 14:28:16.793 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.33752 s, 115 MB/s\n"]
    8. [2021/05/20 14:28:16.793 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.33295 s, 115 MB/s\n"]
    9. [2021/05/20 14:28:16.794 +08:00] [INFO] [disk.go:147] ["256+0 records in\n256+0 records out\n268435456 bytes (268 MB, 256 MiB) copied, 2.3359 s, 115 MB/s\n"]
    10. Write file /home/andrew/chaosd/bin/example255569279 successfully, uid: e66afd86-6f3e-43a0-b161-09447ed84856

    Command for simulating disk fill

    1. chaosd attack disk fill -h
    1. fill disk
    3. Usage:
    4.   chaosd attack disk fill [flags]
    6. Flags:
    7.   -d, --destroy          destroy file after filled in or allocated
    8.   -f, --fallocate        fill disk by fallocate instead of dd (default true)
    9.   -h, --help             help for fill
    10.   -p, --path string      'path' specifies the location to fill data in.If path not provided, a temp file will be generated and deleted immediately after data filled in or allocated
    11.   -c, --percent string   'percent' how many percent data of disk will fill in the file path
    12.   -s, --size string      'size' specifies how many units of data will fill in the file path.'unit' specifies the unit of data, support c=1, w=2, b=512, kB=1000, K=1024, MB=1000*1000,M=1024*1024, , GB=1000*1000*1000, G=1024*1024*1024 BYTESexample : 1M | 512kB
    14. Global Flags:
    15.       --log-level string   the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error'

    Configuration description for simulating disk fill

    Example for simulating disk fill

    The result is as follows:

    1. [2021/05/20 14:30:02.943 +08:00] [INFO] [disk.go:215]
    2. Fill file /home/andrew/chaosd/bin/example623832242 successfully, uid: 097b4214-8d8e-46ad-8768-c3e0d8cbb326

    Create experiments using the service mode

    This section describes how to create disk fault experiments using the service mode.

    Simulate disk read load using the service mode

    Simulating disk read load is a one-time operation, so the experiment does not need to be recovered.

    Parameters for simulating disk read load

    ParameterDescriptionValue
    actionActions of the experimentSet to “read-payload”
    pathSpecifies the file path to read the data. If this parameter is not specified, or the parameter value is set to an empty string, Chaosd reads from the virtual disk files mounted in the “/“ directory. Depending on the permissions to read the files, you might be required to run this program using certain permissions.type: string; default: “””
    payload-process-numSpecifies the number of concurrently running dd programs to be used.type: uint8; default: 1; range: 1 to 255
    sizeSpecifies the volume of data to be read. It is the total size of data that <0>dd</0> reads.type: string; default: “”; required; legal form: the combination of an integer and a unit. For example, 1M, 512kB. Supported units are c=1, w=2, b=512, kB=1000, K=1024, MB=10001000, M=10241024, GB=100010001000, G=102410241024*1024 BYTE and so on.

    Example for simulating disk read load using the service mode

    1. curl -X POST 172.16.112.130:31767/api/attack/disk -H "Content-Type:application/json" -d '{"action":"read-payload","path":"/dev/zero", "payload-process-num":7,"size":"1000G"}'

    The result is as follows:

    1. {"status":200,"message":"attack successfully","uid":"a551206c-960d-4ac5-9056-518e512d4d0d"}

    Parameters for simulating disk write load

    Example for simulating disk write load using the service mode

    1. curl -X POST 172.16.112.130:31767/api/attack/disk -H "Content-Type:application/json" -d '{"action":"write-payload","path":"/tmp/test", "payload-process-num":7,"size":"1000G"}'
    1. {"status":200,"message":"attack successfully","uid":"a551206c-960d-4ac5-9056-518e512d4d0d"}

    Simulate disk fill using the service mode

    Parameters for simulating disk fill

    ParameterDescriptionValue
    actionActions of the experimentSet to “fill”
    destroyIf this parameter is set to true, the fill file is immediately deleted after being filled.type: bool; default: false
    fill-by-fallocateIf this parameter is set to true, Chaosd uses Linux to call fallocate to apply for disk space quickly, and you must set size to a value greater than 0. If this parameter is set to false, Chaosd uses Linux to call dd to fill disks at a relatively slow pace.type: bool; default: true
    pathSpecifies the file path to write the data. If this parameter is not specified, or the parameter value is set to an empty string, a temporary file will be created in the program execution directory. Depending on the permissions to write the files, you might be required to run this program using certain permissions.type: string; default: “”
    percentSpecifies the percentage of disk size to be filled.type: string; default: “”; positive integer of the uint type is acceptable; You must set one of size or percent (both items cannot be “” at the same time).
    sizeSpecifies the volume of data to be read.type: string; default: “”; legal form: the combination of an integer and a unit. For example, 1M, 512kB. Supported units are c=1, w=2, b=512, kB=1000, K=1024, MB=10001000, M=10241024, GB=100010001000, G=102410241024*1024 BYTE and so on. You must set one of size or percent (both items cannot be at the same time).

    Example for simulating disk fill using the service mode

    The result is as follows: