Pre-Requisites

To inject a fault, the following Pre-Requisites must be met:

  1. The server on which the fault has to be applied must have Ubuntu (U16 or above)/RHEL (7 or above version) operation system.
  2. For agent based deployment, the NH agent should reside on the target server. Users should have connection privileges and ssh connectivity access to the server.
  3. Packages required:
    1. ‘at’ package: The ‘at’ command schedules a command to be run once at a particular time that you normally have permission to run. The ‘at’ command can be anything from a simple reminder message to a complex script.
    2. ‘tc’ package: ‘tc’ command is used to configure Traffic Control in the Linux kernel.
    3. ‘fallocate’ package: ‘fallocate’ command is used to manipulate the allocated disk space for a file.
    4. ‘dd’ package: For I/O Shoot Up, the ‘dd’ package should be installed on the NetHavoc machine.
    5. ‘timedatectl’ package: For Teleport, the ‘timedatectl’ package should be installed on the NetHavoc machine.
    6. ‘az’ package: For Azure Server Termination, the Azure (az) package should be installed on the NetHavoc machine.
    7. ‘gcloud’ package: For GCP, the ‘gcloud’ package should be installed on the NetHavoc machine.
    8. ‘awscli’ package: For AWS, the ‘awscli’ package should be installed on the NetHavoc machine.
  1. For Fill Up Disk fault:
    1. For SSH connection type, the partition selected by the given SSH user should have to write permission to fill up the disk.
    2. For the agent connection type, the partition selected by the user (who has read, and write all capabilities) should have to write permission to fill up the disk.
  2. For Application Termination fault:
    1. For the SSH connection type, the SSH user or SSH group user must have permission to terminate the application/process.
    2. For the agent connection type, the ‘cavisson’ user or ‘cavisson’ group user must have permission to terminate the application/process.
Once a fault is cleared, all the remote programs/scripts and related files made by the script to inject the havoc are removed.
  1. For agentless deployment of Havocs via SSH, you would need to have the SSH credentials along with the SSH port number via which the connectivity will be established with the target server. The credentials provided should have the read/write permission to execute the Havoc e.g. for filling up disk havoc, the credentials should have the write permission to fill up the disk, similarly for application termination, the SSH user should have the permission to terminate processes and application(s).

Pre-Requisites for Cloud-based Applications

To inject havoc(s) in applications deployed on the cloud via the cmon agent, you would need to install the agent on the cloud instances where the application resides.  No special authentication details like service key, authorization token, etc. are required to inject havoc(s).

Agentless Havoc Deployment

You can also deploy Havoc on cloud instances in an agentless manner via SSH. SSH connectivity should be allowed on the targeted instance:

  1. SSH to be installed on the cloud instance where havoc is to be applied.
  2. SSH Port should not be blocked for outside connections so that we can establish connectivity between cloud instances and our NetHavoc machine.

The target cloud machine should be added via the Application Topology option under Configurations.

Figure 1: Application Topology

The next step in deploying agentless Havoc is to provide either the SSH username/password or SSH username/passphrase. In the latter option, you will have to upload the passphrase’ private and public keys. Once the target cloud instance details are added, they will automatically reflect under SSH configuration in NetHavoc. NetHavoc gives you the option to test the SSH connection via the UI itself.

Injecting Faults

NH manager can inject fault(s) in any running instance/server at a specified time, current time or random time. NetHavoc simulates failure at a random point in the time interval. The user needs to provide specific inputs according to faults. The system calls some APIs accordingly, which injects the specified fault into the running instance/server.

Follow the below-mentioned steps for injecting faults and analyzing the application behavior:

To access NetHavoc:

  1. Open a Web browser, and type the following URL:

 https://hostname/UnifiedDashboard/

  1. Enter your username and password on the login page:
 
Figure 2: Login Page

Click Login. The landing screen is displayed. 

4. On the product UI home page, click NetHavoc on the top panel as shown in the below figure.

 
Figure 3: Dashboard

5. This displays the NetHavoc The window displays the Havoc Report section by default.

 
Figure 4: Havoc Report Page(Default)