Cavisson Agent Installation on Cloud Foundry (Tanzu)

Overview

This document details the Cavisson observability agent installation process in CloudFoundry platform to monitor its components and applications deployed in Tanzu Application Service. VMware Tanzu Application Service is an app development and deployment platform for public and private clouds. The observability agents provide metrics across the infrastructure and application components.

Installation Steps 

Machine Agent Installation using BOSH release 

1.  Login to a VM which has BOSH CLI available and download the latest version of machine agent release . You can contact Cavisson sales to get the latest version.

wget https://nde.cav-test.com/HUB/CavMachineAgent-4.x.x.x.tar.gz –no-check-certificate

2.Untar the package downloaded in step1. 

tar -xvzf CavMachineAgent-4.x.x.x.tar.gz

3. Go to the directory CavMachineAgent extracted after untar. 

4. Clone the java release for BOSH Packages from the official bosh git repository. 

git clone https://github.com/bosh-packages/java-release

Upon successful completion of the clone command, a new directory with the name ‘java-release’ will be created in the CavMachine agent directory. .

5. Vendor package the java-release package to Cavisson machine This creates a local blobstore in /opt that contains the vendored package .

bosh vendor-package openjdk-8 java-release

6. Create bosh release for Cavisson machine agent

bosh create-release –sha2 –tarball=CavMachineAgent-4.x.x.x.tgz –version=4.x.x — name=CavMachineAgent

7. Upload the BOSH release to the BOSH

bosh upload-release CavMachineAgent-4.x.x.x.tgz

8. Modify machine agent manifest file (located in the CavMachineAgent directory) to add the controller details and stemcell version.

   

  • Stemcells version can be obtained using command ‘bosh stemcells’.
  • Same version should be mentioned in the manifest file

 

name: CavMachineAgent

version: 4.13.0.46

name: CavMachineAgent version: 4.13.0.46 releases:

  • name: CavMachineAgent version: 4.13.0.46

stemcells:

  • alias: bosh-google-kvm-ubuntu-xenial-go_agent os: ubuntu-xenial

version: “621.176”
instance_groups:

  • name: cmon azs:
    • asia-south1-c instances: 1 jobs:
    • name: cmon version: 13.0.46

release: CavMachineAgent properties:

       cav_controller: nspoc2.cav-test.com:6666

       cav_tier: Tanzu vm_type: micro

stemcell: bosh-google-kvm-ubuntu-xenial-go_agent networks:

  • name: pas addons: [] update:

canaries: 1

max_in_flight: 1

canary_watch_time: 1000-60000

update_watch_time: 1000-60000

Sample Controller parameters:- 

cav_controller 

Mention Cavisson Server IP/Host where data is collected. 

tiername 

Mention the tier name as per your application architecture. 

nfdbip and nfdbport(optional)  

If user wants to add logs in a separate server.Mention Cavisson log collector IP/Host IP and port. 

env_name(optional) 

Mention Environment Name based on which environment is to be monitored. ex if it is preprod you can mention pre-prod env. 

9. Deploy release using updated manifest

bosh -n -d CavMachineAgent deploy manifest.yml

10. Add same controller details in yml as mentioned in manifest.yml

addons:

  • name: CavMachineAgentAddon jobs:

– name: cmon

release: CavMachineAgent properties:
cav_controller: nspoc2.cav-test.com:6666cav_tier: Tanzu
include: stemcell:

– os: ubuntu-xenial releases:

  • name: CavMachineAgent version: 4.13.0.46

11. Update the BOSH director runtime configuration .This runtime configuration applies to all new BOSH deployments going forward.

bosh update-runtime-config runtime.yml

12.

  1. Deploy your

Since existing BOSH deployments won’t be automatically updated with the jobs specified in the runtime configuration, you need to redeploy them so that BOSH rolls out the Cavisson agent

Note:-Replace <cf-name> with your cf deployment name and <deployment yml path> with path of your cf deployment yml.

bosh -d <cf-name> deploy <deployment yml path>  

   

Here is the video link for installation

https://www.cavisson.com/product-resources/netdiagnostics-enterprise- resources/videos-netdiagnostics/agent-installation-on-tas/

Youtube Video Link for Cavisson Agent installation on cloud foundry using BOSH release:-https://youtu.be/Pa7pNR6ewLk

Application Agent Installation using Sidecar 

1. Login to a VM which has cf CLI available and download the latest version of application agent sidecar . You can contact Cavisson sales to get the latest version.

wget https://nde.cav-test.com/HUB/cavisson-agent-sidecar_4xx.zip –no-check-certificate

2. Create cavisson agent build pack 

cf create-buildpack cavisson-agent-sidecar-buildpack cavisson-agent-sidecar_4xx.zip 1

3. Validate if buildpacks got successfully created  

cf buildpacks |grep cavisson-agent-sidecar-buildpack

 4. Modify your application manifest.yml to include cavisson buildpack created in step2 and add controller

The below snippet shows a sample configuration for an application called shopapp

applications: 

  – name: shopapp 

    application: shopapp.war 

    disk_quota: 2G 

    instances: 2 

    memory: 1G 

    buildpacks: 

    – cavisson-agent-sidecar-buildpack 

    – java_buildpack_offline 

    env: 

      JBP_CONFIG_OPEN_JDK_JRE: ‘{ jre: { version: 11.+ }, memory_calculator: { stack_threads: 30 } }’ 

      JAVA_OPTS: ‘-XX:MaxDirectMemorySize=100M -javaagent:/home/vcap/deps/netdiagnostics/lib/ndmain.jar=time,ndAgentJar=/home/vcap/deps/netdiagnostics/lib/ndagent-with-dep.jar,ndHome=/home/vcap/deps/netdiagnostics’ 

      CAV_APP_AGENT_TIER: ShopApp 

      CAV_APP_AGENT_NDCHOST: nspoc2.cav-test.com 

      CAV_APP_AGENT_NDCPORT: 6666 

      CAV_APP_AGENT_NDC_COMM_PROTOCOL: WSS 

      CAV_APP_AGENT_SERVER: server1 

      CAV_APP_AGENT_INSTANCE: instance 

Parameters 

JAVA_OPTS

JAVA_OPTS is an environment variable often used with Java applications to set various command-line options and system properties for the Java Virtual Machine (JVM) when the application is launched. This variable allows you to customize the behavior and configuration of the JVM according to your application’s needs.

CAV_APP_AGENT_NDCHOST

Mention Cavisson Server IP/Host where data is collected.

CAV_APP_AGENT_TIER

Tier is a logical entity that constitutes grouping of individual servers. The value entered here will be visible as a Tier in the metric tree in the product UI.

CAV_APP_AGENT_SERVER

Server constitutes grouping of instances. The server mentioned here comes under the tier defined in the above step in the metric tree.

CAV_APP_AGENT_INSTANCE

Provide an appropriate instance name. The instance(s) will be shown under Tier>Server in the metric tree and all the captured metrics will be visible under the instances.

5. Redeploy the application to get it effective. 

 

 

Here is the video link for installation  
https://www.cavisson.com/product-resources/netdiagnostics-enterprise-resources/videos-netdiagnostics/agent-installation-on-tas/ 

 

Youtube link for Java Application Agent Installation using sidecar:https://youtu.be/Q1C4EHjrJxg 

Uninstallation Steps 

1. To uninstall the machine agent,  delete the release from your BOSH 

bosh delete-release <Release Name> 

 2. To uninstall the Application agent , delete the buildpack 

cf delete-buildpack cavisson-agent-sidecar-buildpack 

Monitoring 

Once you’ve completed the agent Installation you can able to see all listed metrics in Appendix section with in pre-bundled dashboard.

The dashboard provides deep level insights and facilitates a deeper understanding of trends and patterns which includes

Appendix 

Metrics Group

Metric

Cloud Foundry CPU Memory Stats

 

 Memory Active Anonymous (MB)

 Memory Active Anonymous Bytes/Sec

 Memory Inactive Anonymous (MB)

Memory Inactive Anonymous Bytes/Sec

Active File Memory (MB)

Active File Bytes/Sec

 

 

Metrics Group

 

Metric

Cloud Foundry CPU Memory Stats

 Inactive File Memory (MB)

 Inactive File Bytes/Sec

 Mapped File Memory (MB)

 Mapped File Memory Bytes/Sec

 Page Faults

 Page Faults/Sec

 Major Page Faults

 Major Page Faults/Sec

 PageIn

 PageIn/Sec

 PageOut

 PageOut/Sec

RSS Memory (MB)

 RSS Memory Bytes/Sec

 Unevictable Memory(MB)

 Unevictable Memory Bytes/Sec

 Swap Memory (MB)

 

 

Metrics Group

Metric

Cloud Foundry CPU Memory Stats

 Total Swap Memory Bytes/Sec

 Total Memory Usage Toward Limit

 Hierarchical Memory Limit (MB)

 Hierarchical Swap Memory Limit (MB)

 Cache Memory (MB)

 Total Cache Memory Bytes/Sec

 Memory Usage(Pct)

 Memory Usage(MB)

 CPU Usage(Cores)

 CPU Usage Time/Sec

CPU User Usage Time/Sec

 CPU Usage User(Cores)

 

CPU System Usage Time/Sec

 CPU Usage System(Cores)

 Disk Usage (MB)

 Total INodes Used

 

 Metrics Group

Metric

 

 Exclusive Disk Used (MB)

 Exclusive INodes Used

 Transferred Bytes/Sec

 Received Bytes/Sec

 Process Count

 Process Limit

 CPU Limit(Shares)

 Memory Limit(MB)

CPU Cores

  

Cloud Foundry Garden Container IO Stats

 IO Read Bytes/Sec

 IO Write Bytes/Sec

 IO Sync Bytes/Sec

 IO Async Bytes/Sec

 Total IO Bytes/Sec

CF Disk Stats

 Disk Partition Size(MB)

 Disk Partition Used (MB)

Disk Partition Available(MB)

 Disk Partition Used (Pct)

Metrics Group

Metric

Cloud Foundry App Stats

 

App Stats

Cloud Foundry Process Stats

Process CPU Usage

 Process Memory Usage

 Process Disk Usage

 Process Memory Quota

 Process Disk Quota

 Process State

Cloud Foundry Task Stats

 Task State

 Task Memory Limit (MB)

 Task Disk Limit (MB)

 

Application Metrics Group

 Metrics Group

 Metric

Integration Points

 Total Calls

 Calls/Sec

 Avg. Response Time

 Errors

 Average Network Delay in Request

Average Network Delay in Response

 Errors/Sec

 Errors Response Time

 TCP Latency

 Bandwidth(KBPS)

Business Transaction

 Requests per sec

 Average Response Time

 Average Request Size

 Errors/sec

 Normal Average Response Time

 Slow Average Response Time

 Very Slow Average Response Time

 

Metrics Group

Metric

 

Error Average Response Time

Percent Slow and Very Slow Transactions

Average Cpu Time

Average Wait Time

Average Sync Time

Average Queue Time

Thread Hotspot Stats

 Number Of Threads

 Number Of All Hotspot Threads

 All Hotspot Threads Duration (Secs)

 Number Of New Hotspot Threads

 New Hotspot Threads Duration (Secs)

 Number Of Blocked Threads

 Number Of Runnable Threads

 Number Of Time Waiting Threads

 Number Of Waiting Threads

 Number Of Terminated Threads

For details on stem cells refer to https://bosh.cloudfoundry.org/stemcells/