How to work with cfxdx

How to access and execute various commands on cfxdx

How to access and initialise cfxdx?

The sources that needs to be initialised are maintained in YAML (.yml extension) file. Please check with CloudFabrix for the right yml file to be used for initialisation. The syntax and an example are as given.

$ cfxdx <location of yml file>/<filename>.yml

Example

$ cfxdx http://10.95.1.1/html/cfxdx/all_sources_with_ai.yml

The above is sample command and you need to give right path to the yml file. In the yml file above, all sources and access details are stored.

After Copyright information you are presented with cfxdx command line prompt where you can execute various commands including CloudFabrix's Query Language (cfxql).

How to check what packages are available and their versions?

$ pip freeze | grep cfxdx
Example query and output as follows:
$ (cfxdx_virtual) ram@Rams-MacBook-Pro ~ % pip freeze | grep cfxdx
cfxdx==20.12.20
cfxdx-bundle-centos-all==20.12.20.3
cfxdx-ext-aws==20.12.4
cfxdx-ext-aws-cloudwatch==20.12.9
cfxdx-ext-azure==20.12.15
cfxdx-ext-cfxai-clustering==20.12.18
cfxdx-ext-cfxai-regression==20.12.20.1
cfxdx-ext-cfxql-filter==20.12.17
cfxdx-ext-elasticsearch==20.12.20
cfxdx-ext-file==20.12.19
cfxdx-ext-hdfs==20.12.16
cfxdx-ext-jira==20.12.3
cfxdx-ext-minio==20.12.9
cfxdx-ext-openai==20.12.16
cfxdx-ext-prometheus==20.12.9.2
cfxdx-ext-servicenow==20.12.20.1
cfxdx-ext-slack==20.12.19
cfxdx-ext-splunk==20.12.10
cfxdx-ext-sqlite==20.12.9
cfxdx-ext-vmware==20.12.3

In the above output you can see the version numbers mentioned after ==. The 'ext' after cfxdx indicates that it is an extension and the source name is mentioned after hyphen '-'. For example cfxdx-ext-aws-cloudwatch is a connector package that connects to AWS Cloudwatch for data ingestion.

Please note the above is sample output and the packages and their versions may / may not be the same as above

How to view all available Plugins?

> plugins

Sample Output as shown

Run Plugins from cfxdx command line

How to view Tags?

A tag could be a data source (data producer), a data sinc (data consumer) or an analytical engine (data consumer and also enhancer with additional data).

> tags

Sample output is as follows (Refer to Tag definitions for meaning of *, #, @ at cfxdx Terminology)

How to retrieve a particular tags?

If you know the tag name, just enter as follows

> tags aws

A sample results page is as shown below.

tags aws

How to search or filter by tags?

You can filter all the tags by a particular word.

> tags <word>

For example to retrieve all the tags related to Azure, enter the following command.

> tags azure

A sample results page is as displayed

You see a special character like *, # or @ as first character in tags.

'*' as first character indicates, you can apply destination filter

'#' as first character indicates, you can apply source filter. This means the values are filtered at the source.

'@' as first character indicates, it is an API call.

There could be an occasion where a particular operation is not supported due to limitation at source or destination. Then you will receive a message similar to 'Operation not supported'.

How to view capabilities available for each tag?

A capability is like a function / method executed against the tag.

Execute the following command to show available capabilities for the tag.

> cap

A sample result is as displayed below:

Capabilities

From above, you can notice that get-data capability is available for the tag *slack:users.

How to check what Capabilities are available for a tag?

Capability

Description

get-data

The dataset provides get_data API to retrieve the data.

data-stream

The dataset provides get_data_stream API which is useful when accessing large datasets programmatically

count

Number of rows of data in the datset.

meta-data

Provides Metadata information for the dataset

data-append

Appends data to existing dataset

data-update

Merges data to existing dataset using unique keys

data-replace

Replaces entire dataset with new data supplied.

How to limit number of rows displayed?

You can limit the number of rows displayed by executing the following command

> limit <number>

For example, if you wan to limit the number of rows to 100.

> limit 100

Now whenever you apply a query or get data using 'data', the number of rows of data limited to 100.

While exploring the data using cfxdx, Cloudfabrix recommends setting the number of records to 100.

How to access data with end to end example?

The following procedure displays an end to end example on accessing the data from a source system like AWS.

Step 1: Select the tag for which tag should be viewed

Once in cfxdx command line, enter tag <tagname>.

> tag <tag_name>
example:
> tag *aws-prod:us-east-1:ec2:instances

The above tag returns the following result in setting the tag and also displays the available capabilities.

Step 2 (optional): limit number of data to be shown to say 100.

> limit 100

Step 3: Enter 'data' at command and press 'Enter' to display the data in visualization format.

> data

Please note a visualization console is open with more powerful options available to query the displayed data.

The result is as shown below.

How to navigate against the data in (Data Frame) Visualization (DFViz)?

> ?

Enter '?' (Question mark) on displayed data and all available options are as displayed to navigate against the data (selected on previous data command).

The following help key and description on using DFViz is displayed.

cfxdx DFViz help

Few examples are:

How to do column filter?

After selecting the data of a tag, enter 'c' and press 'enter' and column filter text is open to search for any column.

> c

The result will be as follows. The column names will change depending on the selected tag. An example screen is displayed below.

Now enter Hyperviser and press 'Enter'. The results is as displayed

You can select multiple columns by separating column names with pipeline symbol (|). An example query is as displayed below

The results for above filter is as follows:

How to show all tags related to search term?

Example - Return all tags related to Splunk.
> tags splunk

It returns all the tags with a word splunk in them. An example result is as shown below.

Then run the following command

> tag *splunk:splunkApps

It will set tag to splunk:splunkApps so that queries can be made against the selected tag.

When you execute 'tag <tagname>', the tag is set to the <tagname> you supplied and also shows all capabilities available with the tag so that you can execute available capability. Please refer or click on capabilities to find out more.

How to quit or exit from cfxdx terminal?

Type 'q' and press 'Enter' to quit from cfxdx terminal.

> q

How to filter data by column value?

You can filter displayed data by passing a query string. A query syntax and an example are as shown below.

As example, we would like to return results where description column contains some text.

> data <col_name> contains 'query string'

Example

> data description contains 'My computer is not'

Enter 'c' and 'descr', it shows the following. Click 'q' to come out of the report view

You can also save this result to a file (an example is as given below) rather viewing on screen

> data description contains 'My computer is no' --> /tmp/mytickets.json

How to use API call for @tag to get data?

All tag will have a special character as first character as part of tag name. The '@' as first character shows that it is to be queried using API call.

For example, @cfxobs:metric-instant, @cfxobs:metric-range are some of the tags related to CloudFabrix's Observability solution.

An example query is as shown below

Step 1: Navigate to obs tags (tags related to Observability)

> tags obs

A sample output is as shown

Step 2: Let us chose @cfxobs-metric-instant

> tag @cfxobs:metric-instant

It displays the capabilities. For example, get-data and count are allowed capabilities for above selected tag.

If you type data at the prompt and enter, it will show what parameters are mandatory and what parameters are optional.

A sample output is as shown below

Step 3: Enter the query string (an example query is shown below)

> data metric is 'windows_os_info'

Sample output is as follows:

Step 4 (optional): Query with multiple arguments

> data metric is 'windows_os_info' and timestamp is '10am'

There are various options for the parameters, for example specifying time zone, etc.

How to download Data?

Run the following command to download data into a file like csv.

> data -->./<filename>.csv

The following example demonstrates full process from connecting to tag and downloading the data from say ServiceNow.

Step 1: connect to the tag #snow:indents-update

> tag #snow:indidents-update

Step 2 (Optional): limit number of rows to 100

> limit 100

Step 3 (Optional): Check the data

> data

Step 4: Run data --> <filename>.<ext>

> data --> ./example.csv

File is saved at the path specified.

Various file formats are supported. Some of the formats are .csv, .xsls, .json, .tar (archive) etc.

How to do complex queries?

cfxdx accepts wide range of query strings including the complex queries. An example of a complex query as displayed below.

> tag @cfxobs:metric-range
> metric is '100 - (avg by (instance) (irate(windows_cpu_time_total{mode="idle"}[1m])) * 100)'and timestamp is after -30 days and step is '60m' --> *cfxql:filter instance is '10.95.124.114:9443' --> @cfxml:1hour

In the above query, we are able to pass source (in this case Prometheus) - '100 - (avg by (instance) (irate(windows_cpu_time_total{mode="idle"}[1m])) 100)' and you can further filter by timestamp at the same time pass cfxql (CloudFabrix's Query language) query allowing mixing of various selection criteria in single query line.

Some systems may not allow query at source and in that case

The result is as follows

Access using Command Line

Execute the following commands.

cfxdx credential_file [--http1]
or
cfxdx --host <CFXSERVER> --user <USER_ID>
or
cfxdx credential_file --file <commands_file>
or
cfxdx credential_file --cmds "semicolon separated cfxdx intreractive commands"
--file and --cmds options will execute specified commands and exit (non interactive mode)

credential_file is a json file with one of the following formats.

Format 1: Access Key Based Authentication (Recommended)

{
"host": "<IP-or-Hostname-of-CFX-Platform>",
"access-key": "<user-access-key>",
"secret-key": "<user-secret-key>",
"login-method": "access-key"
}

access-key and secret-key are typically generated from the user's portal in CloudFabrix's Dimensions platform. They are typically in UUID format.

Format 2: Username and Password Based Authentication

{
"host": "<IP-or-Hostname-of-CFX-Platform>",
"username": "<username>",
"password": "<password>", <-- Password can be omitted in that case, tool will prompt
"login-method": "password"
}

Setting Access Keys for API

  • Login to CloudFabrix's Dimensions platform.

  • Click on top left corner (hamburger menu), then click 'Access Keys' as shown below image.

  • Click on top left corner (hamburger menu), then click 'Access Keys' as shown in the above image.

  • Click on '+' to add new access key. It is automatically generated along with the secret key. Copy the secret key from the pop-up view and save it in a file.

  • Copy the access the key from the displayed table.

Please note you can only copy the secret key when it is generated and not later. Please make sure you copy and save the key in a file.