Reporting Resource Catalog Data¶

To report Resource Catalog data, you must first install and connect DataKit and DataFlux Func, then use DataFlux Func to report data to DataKit, which finally reports the data to the Guance workspace.

Install DataKit¶

In the Guance workspace, click Integrations > DataKit, select the DataKit installation method as shown below, then copy the Install Command and execute it on the host.

Open a command-line terminal, log in to the host, and execute the copied Install Command. After installation, you will see the message Install Success.

Learn more about DataKit getting started.

Install DataFlux Func¶

In the Guance workspace, click Integrations > Extensions and follow the steps below to install Func in the command-line terminal.

1. Download the portable version;

1. Install using the auto-deployment script;

1. After installation, enter http://server-IP:8088 in a browser, click Save and Initialize Database to initialize.

For more information about Func installation, refer to Quick Start.

Connect DataFlux Func and DataKit¶

Before using DataFlux Func to write data to DataKit, you must first ensure connectivity. Therefore, after installing DataKit, you need to modify the configuration to allow DataFlux Func to connect.

1. Open the DataKit configuration: sudo vim /usr/local/datakit/conf.d/datakit.conf;

2. Change http_listen = "localhost:9529" to http_listen = "0.0.0.0:9529";

3. Restart DataKit.

sudo datakit --restart

For more details, refer to Connect and Operate DataKit.

Report Resource Catalog Data¶

After DataFlux Func and DataKit are connected, you can write functions in DataFlux Func to report Resource Catalog data.

For API descriptions of DataFlux Func function calls, refer to DataKit API;

For instructions on how DataFlux Func writes data to DataKit, refer to Write Data via DataKit.

Example¶

The following example uses Alibaba Cloud products to illustrate how to report Resource Catalog data through DataFlux Func.

Prerequisites: DataKit and DataFlux Func have been installed and connected.

1. Enter http://server-IP:8088 in a browser, enter the account and password. The default account/password configured during initialization is admin/admin.

1. After logging into DataFlux Func, you can add scripts in the Script Editor.

1. DataFlux Func supports writing scripts in Python to report data. Before adding a script, you need to install the Alibaba Cloud Python SDK.

1) In DataFlux Func, go to Manage > Experimental Features > Enable PIP Tools Module.

2) In the PIP Tools module, click to open the Alibaba Cloud Python SDK URL, copy aliyun-python-sdk-core and install it.

1. After installing the Alibaba Cloud Python SDK, click Script Editor > Add Script Set.

Refer to the DataKit API documentation to obtain the reporting API endpoint and write the script to report data to DataFlux Func. Refer to the Alibaba Cloud API documentation to obtain the fields of Alibaba Cloud product basic information.

After writing the script, click Execute in the upper-right corner to check if the code runs correctly.

Script code reference (replace the relevant content as prompted in the image above):

from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.client import AcsClient
import requests
import json
import time

# Push data to DF
def pushdata(data):
    params = (
        ('token', 'tokn_bW47smmgQpoZxxxxxxx'),
    )
    response = requests.post('http://server-IP:9529/v1/write/custom_object', params=params, data=data)
    print(response.status_code, response.text)

# Main function
@DFF.API('custom_object')
def main():
    # 1. Initialize SDK
    client = AcsClient(
        'LTAI5t6d3sRh3xxxxxxxx',  # your-access-key-id
        'CHdrce2XtMyDAnYJxxxxxxxxxxxxxx',  # your-access-key-secret
        'cn-qingdao',  # your-region-id
    )
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain('ecs.aliyuncs.com')
    request.set_method('POST')
    request.set_version('2014-05-26')
    request.set_action_name('DescribeInstances')
    request.add_query_param('RegionId', 'cn-qingdao')
    response = json.loads(client.do_action_with_exception(request))
    for i in response["Instances"]["Instance"]:
        data = ''' aliyun_ecs,name=%s,host=%s instanceid="%s",os="%s",status="%s",creat_time="%s",publicip="%s",regionid="%s",privateip="%s",cpu=%s,memory=%s %s''' %(
            i['HostName'], 
            i['InstanceId'], 
            i['InstanceId'], 
            i['OSType'],
            i['Status'], 
            i['CreationTime'], 
            i['PublicIpAddress']['IpAddress'],
            i['RegionId'], 
            i['NetworkInterfaces']['NetworkInterface'][0]['PrivateIpSets']['PrivateIpSet'][0]['PrivateIpAddress'],
            i['Cpu'],
            i['Memory'], 
            int(time.time()))
        print(data)
        pushdata(data)

1. After the DataFlux Func script is configured and executed, you can view the reported data in the Guance workspace under Infrastructure > Custom.

1. If you need to schedule the script execution, you can create a timed reporting task in DataFlux Func > Manage > Auto-trigger Configuration.