Resource Catalog Data Reporting¶

To report resource catalog data, you must first install and connect DataKit and DataFlux Func, then report the data through DataFlux Func to DataKit, and finally DataKit will report the data to the Guance workspace.

Installing DataKit¶

In the Guance workspace, click sequentially on Integration > DataKit, choose the DataKit installation method as shown in the following information, then copy the installation command and execute it on the host.

Open a command-line terminal tool, log into the host, and execute the copied installation command. After the installation is complete, it will prompt Install Success.

Learn more about DataKit Getting Started.

Installing DataFlux Func¶

In the Guance workspace, click sequentially on Integration > Extensions, and follow the steps below to install Func in the command-line terminal tool.

1. Download the portable version;

1. Automatic deployment script installation;

1. After installation, enter http://Server IP Address:8088 in the browser, and click Save and Initialize Database to initialize.

For more Func installation instructions, refer to Quick Start.

Connecting DataFlux Func and DataKit¶

Before using DataFlux Func to write data to DataKit, ensure connectivity first. Therefore, after installing DataKit, adjust the configuration to allow DataFlux Func to connect.

1. Open the DataKit configuration: sudo vim /usr/local/datakit/conf.d/datakit.conf;

2. Change http_listen = "localhost:9529" to http_listen = "0.0.0.0:9529";

3. Restart DataKit.

sudo datakit --restart

For more details, refer to Connect and Operate DataKit.

Reporting Resource Catalog Data¶

After connecting DataFlux Func and DataKit, you can write functions in DataFlux Func to complete the reporting of resource catalog data.

For API documentation regarding DataFlux Func function calls, refer to DataKit API;

For documentation on how DataFlux Func writes data to DataKit, refer to Write Data via DataKit.

Example Description¶

The example below mainly uses Alibaba Cloud products to illustrate how to report resource catalog data through DataFlux Func.

Prerequisites: Completion of DataKit and DataFlux Func installation and connectivity.

1. Enter http://Server IP Address:8088 in the browser, input the account and password (which can be configured during initialization, default is admin/admin).

1. After logging into DataFlux Func, you can add scripts in the Script Editor.

1. DataFlux Func supports writing scripts in Python for reporting data. Before adding scripts, you need to install the Alibaba Cloud Python SDK first.

1) In DataFlux Func Management > Experimental Features > Enable PIP Tool Module.

2) In the PIP Tool module, open the Alibaba Cloud Python SDK website, copy aliyun-python-sdk-core and install it.

1. After installing the Alibaba Cloud Python SDK, click to enter Script Editor > Add Script Set.

Based on the DataKit API documentation, obtain the reporting API interface and write the script for reporting data to DataFlux. Based on the Alibaba Cloud API documentation, write the fields to get basic information about Alibaba Cloud products.

After finishing the script, you can click Execute in the top-right corner to check if the code runs normally.

The reference script code is as follows, and you can replace relevant content according to the prompts in the image above.

from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.client import AcsClient
import requests
import json
import time

# Push data to DF
def pushdata(data):
    params = (
        ('token', 'tokn_bW47smmgQpoZxxxxxxx'),
    )
    response = requests.post('http://Server IP Address:9529/v1/write/custom_object', params=params, data=data)
    print(response.status_code, response.text)

# Main function
@DFF.API('custom_object')
def main():
    # 1. Initialize SDK
    client = AcsClient(
        'LTAI5t6d3sRh3xxxxxxxx',  # your-access-key-id
        'CHdrce2XtMyDAnYJxxxxxxxxxxxxxx',  # your-access-key-secret
        'cn-qingdao',  # your-region-id
    )
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain('ecs.aliyuncs.com')
    request.set_method('POST')
    request.set_version('2014-05-26')
    request.set_action_name('DescribeInstances')
    request.add_query_param('RegionId', 'cn-qingdao')
    response = json.loads(client.do_action_with_exception(request))
    for i in response["Instances"]["Instance"]:
        data = ''' aliyun_ecs,name=%s,host=%s instanceid="%s",os="%s",status="%s",creat_time="%s",publicip="%s",regionid="%s",privateip="%s",cpu=%s,memory=%s %s''' %(
            i['HostName'], 
            i['InstanceId'], 
            i['InstanceId'], 
            i['OSType'],
            i['Status'], 
            i['CreationTime'], 
            i['PublicIpAddress']['IpAddress'],
            i['RegionId'], 
            i['NetworkInterfaces']['NetworkInterface'][0]['PrivateIpSets']['PrivateIpSet'][0]['PrivateIpAddress'],
            i['Cpu'],
            i['Memory'], 
            int(time.time()))
        print(data)
        pushdata(data)

1. After configuring and executing the DataFlux Func script, you can view the reported data in the Guance workspace under Infrastructure > Custom.

1. If you need to schedule the execution of the script tasks, you can create scheduled reporting tasks in DataFlux Func > Management > Auto Trigger Configuration.