Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to to parse the telegraf data #25430

Open
thrinathadevops opened this issue Oct 7, 2024 · 0 comments
Open

How to to parse the telegraf data #25430

thrinathadevops opened this issue Oct 7, 2024 · 0 comments

Comments

@thrinathadevops
Copy link

we are pushing the data from NATS server to influxDb using telegraf. The telegraf is parsing the NATS (json array) to required format. But still we are not receiving the data to influx. if any can tell me how i can parse the data.

Below is my telegraf configuration, NATS output json array and telegraf output.

[[inputs.nats_consumer]]
servers = ["nats://localhost:4222"]
subjects = ["nats.subject.requestsraw"]
data_format = "json_v2"
name_override = "requestsRaw"

Time-related settings

json_time_key = "time"
json_time_format = "unix_ns" # Unix timestamp in nanoseconds
tagexclude = ["_time"]

[[inputs.nats_consumer.json_v2]]
[[inputs.nats_consumer.json_v2.object]]
path = "@this"
disable_prepend_keys = true
tags = ["runId", "testName", "nodeName", "ObjectType", "errorMessage", "errorResponseBody", "requestName", "responseCode", "result", "samplerType"]

# Field mappings (convert fields to correct types)
[inputs.nats_consumer.json_v2.object.fields]
  connectTime = "integer"
  count = "integer"
  errorCount = "integer"
  latency = "integer"
  processingTime = "integer"
  receivedBytes = "integer"
  sentBytes = "integer"
  responseTime = "integer"

nats json arry

{"runId":"R_20241007-1114","testName":"Test","nodeName":"Test-Node","errorMessage":"noData","errorResponseBody":"noData","requestName":"WebSocket request-response Sampler","responseCode":"200","result":"pass","samplerType":"transaction","connectTime":0,"count":1,"errorCount":0,"latency":0,"processingTime":0,"receivedBytes":59275,"sentBytes":1467,"responseTime":1865,"timeStamp":1728279893708,"time":1728279894329000300,"ObjectType":"requestsRaw"}, {"runId":"R_20241007-1114","testName":"Test","nodeName":"Test-Node","errorMessage":"noData","errorResponseBody":"noData","requestName":"WebSocket request-response Sampler","responseCode":"200","result":"pass","samplerType":"transaction","connectTime":0,"count":1,"errorCount":0,"latency":0,"processingTime":0,"receivedBytes":59275,"sentBytes":1467,"responseTime":1552,"timeStamp":1728279894021,"time":1728279894329000400,"ObjectType":"requestsRaw"}, {"runId":"R_20241007-1114","testName":"Test","nodeName":"Test-Node","errorMessage":"noData","errorResponseBody":"noData","requestName":"WebSocket request-response Sampler","responseCode":"200","result":"pass","samplerType":"transaction","connectTime":0,"count":1,"errorCount":0,"latency":0,"processingTime":0,"receivedBytes":59275,"sentBytes":1467,"responseTime":1448,"timeStamp":1728279894125,"time":1728279894329000500,"ObjectType":"requestsRaw"}, {"runId":"R_20241007-1114","testName":"Test","nodeName":"Test-Node","errorMessage":"noData","errorResponseBody":"noData","requestName":"WebSocket request-response Sampler","responseCode":"200","result":"pass","samplerType":"transaction","connectTime":0,"count":1,"errorCount":0,"latency":0,"processingTime":0,"receivedBytes":59275,"sentBytes":1467,"responseTime":1425,"timeStamp":1728279894148,"time":1728279894329000600,"ObjectType":"requestsRaw"}]

telegraf output

requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241007-1114,samplerType=transaction,testName=Test connectTime=0,count=1,errorCount=0,latency=0,processingTime=0,receivedBytes=59275,sentBytes=1467,responseTime=1446,timeStamp=1728279894126,time=1728279894329000200 1728279895627287728
requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241007-1114,samplerType=transaction,testName=Test connectTime=0,count=1,errorCount=0,latency=0,processingTime=0,receivedBytes=59275,sentBytes=1467,responseTime=1913,timeStamp=1728279893660,time=1728279894329000200 1728279895627287728
requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241007-1114,samplerType=transaction,testName=Test connectTime=0,count=1,errorCount=0,latency=0,processingTime=0,receivedBytes=59275,sentBytes=1467,responseTime=1865,timeStamp=1728279893708,time=1728279894329000200 1728279895627287728
requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241007-1114,samplerType=transaction,testName=Test connectTime=0,count=1,errorCount=0,latency=0,processingTime=0,receivedBytes=59275,sentBytes=1467,responseTime=1552,timeStamp=1728279894021,time=1728279894329000400 1728279895627287728
requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241007-1114,samplerType=transaction,testName=Test connectTime=0,count=1,errorCount=0,latency=0,processingTime=0,receivedBytes=59275,sentBytes=1467,responseTime=1448,timeStamp=1728279894125,time=1728279894329000400 1728279895627287728
requestsRaw,ObjectType=requestsRaw,errorMessage=noData,errorResponseBody=noData,host=server-3,nodeName=Test-Node,requestName=WebSocket\ request-response\ Sampler,responseCode=200,result=pass,runId=R_20241007-1114,samplerType=transaction,testName=Test connectTime=0,count=1,errorCount=0,latency=0,processingTime=0,receivedBytes=59275,sentBytes=1467,responseTime=1425,timeStamp=1728279894148,time=1728279894329000700 1728279895627287728

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant