APIM Statistics introduce new field to the Analytics events

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

APIM Statistics introduce new field to the Analytics events

Rukshan Premathunga
Hi All,

Can we check the possibility of the $Subject. In the latest APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod) is include in the event as JSON attribute[1]. I think it is better to moved those attributes as a single value attribute and passed to the Analyzer.
So we can upgrade the stream versioning and do these changes.

Also it is better to persist key type, data center ID in the STAT_DB and we can filter production and Sandbox analytics future. So when migrating we need to alter tables with key type. But we can assume that existing data are from the production key and insert those key type when migrating.

So Any idea about this?


[1]  meta_clientType:{"keyType":"PRODUCTION","correlationID", "19c4350a-2079-406a-bece-808954b4f2d7"} 
Thanks and Regards

--
Rukshan Chathuranga.
Software Engineer.
WSO2, Inc.
+94711822074

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: APIM Statistics introduce new field to the Analytics events

Harsha Kumara-2
Adding Fazlan. Shall we create a JIRA and prioritize this. 

On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara <[hidden email]> wrote:
+1 to include this. It's a basic requirement to differentiate the prod and sandbox traffic in the system. Also, correlation Id will help to track the request and response if needed.

On Sat, Dec 23, 2017 at 12:25 PM, Rukshan Premathunga <[hidden email]> wrote:
Hi All,

Can we check the possibility of the $Subject. In the latest APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod) is include in the event as JSON attribute[1]. I think it is better to moved those attributes as a single value attribute and passed to the Analyzer.
So we can upgrade the stream versioning and do these changes.

Also it is better to persist key type, data center ID in the STAT_DB and we can filter production and Sandbox analytics future. So when migrating we need to alter tables with key type. But we can assume that existing data are from the production key and insert those key type when migrating.

So Any idea about this?


[1]  meta_clientType:{"keyType":"PRODUCTION","correlationID", "19c4350a-2079-406a-bece-808954b4f2d7"} 
Thanks and Regards

--
Rukshan Chathuranga.
Software Engineer.
WSO2, Inc.
<a href="tel:+94%2071%20182%202074" value="+94711822074" target="_blank">+94711822074

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: <a href="tel:+94%2077%20550%205618" value="+94775505618" target="_blank">+94775505618
Blog:harshcreationz.blogspot.com



--
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: +94775505618
Blog:harshcreationz.blogspot.com

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: APIM Statistics introduce new field to the Analytics events

Fazlan Nazeem
+1 to add those extra information. This will strengthen our analytics story. 

@Rukshan, 

1) Any specific reason to move those attributes from meta to value fields? 
2) How are we planning to populate the data center ID? Environment variable?  

Please create a git issue if we haven't done it yet. 

On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara <[hidden email]> wrote:
Adding Fazlan. Shall we create a JIRA and prioritize this. 

On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara <[hidden email]> wrote:
+1 to include this. It's a basic requirement to differentiate the prod and sandbox traffic in the system. Also, correlation Id will help to track the request and response if needed.

On Sat, Dec 23, 2017 at 12:25 PM, Rukshan Premathunga <[hidden email]> wrote:
Hi All,

Can we check the possibility of the $Subject. In the latest APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod) is include in the event as JSON attribute[1]. I think it is better to moved those attributes as a single value attribute and passed to the Analyzer.
So we can upgrade the stream versioning and do these changes.

Also it is better to persist key type, data center ID in the STAT_DB and we can filter production and Sandbox analytics future. So when migrating we need to alter tables with key type. But we can assume that existing data are from the production key and insert those key type when migrating.

So Any idea about this?


[1]  meta_clientType:{"keyType":"PRODUCTION","correlationID", "19c4350a-2079-406a-bece-808954b4f2d7"} 
Thanks and Regards

--
Rukshan Chathuranga.
Software Engineer.
WSO2, Inc.
<a href="tel:+94%2071%20182%202074" value="+94711822074" target="_blank">+94711822074

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: <a href="tel:+94%2077%20550%205618" value="+94775505618" target="_blank">+94775505618
Blog:harshcreationz.blogspot.com



--
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: <a href="tel:+94%2077%20550%205618" value="+94775505618" target="_blank">+94775505618
Blog:harshcreationz.blogspot.com



--
Thanks & Regards,

Fazlan Nazeem
Senior Software Engineer
WSO2 Inc
Mobile : <a href="tel:%2B94%20%280%29%20773%20451194" value="+94773451194" target="_blank">+94772338839

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: APIM Statistics introduce new field to the Analytics events

Hasitha Hiranya
Hi,

One scenario I faced with a customer is as follows.

1. A Custom HTTP header is coming with each request specifying the region request was originated.
2. At gateway, this needs to be extracted as an field and published to Analyser. 
3. Analyser summarises the data and put to a DB (hourly/daily/monthly stats)
4. A dashboard can pull that data and show. 

As per current implementation, we need to 

1. Write a custom handler to extract values
2. Create a new stream to publish new field
3. Learn apache spark queries to perform summarisation
4. Learn Jaggery and create the dashboard

Are we discussing a customisable way to provide above functionality here? If so, yes, it will be a great value addition. 

Thanks  

On Tue, Jan 2, 2018 at 9:26 AM, Fazlan Nazeem <[hidden email]> wrote:
+1 to add those extra information. This will strengthen our analytics story. 

@Rukshan, 

1) Any specific reason to move those attributes from meta to value fields? 
2) How are we planning to populate the data center ID? Environment variable?  

Please create a git issue if we haven't done it yet. 

On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara <[hidden email]> wrote:
Adding Fazlan. Shall we create a JIRA and prioritize this. 

On Mon, Dec 25, 2017 at 10:14 AM, Harsha Kumara <[hidden email]> wrote:
+1 to include this. It's a basic requirement to differentiate the prod and sandbox traffic in the system. Also, correlation Id will help to track the request and response if needed.

On Sat, Dec 23, 2017 at 12:25 PM, Rukshan Premathunga <[hidden email]> wrote:
Hi All,

Can we check the possibility of the $Subject. In the latest APIM(wso2am-2.1.0-update4), correlation ID and the key type(sand or prod) is include in the event as JSON attribute[1]. I think it is better to moved those attributes as a single value attribute and passed to the Analyzer.
So we can upgrade the stream versioning and do these changes.

Also it is better to persist key type, data center ID in the STAT_DB and we can filter production and Sandbox analytics future. So when migrating we need to alter tables with key type. But we can assume that existing data are from the production key and insert those key type when migrating.

So Any idea about this?


[1]  meta_clientType:{"keyType":"PRODUCTION","correlationID", "19c4350a-2079-406a-bece-808954b4f2d7"} 
Thanks and Regards

--
Rukshan Chathuranga.
Software Engineer.
WSO2, Inc.
<a href="tel:+94%2071%20182%202074" value="+94711822074" target="_blank">+94711822074

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: <a href="tel:+94%2077%20550%205618" value="+94775505618" target="_blank">+94775505618
Blog:harshcreationz.blogspot.com



--
Harsha Kumara
Software Engineer, WSO2 Inc.
Mobile: <a href="tel:+94%2077%20550%205618" value="+94775505618" target="_blank">+94775505618
Blog:harshcreationz.blogspot.com



--
Thanks & Regards,

Fazlan Nazeem
Senior Software Engineer
WSO2 Inc
Mobile : <a href="tel:%2B94%20%280%29%20773%20451194" value="+94773451194" target="_blank">+94772338839

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Hasitha Abeykoon
Associate Technical LeadWSO2, Inc.; http://wso2.com


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture