Implementing consent receipt specification in WSO2 Identity Server

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : +94702062877

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

project_gdpr_new_erd.png (191K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Pushpalanka Jayawardhana
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Hasanthi Purnima Dissanayake
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Johann Nallathamby
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : +94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

receipt_tracker.pdf (33K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : +94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Pushpalanka Jayawardhana
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.

On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka
Hi all,


I attached the [1]revised DB design to this mail. Following is a brief description of the tables in our revised DB design.

RECEIPT_TRACKER : This table will keep a track of all the consent receipts which created by the organization ( Currently WSO2 ). This is for the legal purposes.
TRANSACTION_DETAILS : This table will keep the details regarding to the user and a system generated user id for each user by each data controller. ( one user can give their consent to one to many data controllers ).
DATA_CONTROLLER : This table will keep a track of all the data controllers in this scope.
SERVICES : This table will hold all the services given to the user.
PURPOSES : This table will hold all the purposes given to the user to obtain their consent.
SERVICE_MAP_CRID ( will change the name ) : This will keep track of consents which the user gave. ( For this purpose in this service ).
THIRD_PARTY : This table will contain the details of the third party organizations that are going to use the user details for some purposes.
PII_CATEGORY ( will merge with the scopes in IS ) : This will keep the details of personally identifiable information category that are going to collect.
RECEIVED_CR_TRACKER : This table works as a log. This will keep the details of uploaded consent receipt with a status accepted or rejected.
Other tables are to map all these above tables. This is for the legal purposes.

Kindly requesting your feedbacks.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : +94702062877


On Wed, Nov 8, 2017 at 9:50 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.


On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

revised_db_schema_20171108.pdf (44K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Prasanna Dangalla
Hi,

In the attached ER diagram, time stamps are defined to store using 100 characters in VARCHAR data type. I think we can reduce this size of this field If you are using unix time stamps.

Thanks
Prasanna

Prasanna Dangalla
Senior Software Engineer, WSO2, Inc.; http://wso2.com/

lean.enterprise.middleware

cell: +94 718 11 27 51
twitter: @prasa77

On Wed, Nov 8, 2017 at 11:55 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,


I attached the [1]revised DB design to this mail. Following is a brief description of the tables in our revised DB design.

RECEIPT_TRACKER : This table will keep a track of all the consent receipts which created by the organization ( Currently WSO2 ). This is for the legal purposes.
TRANSACTION_DETAILS : This table will keep the details regarding to the user and a system generated user id for each user by each data controller. ( one user can give their consent to one to many data controllers ).
DATA_CONTROLLER : This table will keep a track of all the data controllers in this scope.
SERVICES : This table will hold all the services given to the user.
PURPOSES : This table will hold all the purposes given to the user to obtain their consent.
SERVICE_MAP_CRID ( will change the name ) : This will keep track of consents which the user gave. ( For this purpose in this service ).
THIRD_PARTY : This table will contain the details of the third party organizations that are going to use the user details for some purposes.
PII_CATEGORY ( will merge with the scopes in IS ) : This will keep the details of personally identifiable information category that are going to collect.
RECEIVED_CR_TRACKER : This table works as a log. This will keep the details of uploaded consent receipt with a status accepted or rejected.
Other tables are to map all these above tables. This is for the legal purposes.

Kindly requesting your feedbacks.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 9:50 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.


On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka
Hi all,

First of all I would like to thank you for the kind feedbacks that you have provided and I’m very sorry for the late reply for this feedbacks.

As the feedbacks provided by you, we decided to go through this kind of solution for the PII CATEGORY and the Scopes.


As an example let’s take a PII_CATEGORY called Contact. To this PII_CATEGORY we can add attributes like Address, Email and Telephone number.

Regarding to the scope claim domain I think the PII_CATEGORY Contact is similar to a Scope and the attributes of the Contact is similar to the Claims. So I think we can create a custom scope in the registry to map the above scope and claims.


Kindly requesting the feedbacks from you as previous.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : +94702062877


On Wed, Dec 13, 2017 at 8:54 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi Prasanna,

I changed the data type of the Timestamp to integer. Thank you for your kind feedback.

Thanks,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 3:26 PM, Prasanna Dangalla <[hidden email]> wrote:
Hi,

In the attached ER diagram, time stamps are defined to store using 100 characters in VARCHAR data type. I think we can reduce this size of this field If you are using unix time stamps.

Thanks
Prasanna

Prasanna Dangalla
Senior Software Engineer, WSO2, Inc.; http://wso2.com/

lean.enterprise.middleware

cell: +94 718 11 27 51
twitter: @prasa77

On Wed, Nov 8, 2017 at 11:55 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,


I attached the [1]revised DB design to this mail. Following is a brief description of the tables in our revised DB design.

RECEIPT_TRACKER : This table will keep a track of all the consent receipts which created by the organization ( Currently WSO2 ). This is for the legal purposes.
TRANSACTION_DETAILS : This table will keep the details regarding to the user and a system generated user id for each user by each data controller. ( one user can give their consent to one to many data controllers ).
DATA_CONTROLLER : This table will keep a track of all the data controllers in this scope.
SERVICES : This table will hold all the services given to the user.
PURPOSES : This table will hold all the purposes given to the user to obtain their consent.
SERVICE_MAP_CRID ( will change the name ) : This will keep track of consents which the user gave. ( For this purpose in this service ).
THIRD_PARTY : This table will contain the details of the third party organizations that are going to use the user details for some purposes.
PII_CATEGORY ( will merge with the scopes in IS ) : This will keep the details of personally identifiable information category that are going to collect.
RECEIVED_CR_TRACKER : This table works as a log. This will keep the details of uploaded consent receipt with a status accepted or rejected.
Other tables are to map all these above tables. This is for the legal purposes.

Kindly requesting your feedbacks.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 9:50 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.


On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture





_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka

Hi all,


In this mail I would like to present my current situation of the project.

I implemented some features of the consent receipt generation as the Kantara Consent Receipt specification. In here I will explain those in details. As I mentioned in my previous mails I created a MySQL database  to store the consents of the users. Also I implemented the DAO layer for that database to update and retrieve the data. Now I’m currently implementing the endpoint with RESTful APIs for the above database. I used swagger to create the endpoint’s API documentation and generated the endpoint using CXF. I attached the [1]swagger file( api-user-consent-management.yaml ) below at this mail.


I will briefly describe the APIs here.


/consent/{subjectName} - Getting consent details of an user to populate the UI

/consent/{subjectName}/thirdParty - Getting consent details regarding to the third party which operated on the user data (In here I used a query parameter to send the third party id to the backend)

/consent/{subjectName}/serviceList - Getting details of consented services by an user

/consent/{subjectName}/services/{serviceId} - Getting service details regarding to the subject name and the service id

/consent/{subjectName}/services/{serviceId}/purpose - Getting details of purposes of a service by subject name ( In here I used a query parameter to send the purpose id to the backend )


/consent/configuration/dataController

Post - Adding details of the data controller to the consent database ( used a DTO in the body to send the details. )

Get - Getting data controller details by the id ( used a query parameter to send the data controller id. )

Put - Updating the data controller details of the consent database ( used a DTO in the body to send the details. )


/consent/configuration/personalInfoCategory

Post - Adding a personally identifiable info category to the consent database ( used a DTO in the body to send the details. )

Get - Getting personally identifiable info categories from the database ( used a DTO List to get the details. )

Put - Updating personally identifiable info categories in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/service

Post - Adding details of a service to the consent database ( used a DTO in the body to send the details. )

Get - Getting details of services from the database ( used a DTO List to get the details. )

Put - Updating details of a service in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/purpose

Post - Adding details of a purpose to the consent database ( used a DTO in the body to send the details. )

Get - Getting purpose details from the consent database ( used a DTO List to get the details. )

Put - Update details of a purpose in the consent database ( used a DTO in the body to send the details. )


/consent/revoke

Put - Revoke consent of an user by service and purpose ( used a DTO List to get the details. I used put because I used a flag called STATUS in SERVICE_MAP_CRID table to keep the consent details’ status which can be “Approved” or “Revoked”. I do that because in an organization they keep the revoked user details at least 7 years. )


/consent/{subjectName}/receipt

Get - Getting consent details of an user to create the consent receipt

/consent/receipt/webForm

Post - Adding user consent details from a web form which is provided by the user.


/consent/receipt

Post - Adding user consent details from a JSON which is provided by the user

Further details are in the [1] YAML file below.


[1].api-user-consent-management.yaml
(22K)



Kindly requesting your feedback as before.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : +94702062877


On Wed, Dec 13, 2017 at 10:28 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

First of all I would like to thank you for the kind feedbacks that you have provided and I’m very sorry for the late reply for this feedbacks.

As the feedbacks provided by you, we decided to go through this kind of solution for the PII CATEGORY and the Scopes.


As an example let’s take a PII_CATEGORY called Contact. To this PII_CATEGORY we can add attributes like Address, Email and Telephone number.

Regarding to the scope claim domain I think the PII_CATEGORY Contact is similar to a Scope and the attributes of the Contact is similar to the Claims. So I think we can create a custom scope in the registry to map the above scope and claims.


Kindly requesting the feedbacks from you as previous.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 8:54 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi Prasanna,

I changed the data type of the Timestamp to integer. Thank you for your kind feedback.

Thanks,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 3:26 PM, Prasanna Dangalla <[hidden email]> wrote:
Hi,

In the attached ER diagram, time stamps are defined to store using 100 characters in VARCHAR data type. I think we can reduce this size of this field If you are using unix time stamps.

Thanks
Prasanna

Prasanna Dangalla
Senior Software Engineer, WSO2, Inc.; http://wso2.com/

lean.enterprise.middleware

cell: +94 718 11 27 51
twitter: @prasa77

On Wed, Nov 8, 2017 at 11:55 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,


I attached the [1]revised DB design to this mail. Following is a brief description of the tables in our revised DB design.

RECEIPT_TRACKER : This table will keep a track of all the consent receipts which created by the organization ( Currently WSO2 ). This is for the legal purposes.
TRANSACTION_DETAILS : This table will keep the details regarding to the user and a system generated user id for each user by each data controller. ( one user can give their consent to one to many data controllers ).
DATA_CONTROLLER : This table will keep a track of all the data controllers in this scope.
SERVICES : This table will hold all the services given to the user.
PURPOSES : This table will hold all the purposes given to the user to obtain their consent.
SERVICE_MAP_CRID ( will change the name ) : This will keep track of consents which the user gave. ( For this purpose in this service ).
THIRD_PARTY : This table will contain the details of the third party organizations that are going to use the user details for some purposes.
PII_CATEGORY ( will merge with the scopes in IS ) : This will keep the details of personally identifiable information category that are going to collect.
RECEIVED_CR_TRACKER : This table works as a log. This will keep the details of uploaded consent receipt with a status accepted or rejected.
Other tables are to map all these above tables. This is for the legal purposes.

Kindly requesting your feedbacks.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 9:50 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.


On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture

api-user-consent-management.yaml (29K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Viduranga Gunarathne
Hi Shan,

On Wed, Dec 13, 2017 at 10:39 AM, Shan Jayathilaka <[hidden email]> wrote:

Hi all,


In this mail I would like to present my current situation of the project.

I implemented some features of the consent receipt generation as the Kantara Consent Receipt specification. In here I will explain those in details. As I mentioned in my previous mails I created a MySQL database  to store the consents of the users. Also I implemented the DAO layer for that database to update and retrieve the data. Now I’m currently implementing the endpoint with RESTful APIs for the above database. I used swagger to create the endpoint’s API documentation and generated the endpoint using CXF. I attached the [1]swagger file( api-user-consent-management.yaml ) below at this mail.


I will briefly describe the APIs here.


/consent/{subjectName} - Getting consent details of an user to populate the UI

IMO this should be "/consents" instead of "/consent" because then it will give the idea of selecting a single consent out of a list and can i suggest using "userId" or "userName" instead of "subjectName"

/consent/{subjectName}/thirdParty - Getting consent details regarding to the third party which operated on the user data (In here I used a query parameter to send the third party id to the backend)

Camel case notation is not the best practice when building REST APIs. I think this should be changed to "third-party" 

/consent/{subjectName}/serviceList - Getting details of consented services by an user

serviceList --> /service-list
 

/consent/{subjectName}/services/{serviceId} - Getting service details regarding to the subject name and the service id

/consent/{subjectName}/services/{serviceId}/purpose - Getting details of purposes of a service by subject name ( In here I used a query parameter to send the purpose id to the backend )


/consent/configuration/dataController

Post - Adding details of the data controller to the consent database ( used a DTO in the body to send the details. )

Get - Getting data controller details by the id ( used a query parameter to send the data controller id. )

Put - Updating the data controller details of the consent database ( used a DTO in the body to send the details. )


/consent/configuration/personalInfoCategory

Post - Adding a personally identifiable info category to the consent database ( used a DTO in the body to send the details. )

Get - Getting personally identifiable info categories from the database ( used a DTO List to get the details. )

Put - Updating personally identifiable info categories in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/service

Post - Adding details of a service to the consent database ( used a DTO in the body to send the details. )

Get - Getting details of services from the database ( used a DTO List to get the details. )

Put - Updating details of a service in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/purpose

Post - Adding details of a purpose to the consent database ( used a DTO in the body to send the details. )

Get - Getting purpose details from the consent database ( used a DTO List to get the details. )

Put - Update details of a purpose in the consent database ( used a DTO in the body to send the details. )


/consent/revoke

Put - Revoke consent of an user by service and purpose ( used a DTO List to get the details. I used put because I used a flag called STATUS in SERVICE_MAP_CRID table to keep the consent details’ status which can be “Approved” or “Revoked”. I do that because in an organization they keep the revoked user details at least 7 years. )


/consent/{subjectName}/receipt

Get - Getting consent details of an user to create the consent receipt

/consent/receipt/webForm

Post - Adding user consent details from a web form which is provided by the user.


/consent/receipt

Post - Adding user consent details from a JSON which is provided by the user

Further details are in the [1] YAML file below.


[1].api-user-consent-management.yaml
(22K)



Kindly requesting your feedback as before.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 10:28 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

First of all I would like to thank you for the kind feedbacks that you have provided and I’m very sorry for the late reply for this feedbacks.

As the feedbacks provided by you, we decided to go through this kind of solution for the PII CATEGORY and the Scopes.


As an example let’s take a PII_CATEGORY called Contact. To this PII_CATEGORY we can add attributes like Address, Email and Telephone number.

Regarding to the scope claim domain I think the PII_CATEGORY Contact is similar to a Scope and the attributes of the Contact is similar to the Claims. So I think we can create a custom scope in the registry to map the above scope and claims.


Kindly requesting the feedbacks from you as previous.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 8:54 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi Prasanna,

I changed the data type of the Timestamp to integer. Thank you for your kind feedback.

Thanks,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 3:26 PM, Prasanna Dangalla <[hidden email]> wrote:
Hi,

In the attached ER diagram, time stamps are defined to store using 100 characters in VARCHAR data type. I think we can reduce this size of this field If you are using unix time stamps.

Thanks
Prasanna

Prasanna Dangalla
Senior Software Engineer, WSO2, Inc.; http://wso2.com/

lean.enterprise.middleware

cell: +94 718 11 27 51
twitter: @prasa77

On Wed, Nov 8, 2017 at 11:55 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,


I attached the [1]revised DB design to this mail. Following is a brief description of the tables in our revised DB design.

RECEIPT_TRACKER : This table will keep a track of all the consent receipts which created by the organization ( Currently WSO2 ). This is for the legal purposes.
TRANSACTION_DETAILS : This table will keep the details regarding to the user and a system generated user id for each user by each data controller. ( one user can give their consent to one to many data controllers ).
DATA_CONTROLLER : This table will keep a track of all the data controllers in this scope.
SERVICES : This table will hold all the services given to the user.
PURPOSES : This table will hold all the purposes given to the user to obtain their consent.
SERVICE_MAP_CRID ( will change the name ) : This will keep track of consents which the user gave. ( For this purpose in this service ).
THIRD_PARTY : This table will contain the details of the third party organizations that are going to use the user details for some purposes.
PII_CATEGORY ( will merge with the scopes in IS ) : This will keep the details of personally identifiable information category that are going to collect.
RECEIVED_CR_TRACKER : This table works as a log. This will keep the details of uploaded consent receipt with a status accepted or rejected.
Other tables are to map all these above tables. This is for the legal purposes.

Kindly requesting your feedbacks.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 9:50 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.


On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Regards,
Viduranga Gunarathne
Software Engineer Intern
WSO2

Email : [hidden email]
Mobile : +94712437484
Web : http://wso2.com
https://wso2.com/signature

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka
Hi all,

In this mail I attached the [1]swagger hub link for the api-user-consent-management.yaml for a better view.

Thanks and regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : +94702062877


On Wed, Dec 13, 2017 at 4:26 PM, Viduranga Gunarathne <[hidden email]> wrote:
Hi Shan,

On Wed, Dec 13, 2017 at 10:39 AM, Shan Jayathilaka <[hidden email]> wrote:

Hi all,


In this mail I would like to present my current situation of the project.

I implemented some features of the consent receipt generation as the Kantara Consent Receipt specification. In here I will explain those in details. As I mentioned in my previous mails I created a MySQL database  to store the consents of the users. Also I implemented the DAO layer for that database to update and retrieve the data. Now I’m currently implementing the endpoint with RESTful APIs for the above database. I used swagger to create the endpoint’s API documentation and generated the endpoint using CXF. I attached the [1]swagger file( api-user-consent-management.yaml ) below at this mail.


I will briefly describe the APIs here.


/consent/{subjectName} - Getting consent details of an user to populate the UI

IMO this should be "/consents" instead of "/consent" because then it will give the idea of selecting a single consent out of a list and can i suggest using "userId" or "userName" instead of "subjectName"

/consent/{subjectName}/thirdParty - Getting consent details regarding to the third party which operated on the user data (In here I used a query parameter to send the third party id to the backend)

Camel case notation is not the best practice when building REST APIs. I think this should be changed to "third-party" 

/consent/{subjectName}/serviceList - Getting details of consented services by an user

serviceList --> /service-list
 

/consent/{subjectName}/services/{serviceId} - Getting service details regarding to the subject name and the service id

/consent/{subjectName}/services/{serviceId}/purpose - Getting details of purposes of a service by subject name ( In here I used a query parameter to send the purpose id to the backend )


/consent/configuration/dataController

Post - Adding details of the data controller to the consent database ( used a DTO in the body to send the details. )

Get - Getting data controller details by the id ( used a query parameter to send the data controller id. )

Put - Updating the data controller details of the consent database ( used a DTO in the body to send the details. )


/consent/configuration/personalInfoCategory

Post - Adding a personally identifiable info category to the consent database ( used a DTO in the body to send the details. )

Get - Getting personally identifiable info categories from the database ( used a DTO List to get the details. )

Put - Updating personally identifiable info categories in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/service

Post - Adding details of a service to the consent database ( used a DTO in the body to send the details. )

Get - Getting details of services from the database ( used a DTO List to get the details. )

Put - Updating details of a service in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/purpose

Post - Adding details of a purpose to the consent database ( used a DTO in the body to send the details. )

Get - Getting purpose details from the consent database ( used a DTO List to get the details. )

Put - Update details of a purpose in the consent database ( used a DTO in the body to send the details. )


/consent/revoke

Put - Revoke consent of an user by service and purpose ( used a DTO List to get the details. I used put because I used a flag called STATUS in SERVICE_MAP_CRID table to keep the consent details’ status which can be “Approved” or “Revoked”. I do that because in an organization they keep the revoked user details at least 7 years. )


/consent/{subjectName}/receipt

Get - Getting consent details of an user to create the consent receipt

/consent/receipt/webForm

Post - Adding user consent details from a web form which is provided by the user.


/consent/receipt

Post - Adding user consent details from a JSON which is provided by the user

Further details are in the [1] YAML file below.


[1].api-user-consent-management.yaml
(22K)



Kindly requesting your feedback as before.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 10:28 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

First of all I would like to thank you for the kind feedbacks that you have provided and I’m very sorry for the late reply for this feedbacks.

As the feedbacks provided by you, we decided to go through this kind of solution for the PII CATEGORY and the Scopes.


As an example let’s take a PII_CATEGORY called Contact. To this PII_CATEGORY we can add attributes like Address, Email and Telephone number.

Regarding to the scope claim domain I think the PII_CATEGORY Contact is similar to a Scope and the attributes of the Contact is similar to the Claims. So I think we can create a custom scope in the registry to map the above scope and claims.


Kindly requesting the feedbacks from you as previous.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 8:54 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi Prasanna,

I changed the data type of the Timestamp to integer. Thank you for your kind feedback.

Thanks,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 3:26 PM, Prasanna Dangalla <[hidden email]> wrote:
Hi,

In the attached ER diagram, time stamps are defined to store using 100 characters in VARCHAR data type. I think we can reduce this size of this field If you are using unix time stamps.

Thanks
Prasanna

Prasanna Dangalla
Senior Software Engineer, WSO2, Inc.; http://wso2.com/

lean.enterprise.middleware

cell: +94 718 11 27 51
twitter: @prasa77

On Wed, Nov 8, 2017 at 11:55 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,


I attached the [1]revised DB design to this mail. Following is a brief description of the tables in our revised DB design.

RECEIPT_TRACKER : This table will keep a track of all the consent receipts which created by the organization ( Currently WSO2 ). This is for the legal purposes.
TRANSACTION_DETAILS : This table will keep the details regarding to the user and a system generated user id for each user by each data controller. ( one user can give their consent to one to many data controllers ).
DATA_CONTROLLER : This table will keep a track of all the data controllers in this scope.
SERVICES : This table will hold all the services given to the user.
PURPOSES : This table will hold all the purposes given to the user to obtain their consent.
SERVICE_MAP_CRID ( will change the name ) : This will keep track of consents which the user gave. ( For this purpose in this service ).
THIRD_PARTY : This table will contain the details of the third party organizations that are going to use the user details for some purposes.
PII_CATEGORY ( will merge with the scopes in IS ) : This will keep the details of personally identifiable information category that are going to collect.
RECEIVED_CR_TRACKER : This table works as a log. This will keep the details of uploaded consent receipt with a status accepted or rejected.
Other tables are to map all these above tables. This is for the legal purposes.

Kindly requesting your feedbacks.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 9:50 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.


On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Regards,
Viduranga Gunarathne
Software Engineer Intern
WSO2

Email : [hidden email]
Mobile : <a href="tel:+94%2071%20243%207484" value="+94712437484" target="_blank">+94712437484
Web : http://wso2.com
https://wso2.com/signature

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: Implementing consent receipt specification in WSO2 Identity Server

Shan Jayathilaka
Hi Viduranga,

I will put my attention to the changes that u have suggested. Thank you for the kind feedback.

Thanks and regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : +94702062877


On Thu, Dec 14, 2017 at 9:16 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

In this mail I attached the [1]swagger hub link for the api-user-consent-management.yaml for a better view.

Thanks and regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 4:26 PM, Viduranga Gunarathne <[hidden email]> wrote:
Hi Shan,

On Wed, Dec 13, 2017 at 10:39 AM, Shan Jayathilaka <[hidden email]> wrote:

Hi all,


In this mail I would like to present my current situation of the project.

I implemented some features of the consent receipt generation as the Kantara Consent Receipt specification. In here I will explain those in details. As I mentioned in my previous mails I created a MySQL database  to store the consents of the users. Also I implemented the DAO layer for that database to update and retrieve the data. Now I’m currently implementing the endpoint with RESTful APIs for the above database. I used swagger to create the endpoint’s API documentation and generated the endpoint using CXF. I attached the [1]swagger file( api-user-consent-management.yaml ) below at this mail.


I will briefly describe the APIs here.


/consent/{subjectName} - Getting consent details of an user to populate the UI

IMO this should be "/consents" instead of "/consent" because then it will give the idea of selecting a single consent out of a list and can i suggest using "userId" or "userName" instead of "subjectName"

/consent/{subjectName}/thirdParty - Getting consent details regarding to the third party which operated on the user data (In here I used a query parameter to send the third party id to the backend)

Camel case notation is not the best practice when building REST APIs. I think this should be changed to "third-party" 

/consent/{subjectName}/serviceList - Getting details of consented services by an user

serviceList --> /service-list
 

/consent/{subjectName}/services/{serviceId} - Getting service details regarding to the subject name and the service id

/consent/{subjectName}/services/{serviceId}/purpose - Getting details of purposes of a service by subject name ( In here I used a query parameter to send the purpose id to the backend )


/consent/configuration/dataController

Post - Adding details of the data controller to the consent database ( used a DTO in the body to send the details. )

Get - Getting data controller details by the id ( used a query parameter to send the data controller id. )

Put - Updating the data controller details of the consent database ( used a DTO in the body to send the details. )


/consent/configuration/personalInfoCategory

Post - Adding a personally identifiable info category to the consent database ( used a DTO in the body to send the details. )

Get - Getting personally identifiable info categories from the database ( used a DTO List to get the details. )

Put - Updating personally identifiable info categories in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/service

Post - Adding details of a service to the consent database ( used a DTO in the body to send the details. )

Get - Getting details of services from the database ( used a DTO List to get the details. )

Put - Updating details of a service in the consent database ( used a DTO in the body to send the details. )


/consent/configuration/purpose

Post - Adding details of a purpose to the consent database ( used a DTO in the body to send the details. )

Get - Getting purpose details from the consent database ( used a DTO List to get the details. )

Put - Update details of a purpose in the consent database ( used a DTO in the body to send the details. )


/consent/revoke

Put - Revoke consent of an user by service and purpose ( used a DTO List to get the details. I used put because I used a flag called STATUS in SERVICE_MAP_CRID table to keep the consent details’ status which can be “Approved” or “Revoked”. I do that because in an organization they keep the revoked user details at least 7 years. )


/consent/{subjectName}/receipt

Get - Getting consent details of an user to create the consent receipt

/consent/receipt/webForm

Post - Adding user consent details from a web form which is provided by the user.


/consent/receipt

Post - Adding user consent details from a JSON which is provided by the user

Further details are in the [1] YAML file below.


[1].api-user-consent-management.yaml
(22K)



Kindly requesting your feedback as before.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 10:28 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

First of all I would like to thank you for the kind feedbacks that you have provided and I’m very sorry for the late reply for this feedbacks.

As the feedbacks provided by you, we decided to go through this kind of solution for the PII CATEGORY and the Scopes.


As an example let’s take a PII_CATEGORY called Contact. To this PII_CATEGORY we can add attributes like Address, Email and Telephone number.

Regarding to the scope claim domain I think the PII_CATEGORY Contact is similar to a Scope and the attributes of the Contact is similar to the Claims. So I think we can create a custom scope in the registry to map the above scope and claims.


Kindly requesting the feedbacks from you as previous.


Thanks and regards,


Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Dec 13, 2017 at 8:54 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi Prasanna,

I changed the data type of the Timestamp to integer. Thank you for your kind feedback.

Thanks,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 3:26 PM, Prasanna Dangalla <[hidden email]> wrote:
Hi,

In the attached ER diagram, time stamps are defined to store using 100 characters in VARCHAR data type. I think we can reduce this size of this field If you are using unix time stamps.

Thanks
Prasanna

Prasanna Dangalla
Senior Software Engineer, WSO2, Inc.; http://wso2.com/

lean.enterprise.middleware

cell: +94 718 11 27 51
twitter: @prasa77

On Wed, Nov 8, 2017 at 11:55 AM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,


I attached the [1]revised DB design to this mail. Following is a brief description of the tables in our revised DB design.

RECEIPT_TRACKER : This table will keep a track of all the consent receipts which created by the organization ( Currently WSO2 ). This is for the legal purposes.
TRANSACTION_DETAILS : This table will keep the details regarding to the user and a system generated user id for each user by each data controller. ( one user can give their consent to one to many data controllers ).
DATA_CONTROLLER : This table will keep a track of all the data controllers in this scope.
SERVICES : This table will hold all the services given to the user.
PURPOSES : This table will hold all the purposes given to the user to obtain their consent.
SERVICE_MAP_CRID ( will change the name ) : This will keep track of consents which the user gave. ( For this purpose in this service ).
THIRD_PARTY : This table will contain the details of the third party organizations that are going to use the user details for some purposes.
PII_CATEGORY ( will merge with the scopes in IS ) : This will keep the details of personally identifiable information category that are going to collect.
RECEIVED_CR_TRACKER : This table works as a log. This will keep the details of uploaded consent receipt with a status accepted or rejected.
Other tables are to map all these above tables. This is for the legal purposes.

Kindly requesting your feedbacks.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Wed, Nov 8, 2017 at 9:50 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

As discussed offline, please share the current revised DB design and the assumptions we made on the things that are not certain in the specification.
Also +1 for Johann's suggestion to store PII categories separately then depending on scopes. Hence you can define a separate file in registry to keep PII category to user claims, similar to how we have stored scopes to user claims mapping.


On Mon, Nov 6, 2017 at 3:19 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

As I am developing the consent receipt specification I have to keep the details of the consents which are provided by the users. This is the database design that I designed to store those consents. As a standard, data in a database are not deleted permanently until some time is elapsed. Regarding to the above scenario we need to put a flag to those data to detect that as a deleted one or an active one. In this database mainly two tables are populating from the user actions( TRANSACTION_DETAILS, SERVICE_MAP_CRID). I already put flags to those two tables. Do we need to put a flag to each and every table in this database to detect deleted records.
Ex:- There is a table called PURPOSES to store predefined purposes to capture user consents. When we want to delete a purpose from that can we delete it normally or do we need to keep that with a flag to detect that as a deleted one.

Regards,

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Mon, Oct 23, 2017 at 4:49 PM, Shan Jayathilaka <[hidden email]> wrote:
Hi all,

Regarding to the GDPR regulation, the users have the chance to modify the consents which they provided before for the corresponding organization. When an user requests for his/her consents, the above organization must send the corresponding consents as a consent receipt. This receipt contains an unique ID called Consent Receipt Id. This ID is unique for the consent receipt. When the user modified his/her consent, this receipt id should be changed. Also the organization must keep track of these consent receipts due to legal issues. So we proposed a modification to the consent store.



The above table will keep track of the consent receipts according to the user. This table will populate after the consent receipt is generated to the user.

Brief explanation about the table

CONSENT_RECEIPT_ID - An unique id for the consent receipt.
CONSENT_RECEIPT - Consents of the corresponding consent receipt. (unsigned JWT token)
STATUS - 2 steps.
                      1. When the consent receipt is generated, STATUS will be "Generated" for that receipt,
                      2. If the consents are modified by the user, STATUS will be "modified" at that receipt and if the user is requesting a consent receipt, organization have to check the STATUS before sending the receipt. If the STATUS="Modified", a new ID will generated  with the  STATUS="Generated" for the new receipt and the previous receipt will be at the table with the STATUS="Modified".
SGUID - This is a system generated user id to keep track with the user.
RECEIPT_TIMESTAMP - The date and time of the consent receipt creation.

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2 Inc.

Mobile : <a href="tel:+94%2070%20206%202877" value="+94702062877" target="_blank">+94702062877


On Sun, Oct 1, 2017 at 7:08 PM, Johann Nallathamby <[hidden email]> wrote:
I think it should be the other way around. PII category is protocol agnostic. So we shouldn't store scopes in this new schema Shan is proposing. Instead PII category can be referenced along with the scopes, in registry if that's where scopes are stored.

Regards,
Johann.

On Wed, Sep 20, 2017 at 9:45 AM, Hasanthi Purnima Dissanayake <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
 
With our current implementation in Identity Server we maintain a scope-claim mapping in the registry level. For a scope a single or multiple claims can be mapped and we can define any custom or scope or claim. So IIUC here we can map PII category with scope. So indirectly we can map PII category with claims. But at the moment we don't store those scope - claim mapping in our database. So if we are to map PII category with the scopes we need to store the scopes in the db level.

Thanks,

Hasanthi Dissanayake

Software Engineer | WSO2

E: [hidden email]

M :<a href="tel:071%20840%207133" value="+94718407133" target="_blank">0718407133| http://wso2.com

On Wed, Sep 20, 2017 at 9:09 AM, Pushpalanka Jayawardhana <[hidden email]> wrote:
Hi Shan,

Along with these detail we save in these tables, we need to  keep a mapping to what each PII category means to WSO2 server.
In that case we can think of a PII category as a collection of claims. 

In IS we already have this concept of collection of claims, where we categorize them into a scope. WSO2 APIM already make use of these scopes to provide role based access to resources. We can try to make use of scopes in the place of PII category to establish this mapping with server claims which are actually PII keys. In the 'PII_CATEGORY' table we can keep track of this.

Thanks,

On Wed, Sep 13, 2017 at 2:45 PM, Shan Jayathilaka <[hidden email]> wrote:
There is a new regulation called the EU General Data Protection Regulation (GDPR) which replaces the Data Protection Directive 95/46/EC and was designed to harmonize data privacy laws across Europe. GDPR was passed as a regulation on 27th April 2016 and will be effective from 25th May 2018. Regarding to this regulation any organization who is collecting user data must collect data according to the user's consent. Also if an user request about his/her consents about the user data, the data collecting organization must provide those consents regarding to the user. In here we have to record what are the consents of the user to a database. I designed an [1]ER diagram for the database which collects the user consent. Also I attached [2] GDPR Regulation document ,[3] a blog to understand the GDPR and [4] Kantara Consent Receipt Management to this email. I hope they will be helpful to all.

Brief explanation about the database tables

  • TRANSACTION_DETAILS: Contains details about the consent receipt id and user identification.
  • DATA_CONTROLLER: Contains details about the organization which collects the user data.
  • SERVICES: Contains details about the services provided to the user data.
  • PURPOSES: Contains details about the purposes to collect the user data.
  • THIRD_PARTY: Contains details about the third party organizations which take the user data shared by the data controllers.
  • PII_CATEGORY: Contains details about the personally identifiable information (pii) categories.
[1] 

[2] 

[3]

[4]

Appreciate your feedback.

Regards, 

Shan Chathusanda Jayathilaka
Software Engineer (Intern)
WSO2

Mobile : <a href="tel:070%20206%202877" value="+94702062877" target="_blank">+94702062877



--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Thanks & Regards,

Johann Dilantha Nallathamby
Senior Lead Solutions Engineer
WSO2, Inc.
lean.enterprise.middleware

Mobile - +94777776950

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






--
Pushpalanka.
-- 
Pushpalanka Jayawardhana, B.Sc.Eng.(Hons).
Senior Software Engineer, WSO2 Lanka (pvt) Ltd;  wso2.com/
Mobile: +94779716248
Blog: pushpalankajaya.blogspot.com/ | LinkedIn: lk.linkedin.com/in/pushpalanka/ Twitter: @pushpalanka 



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture






_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




--
Regards,
Viduranga Gunarathne
Software Engineer Intern
WSO2

Email : [hidden email]
Mobile : <a href="tel:+94%2071%20243%207484" value="+94712437484" target="_blank">+94712437484
Web : http://wso2.com
https://wso2.com/signature

_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture




_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture