GSOC 2016 - Project 21 : MongoDB Userstore Development

[Asantha Thilina]

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[tharindue]

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[tharindue]

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[tharindue]

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[tharindue]

The call details are as following.

1. Demonstrated how to use WSO2 admin services.

2. All the user operations in RemoteUserStoreManagerService [1] should be tested (using SOAP UI) for the mongodb userstore.

3. When the user profile is saved in a JDBC userstore, for each attribute of the user, it will add a new entry in UM_USER_ATTRIBUTE (Refer [2] for more information) like below.

+-------+----------------------+-----------------+---------------+------------+--------------+
| UM_ID | UM_ATTR_NAME         | UM_ATTR_VALUE   | UM_PROFILE_ID | UM_USER_ID | UM_TENANT_ID |
+-------+----------------------+-----------------+---------------+------------+--------------+
|     1 | im                   |                 | default       |          1 |        -1234 |
|     2 | region               | Western         | default       |          1 |        -1234 |
|     3 | streetAddress        |                 | default       |          1 |        -1234 |
|     4 | country              |                 | default       |          1 |        -1234 |
|     5 | mobile               |                 | default       |          1 |        -1234 |
|     6 | sn                   | NewLastname     | default       |          1 |        -1234 |
|     7 | profileConfiguration | default         | default       |          1 |        -1234 |
|     8 | dateOfBirth          |                 | default       |          1 |        -1234 |
|     9 | mail                 | [hidden email] | default       |          1 |        -1234 |
|    10 | organizationName     | WSO2            | default       |          1 |        -1234 |
|    11 | givenName            | NewUser         | default       |          1 |        -1234 |
|    12 | province             | western         | default       |          1 |        -1234 |
+-------+----------------------+-----------------+---------------+------------+--------------+

Performance wise this is not a good design. For mongodb userstore, I suggested to add a new document in UM_USER_ATTRIBUTE collection for each user. If the attribute value is empty in the profile, an empty string can be stored.

{

   "im": "",

   " region": "Western",

   "street": "Address",

   "country": "",

   "mobile": "",

   "sn": "NewLastname",

   "profileConfiguration": "default",

   "dateOfBirth": "",

   "mail": "[hidden email]",

   "organizationName": "WSO2",

   "givenName": "NewUser",

   "province": "western"

}

3. Profile saving is having some issues currently and we need to further investigate what is going wrong. Until the issue is figured out, asked to manually create json documents in UM_USER_ATTRIBUTE collection and implement the retrieval of user attributes. (can test getUserClaimValues method in the admin service using SOAP UI)

4. For the analytics part of the project, suggested to extend the AbstractUserOperationEventListener class [1] and override the methods for publishing events.

5. Developer documentation, Administration Guide and Testing Guide (sample SOAP requests and responses in RemoteUserStoreManagerService API) should be written as deliverables. For all actions related to the mongodb userstore manager (i.e add user, delete user, add role ...) , Selenium scripts should be provided (can use firefox selenium addon and record each operation and provide the scripts).

So far the progress is satisfactory. Keep on the good work !

[1] https://localhost:9443/services/RemoteUserStoreManagerService?wsdl
[2] http://tharindue.blogspot.com/2015/04/wso2-identity-server-data-dictionary.html
[3] https://github.com/wso2/carbon-kernel/blob/v4.4.3/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/common/AbstractUserOperationEventListener.java

Thank you,

TharinduE

On Sat, Jun 4, 2016 at 9:48 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

As we discussed on hangout i doing the changes in UM_USER_ATTRIBUTE collection and also changing the hybrid role to master datasource but when i try to add user to hybrid role i facing a small issue i can't grab the datasource ,in HybridRoleManager class it need to give java.sql.DataSource but in my current user store manager class i have never used a sql datasource(since it nosql) so i think i need to grab the master datasource of IS which is h2 database datasource, is there a way to get the primary datasource?

Thanks,

Asantha

On Sat, Jun 4, 2016 at 1:59 PM, Tharindu Edirisinghe <[hidden email]> wrote:

The call details are as following.

1. Demonstrated how to use WSO2 admin services.

2. All the user operations in RemoteUserStoreManagerService [1] should be tested (using SOAP UI) for the mongodb userstore.

3. When the user profile is saved in a JDBC userstore, for each attribute of the user, it will add a new entry in UM_USER_ATTRIBUTE (Refer [2] for more information) like below.

+-------+----------------------+-----------------+---------------+------------+--------------+
| UM_ID | UM_ATTR_NAME         | UM_ATTR_VALUE   | UM_PROFILE_ID | UM_USER_ID | UM_TENANT_ID |
+-------+----------------------+-----------------+---------------+------------+--------------+
|     1 | im                   |                 | default       |          1 |        -1234 |
|     2 | region               | Western         | default       |          1 |        -1234 |
|     3 | streetAddress        |                 | default       |          1 |        -1234 |
|     4 | country              |                 | default       |          1 |        -1234 |
|     5 | mobile               |                 | default       |          1 |        -1234 |
|     6 | sn                   | NewLastname     | default       |          1 |        -1234 |
|     7 | profileConfiguration | default         | default       |          1 |        -1234 |
|     8 | dateOfBirth          |                 | default       |          1 |        -1234 |
|     9 | mail                 | [hidden email] | default       |          1 |        -1234 |
|    10 | organizationName     | WSO2            | default       |          1 |        -1234 |
|    11 | givenName            | NewUser         | default       |          1 |        -1234 |
|    12 | province             | western         | default       |          1 |        -1234 |
+-------+----------------------+-----------------+---------------+------------+--------------+

Performance wise this is not a good design. For mongodb userstore, I suggested to add a new document in UM_USER_ATTRIBUTE collection for each user. If the attribute value is empty in the profile, an empty string can be stored.

{

   "im": "",

   " region": "Western",

   "street": "Address",

   "country": "",

   "mobile": "",

   "sn": "NewLastname",

   "profileConfiguration": "default",

   "dateOfBirth": "",

   "mail": "[hidden email]",

   "organizationName": "WSO2",

   "givenName": "NewUser",

   "province": "western"

}

3. Profile saving is having some issues currently and we need to further investigate what is going wrong. Until the issue is figured out, asked to manually create json documents in UM_USER_ATTRIBUTE collection and implement the retrieval of user attributes. (can test getUserClaimValues method in the admin service using SOAP UI)

4. For the analytics part of the project, suggested to extend the AbstractUserOperationEventListener class [1] and override the methods for publishing events.

5. Developer documentation, Administration Guide and Testing Guide (sample SOAP requests and responses in RemoteUserStoreManagerService API) should be written as deliverables. For all actions related to the mongodb userstore manager (i.e add user, delete user, add role ...) , Selenium scripts should be provided (can use firefox selenium addon and record each operation and provide the scripts).

So far the progress is satisfactory. Keep on the good work !

[1] https://localhost:9443/services/RemoteUserStoreManagerService?wsdl
[2] http://tharindue.blogspot.com/2015/04/wso2-identity-server-data-dictionary.html
[3] https://github.com/wso2/carbon-kernel/blob/v4.4.3/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/common/AbstractUserOperationEventListener.java

Thank you,

TharinduE

On Sat, Jun 4, 2016 at 9:48 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[tharindue]

Hi Asantha,

We might not need implementing the HybridRoleManager... I'll provide you the exact details after trying out the functionality you have implemented up to now.

Can you please write a basic document with the steps for creating the mongodb database and adding a userstore in IS pointing to the created mongodb database and include in the ReadMe of github repo.

Also if mongodb server is running in a different port other than the default port, can we still connect to that ? I didn't see a place to define the port in the configuration.. should it be added as a connection string ? (didn't go through the code to find what is going wrong)

Thanks,

TharinduE

On Sun, Jun 5, 2016 at 11:55 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

As we discussed on hangout i doing the changes in UM_USER_ATTRIBUTE collection and also changing the hybrid role to master datasource but when i try to add user to hybrid role i facing a small issue i can't grab the datasource ,in HybridRoleManager class it need to give java.sql.DataSource but in my current user store manager class i have never used a sql datasource(since it nosql) so i think i need to grab the master datasource of IS which is h2 database datasource, is there a way to get the primary datasource?

Thanks,

Asantha

On Sat, Jun 4, 2016 at 1:59 PM, Tharindu Edirisinghe <[hidden email]> wrote:

The call details are as following.

1. Demonstrated how to use WSO2 admin services.

2. All the user operations in RemoteUserStoreManagerService [1] should be tested (using SOAP UI) for the mongodb userstore.

3. When the user profile is saved in a JDBC userstore, for each attribute of the user, it will add a new entry in UM_USER_ATTRIBUTE (Refer [2] for more information) like below.

+-------+----------------------+-----------------+---------------+------------+--------------+
| UM_ID | UM_ATTR_NAME         | UM_ATTR_VALUE   | UM_PROFILE_ID | UM_USER_ID | UM_TENANT_ID |
+-------+----------------------+-----------------+---------------+------------+--------------+
|     1 | im                   |                 | default       |          1 |        -1234 |
|     2 | region               | Western         | default       |          1 |        -1234 |
|     3 | streetAddress        |                 | default       |          1 |        -1234 |
|     4 | country              |                 | default       |          1 |        -1234 |
|     5 | mobile               |                 | default       |          1 |        -1234 |
|     6 | sn                   | NewLastname     | default       |          1 |        -1234 |
|     7 | profileConfiguration | default         | default       |          1 |        -1234 |
|     8 | dateOfBirth          |                 | default       |          1 |        -1234 |
|     9 | mail                 | [hidden email] | default       |          1 |        -1234 |
|    10 | organizationName     | WSO2            | default       |          1 |        -1234 |
|    11 | givenName            | NewUser         | default       |          1 |        -1234 |
|    12 | province             | western         | default       |          1 |        -1234 |
+-------+----------------------+-----------------+---------------+------------+--------------+

Performance wise this is not a good design. For mongodb userstore, I suggested to add a new document in UM_USER_ATTRIBUTE collection for each user. If the attribute value is empty in the profile, an empty string can be stored.

{

   "im": "",

   " region": "Western",

   "street": "Address",

   "country": "",

   "mobile": "",

   "sn": "NewLastname",

   "profileConfiguration": "default",

   "dateOfBirth": "",

   "mail": "[hidden email]",

   "organizationName": "WSO2",

   "givenName": "NewUser",

   "province": "western"

}

3. Profile saving is having some issues currently and we need to further investigate what is going wrong. Until the issue is figured out, asked to manually create json documents in UM_USER_ATTRIBUTE collection and implement the retrieval of user attributes. (can test getUserClaimValues method in the admin service using SOAP UI)

4. For the analytics part of the project, suggested to extend the AbstractUserOperationEventListener class [1] and override the methods for publishing events.

5. Developer documentation, Administration Guide and Testing Guide (sample SOAP requests and responses in RemoteUserStoreManagerService API) should be written as deliverables. For all actions related to the mongodb userstore manager (i.e add user, delete user, add role ...) , Selenium scripts should be provided (can use firefox selenium addon and record each operation and provide the scripts).

So far the progress is satisfactory. Keep on the good work !

[1] https://localhost:9443/services/RemoteUserStoreManagerService?wsdl
[2] http://tharindue.blogspot.com/2015/04/wso2-identity-server-data-dictionary.html
[3] https://github.com/wso2/carbon-kernel/blob/v4.4.3/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/common/AbstractUserOperationEventListener.java

Thank you,

TharinduE

On Sat, Jun 4, 2016 at 9:48 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

ok sure i will modify the readme content as appropriately,at the moment i didn't add that feature, set to all default i will modify that to add a port number also,

Thanks,

Asantha

On Mon, Jun 6, 2016 at 12:04 AM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

We might not need implementing the HybridRoleManager... I'll provide you the exact details after trying out the functionality you have implemented up to now.

Can you please write a basic document with the steps for creating the mongodb database and adding a userstore in IS pointing to the created mongodb database and include in the ReadMe of github repo.

Also if mongodb server is running in a different port other than the default port, can we still connect to that ? I didn't see a place to define the port in the configuration.. should it be added as a connection string ? (didn't go through the code to find what is going wrong)

Thanks,

TharinduE

On Sun, Jun 5, 2016 at 11:55 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

As we discussed on hangout i doing the changes in UM_USER_ATTRIBUTE collection and also changing the hybrid role to master datasource but when i try to add user to hybrid role i facing a small issue i can't grab the datasource ,in HybridRoleManager class it need to give java.sql.DataSource but in my current user store manager class i have never used a sql datasource(since it nosql) so i think i need to grab the master datasource of IS which is h2 database datasource, is there a way to get the primary datasource?

Thanks,

Asantha

On Sat, Jun 4, 2016 at 1:59 PM, Tharindu Edirisinghe <[hidden email]> wrote:

The call details are as following.

1. Demonstrated how to use WSO2 admin services.

2. All the user operations in RemoteUserStoreManagerService [1] should be tested (using SOAP UI) for the mongodb userstore.

3. When the user profile is saved in a JDBC userstore, for each attribute of the user, it will add a new entry in UM_USER_ATTRIBUTE (Refer [2] for more information) like below.

+-------+----------------------+-----------------+---------------+------------+--------------+
| UM_ID | UM_ATTR_NAME         | UM_ATTR_VALUE   | UM_PROFILE_ID | UM_USER_ID | UM_TENANT_ID |
+-------+----------------------+-----------------+---------------+------------+--------------+
|     1 | im                   |                 | default       |          1 |        -1234 |
|     2 | region               | Western         | default       |          1 |        -1234 |
|     3 | streetAddress        |                 | default       |          1 |        -1234 |
|     4 | country              |                 | default       |          1 |        -1234 |
|     5 | mobile               |                 | default       |          1 |        -1234 |
|     6 | sn                   | NewLastname     | default       |          1 |        -1234 |
|     7 | profileConfiguration | default         | default       |          1 |        -1234 |
|     8 | dateOfBirth          |                 | default       |          1 |        -1234 |
|     9 | mail                 | [hidden email] | default       |          1 |        -1234 |
|    10 | organizationName     | WSO2            | default       |          1 |        -1234 |
|    11 | givenName            | NewUser         | default       |          1 |        -1234 |
|    12 | province             | western         | default       |          1 |        -1234 |
+-------+----------------------+-----------------+---------------+------------+--------------+

Performance wise this is not a good design. For mongodb userstore, I suggested to add a new document in UM_USER_ATTRIBUTE collection for each user. If the attribute value is empty in the profile, an empty string can be stored.

{

   "im": "",

   " region": "Western",

   "street": "Address",

   "country": "",

   "mobile": "",

   "sn": "NewLastname",

   "profileConfiguration": "default",

   "dateOfBirth": "",

   "mail": "[hidden email]",

   "organizationName": "WSO2",

   "givenName": "NewUser",

   "province": "western"

}

3. Profile saving is having some issues currently and we need to further investigate what is going wrong. Until the issue is figured out, asked to manually create json documents in UM_USER_ATTRIBUTE collection and implement the retrieval of user attributes. (can test getUserClaimValues method in the admin service using SOAP UI)

4. For the analytics part of the project, suggested to extend the AbstractUserOperationEventListener class [1] and override the methods for publishing events.

5. Developer documentation, Administration Guide and Testing Guide (sample SOAP requests and responses in RemoteUserStoreManagerService API) should be written as deliverables. For all actions related to the mongodb userstore manager (i.e add user, delete user, add role ...) , Selenium scripts should be provided (can use firefox selenium addon and record each operation and provide the scripts).

So far the progress is satisfactory. Keep on the good work !

[1] https://localhost:9443/services/RemoteUserStoreManagerService?wsdl
[2] http://tharindue.blogspot.com/2015/04/wso2-identity-server-data-dictionary.html
[3] https://github.com/wso2/carbon-kernel/blob/v4.4.3/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/common/AbstractUserOperationEventListener.java

Thank you,

TharinduE

On Sat, Jun 4, 2016 at 9:48 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

I have fixed the claims issue now it can update and add claims to user but still the db structure is same as in relational structure i have currently mapping it to a way we discussed ,most of the issues being fixed now following tasks can do in now MongoDBUserStoreManager

can add users.

can add roles.

can assign roles to users.

can assign users to roles.

can edit user claim values in update profile.

I creating the documentation of my MongoDBUserStoreManager as well sooner i will publish the document and also test scripts as well

And also following are the issues now i facing in

registered users in my user store domain can't login to the IS it's giving error as MongoDBUserStoreManager can't cast to AbstractUserStoreManager since i done MongoDBUserStoreManager by implementing the UserStoreManager is there any special changes to do in my code to authenticate user ? i debugged and see my doAuthenticate method not get triggered during runtime

expect some advice from you

Thanks,

Asantha

On Mon, Jun 6, 2016 at 1:18 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

ok sure i will modify the readme content as appropriately,at the moment i didn't add that feature, set to all default i will modify that to add a port number also,

Thanks,

Asantha

On Mon, Jun 6, 2016 at 12:04 AM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

We might not need implementing the HybridRoleManager... I'll provide you the exact details after trying out the functionality you have implemented up to now.

Can you please write a basic document with the steps for creating the mongodb database and adding a userstore in IS pointing to the created mongodb database and include in the ReadMe of github repo.

Also if mongodb server is running in a different port other than the default port, can we still connect to that ? I didn't see a place to define the port in the configuration.. should it be added as a connection string ? (didn't go through the code to find what is going wrong)

Thanks,

TharinduE

On Sun, Jun 5, 2016 at 11:55 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

As we discussed on hangout i doing the changes in UM_USER_ATTRIBUTE collection and also changing the hybrid role to master datasource but when i try to add user to hybrid role i facing a small issue i can't grab the datasource ,in HybridRoleManager class it need to give java.sql.DataSource but in my current user store manager class i have never used a sql datasource(since it nosql) so i think i need to grab the master datasource of IS which is h2 database datasource, is there a way to get the primary datasource?

Thanks,

Asantha

On Sat, Jun 4, 2016 at 1:59 PM, Tharindu Edirisinghe <[hidden email]> wrote:

The call details are as following.

1. Demonstrated how to use WSO2 admin services.

2. All the user operations in RemoteUserStoreManagerService [1] should be tested (using SOAP UI) for the mongodb userstore.

3. When the user profile is saved in a JDBC userstore, for each attribute of the user, it will add a new entry in UM_USER_ATTRIBUTE (Refer [2] for more information) like below.

+-------+----------------------+-----------------+---------------+------------+--------------+
| UM_ID | UM_ATTR_NAME         | UM_ATTR_VALUE   | UM_PROFILE_ID | UM_USER_ID | UM_TENANT_ID |
+-------+----------------------+-----------------+---------------+------------+--------------+
|     1 | im                   |                 | default       |          1 |        -1234 |
|     2 | region               | Western         | default       |          1 |        -1234 |
|     3 | streetAddress        |                 | default       |          1 |        -1234 |
|     4 | country              |                 | default       |          1 |        -1234 |
|     5 | mobile               |                 | default       |          1 |        -1234 |
|     6 | sn                   | NewLastname     | default       |          1 |        -1234 |
|     7 | profileConfiguration | default         | default       |          1 |        -1234 |
|     8 | dateOfBirth          |                 | default       |          1 |        -1234 |
|     9 | mail                 | [hidden email] | default       |          1 |        -1234 |
|    10 | organizationName     | WSO2            | default       |          1 |        -1234 |
|    11 | givenName            | NewUser         | default       |          1 |        -1234 |
|    12 | province             | western         | default       |          1 |        -1234 |
+-------+----------------------+-----------------+---------------+------------+--------------+

Performance wise this is not a good design. For mongodb userstore, I suggested to add a new document in UM_USER_ATTRIBUTE collection for each user. If the attribute value is empty in the profile, an empty string can be stored.

{

   "im": "",

   " region": "Western",

   "street": "Address",

   "country": "",

   "mobile": "",

   "sn": "NewLastname",

   "profileConfiguration": "default",

   "dateOfBirth": "",

   "mail": "[hidden email]",

   "organizationName": "WSO2",

   "givenName": "NewUser",

   "province": "western"

}

3. Profile saving is having some issues currently and we need to further investigate what is going wrong. Until the issue is figured out, asked to manually create json documents in UM_USER_ATTRIBUTE collection and implement the retrieval of user attributes. (can test getUserClaimValues method in the admin service using SOAP UI)

4. For the analytics part of the project, suggested to extend the AbstractUserOperationEventListener class [1] and override the methods for publishing events.

5. Developer documentation, Administration Guide and Testing Guide (sample SOAP requests and responses in RemoteUserStoreManagerService API) should be written as deliverables. For all actions related to the mongodb userstore manager (i.e add user, delete user, add role ...) , Selenium scripts should be provided (can use firefox selenium addon and record each operation and provide the scripts).

So far the progress is satisfactory. Keep on the good work !

[1] https://localhost:9443/services/RemoteUserStoreManagerService?wsdl
[2] http://tharindue.blogspot.com/2015/04/wso2-identity-server-data-dictionary.html
[3] https://github.com/wso2/carbon-kernel/blob/v4.4.3/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/common/AbstractUserOperationEventListener.java

Thank you,

TharinduE

On Sat, Jun 4, 2016 at 9:48 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

sorry i forget to attach my github repository here is the repository link i have update the repository with my latest errors fixed
https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 8, 2016 at 8:00 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have fixed the claims issue now it can update and add claims to user but still the db structure is same as in relational structure i have currently mapping it to a way we discussed ,most of the issues being fixed now following tasks can do in now MongoDBUserStoreManager

can add users.

can add roles.

can assign roles to users.

can assign users to roles.

can edit user claim values in update profile.

I creating the documentation of my MongoDBUserStoreManager as well sooner i will publish the document and also test scripts as well

And also following are the issues now i facing in

registered users in my user store domain can't login to the IS it's giving error as MongoDBUserStoreManager can't cast to AbstractUserStoreManager since i done MongoDBUserStoreManager by implementing the UserStoreManager is there any special changes to do in my code to authenticate user ? i debugged and see my doAuthenticate method not get triggered during runtime

expect some advice from you

Thanks,

Asantha

On Mon, Jun 6, 2016 at 1:18 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

ok sure i will modify the readme content as appropriately,at the moment i didn't add that feature, set to all default i will modify that to add a port number also,

Thanks,

Asantha

On Mon, Jun 6, 2016 at 12:04 AM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

We might not need implementing the HybridRoleManager... I'll provide you the exact details after trying out the functionality you have implemented up to now.

Can you please write a basic document with the steps for creating the mongodb database and adding a userstore in IS pointing to the created mongodb database and include in the ReadMe of github repo.

Also if mongodb server is running in a different port other than the default port, can we still connect to that ? I didn't see a place to define the port in the configuration.. should it be added as a connection string ? (didn't go through the code to find what is going wrong)

Thanks,

TharinduE

On Sun, Jun 5, 2016 at 11:55 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

As we discussed on hangout i doing the changes in UM_USER_ATTRIBUTE collection and also changing the hybrid role to master datasource but when i try to add user to hybrid role i facing a small issue i can't grab the datasource ,in HybridRoleManager class it need to give java.sql.DataSource but in my current user store manager class i have never used a sql datasource(since it nosql) so i think i need to grab the master datasource of IS which is h2 database datasource, is there a way to get the primary datasource?

Thanks,

Asantha

On Sat, Jun 4, 2016 at 1:59 PM, Tharindu Edirisinghe <[hidden email]> wrote:

The call details are as following.

1. Demonstrated how to use WSO2 admin services.

2. All the user operations in RemoteUserStoreManagerService [1] should be tested (using SOAP UI) for the mongodb userstore.

3. When the user profile is saved in a JDBC userstore, for each attribute of the user, it will add a new entry in UM_USER_ATTRIBUTE (Refer [2] for more information) like below.

+-------+----------------------+-----------------+---------------+------------+--------------+
| UM_ID | UM_ATTR_NAME         | UM_ATTR_VALUE   | UM_PROFILE_ID | UM_USER_ID | UM_TENANT_ID |
+-------+----------------------+-----------------+---------------+------------+--------------+
|     1 | im                   |                 | default       |          1 |        -1234 |
|     2 | region               | Western         | default       |          1 |        -1234 |
|     3 | streetAddress        |                 | default       |          1 |        -1234 |
|     4 | country              |                 | default       |          1 |        -1234 |
|     5 | mobile               |                 | default       |          1 |        -1234 |
|     6 | sn                   | NewLastname     | default       |          1 |        -1234 |
|     7 | profileConfiguration | default         | default       |          1 |        -1234 |
|     8 | dateOfBirth          |                 | default       |          1 |        -1234 |
|     9 | mail                 | [hidden email] | default       |          1 |        -1234 |
|    10 | organizationName     | WSO2            | default       |          1 |        -1234 |
|    11 | givenName            | NewUser         | default       |          1 |        -1234 |
|    12 | province             | western         | default       |          1 |        -1234 |
+-------+----------------------+-----------------+---------------+------------+--------------+

Performance wise this is not a good design. For mongodb userstore, I suggested to add a new document in UM_USER_ATTRIBUTE collection for each user. If the attribute value is empty in the profile, an empty string can be stored.

{

   "im": "",

   " region": "Western",

   "street": "Address",

   "country": "",

   "mobile": "",

   "sn": "NewLastname",

   "profileConfiguration": "default",

   "dateOfBirth": "",

   "mail": "[hidden email]",

   "organizationName": "WSO2",

   "givenName": "NewUser",

   "province": "western"

}

3. Profile saving is having some issues currently and we need to further investigate what is going wrong. Until the issue is figured out, asked to manually create json documents in UM_USER_ATTRIBUTE collection and implement the retrieval of user attributes. (can test getUserClaimValues method in the admin service using SOAP UI)

4. For the analytics part of the project, suggested to extend the AbstractUserOperationEventListener class [1] and override the methods for publishing events.

5. Developer documentation, Administration Guide and Testing Guide (sample SOAP requests and responses in RemoteUserStoreManagerService API) should be written as deliverables. For all actions related to the mongodb userstore manager (i.e add user, delete user, add role ...) , Selenium scripts should be provided (can use firefox selenium addon and record each operation and provide the scripts).

So far the progress is satisfactory. Keep on the good work !

[1] https://localhost:9443/services/RemoteUserStoreManagerService?wsdl
[2] http://tharindue.blogspot.com/2015/04/wso2-identity-server-data-dictionary.html
[3] https://github.com/wso2/carbon-kernel/blob/v4.4.3/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/common/AbstractUserOperationEventListener.java

Thank you,

TharinduE

On Sat, Jun 4, 2016 at 9:48 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

[Asantha Thilina]

Hi Tharindu,

i rearranged the UM_USER_ATTRIBUTE collection now when ever user update his profile, claim values will save in the collection the way we discussed in hangout i updated my repo[1] with latest modification

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 8, 2016 at 8:03 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry i forget to attach my github repository here is the repository link i have update the repository with my latest errors fixed
https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 8, 2016 at 8:00 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have fixed the claims issue now it can update and add claims to user but still the db structure is same as in relational structure i have currently mapping it to a way we discussed ,most of the issues being fixed now following tasks can do in now MongoDBUserStoreManager

can add users.

can add roles.

can assign roles to users.

can assign users to roles.

can edit user claim values in update profile.

I creating the documentation of my MongoDBUserStoreManager as well sooner i will publish the document and also test scripts as well

And also following are the issues now i facing in

registered users in my user store domain can't login to the IS it's giving error as MongoDBUserStoreManager can't cast to AbstractUserStoreManager since i done MongoDBUserStoreManager by implementing the UserStoreManager is there any special changes to do in my code to authenticate user ? i debugged and see my doAuthenticate method not get triggered during runtime

expect some advice from you

Thanks,

Asantha

On Mon, Jun 6, 2016 at 1:18 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

ok sure i will modify the readme content as appropriately,at the moment i didn't add that feature, set to all default i will modify that to add a port number also,

Thanks,

Asantha

On Mon, Jun 6, 2016 at 12:04 AM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

We might not need implementing the HybridRoleManager... I'll provide you the exact details after trying out the functionality you have implemented up to now.

Can you please write a basic document with the steps for creating the mongodb database and adding a userstore in IS pointing to the created mongodb database and include in the ReadMe of github repo.

Also if mongodb server is running in a different port other than the default port, can we still connect to that ? I didn't see a place to define the port in the configuration.. should it be added as a connection string ? (didn't go through the code to find what is going wrong)

Thanks,

TharinduE

On Sun, Jun 5, 2016 at 11:55 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

As we discussed on hangout i doing the changes in UM_USER_ATTRIBUTE collection and also changing the hybrid role to master datasource but when i try to add user to hybrid role i facing a small issue i can't grab the datasource ,in HybridRoleManager class it need to give java.sql.DataSource but in my current user store manager class i have never used a sql datasource(since it nosql) so i think i need to grab the master datasource of IS which is h2 database datasource, is there a way to get the primary datasource?

Thanks,

Asantha

On Sat, Jun 4, 2016 at 1:59 PM, Tharindu Edirisinghe <[hidden email]> wrote:

The call details are as following.

1. Demonstrated how to use WSO2 admin services.

2. All the user operations in RemoteUserStoreManagerService [1] should be tested (using SOAP UI) for the mongodb userstore.

3. When the user profile is saved in a JDBC userstore, for each attribute of the user, it will add a new entry in UM_USER_ATTRIBUTE (Refer [2] for more information) like below.

+-------+----------------------+-----------------+---------------+------------+--------------+
| UM_ID | UM_ATTR_NAME         | UM_ATTR_VALUE   | UM_PROFILE_ID | UM_USER_ID | UM_TENANT_ID |
+-------+----------------------+-----------------+---------------+------------+--------------+
|     1 | im                   |                 | default       |          1 |        -1234 |
|     2 | region               | Western         | default       |          1 |        -1234 |
|     3 | streetAddress        |                 | default       |          1 |        -1234 |
|     4 | country              |                 | default       |          1 |        -1234 |
|     5 | mobile               |                 | default       |          1 |        -1234 |
|     6 | sn                   | NewLastname     | default       |          1 |        -1234 |
|     7 | profileConfiguration | default         | default       |          1 |        -1234 |
|     8 | dateOfBirth          |                 | default       |          1 |        -1234 |
|     9 | mail                 | [hidden email] | default       |          1 |        -1234 |
|    10 | organizationName     | WSO2            | default       |          1 |        -1234 |
|    11 | givenName            | NewUser         | default       |          1 |        -1234 |
|    12 | province             | western         | default       |          1 |        -1234 |
+-------+----------------------+-----------------+---------------+------------+--------------+

Performance wise this is not a good design. For mongodb userstore, I suggested to add a new document in UM_USER_ATTRIBUTE collection for each user. If the attribute value is empty in the profile, an empty string can be stored.

{

   "im": "",

   " region": "Western",

   "street": "Address",

   "country": "",

   "mobile": "",

   "sn": "NewLastname",

   "profileConfiguration": "default",

   "dateOfBirth": "",

   "mail": "[hidden email]",

   "organizationName": "WSO2",

   "givenName": "NewUser",

   "province": "western"

}

3. Profile saving is having some issues currently and we need to further investigate what is going wrong. Until the issue is figured out, asked to manually create json documents in UM_USER_ATTRIBUTE collection and implement the retrieval of user attributes. (can test getUserClaimValues method in the admin service using SOAP UI)

4. For the analytics part of the project, suggested to extend the AbstractUserOperationEventListener class [1] and override the methods for publishing events.

5. Developer documentation, Administration Guide and Testing Guide (sample SOAP requests and responses in RemoteUserStoreManagerService API) should be written as deliverables. For all actions related to the mongodb userstore manager (i.e add user, delete user, add role ...) , Selenium scripts should be provided (can use firefox selenium addon and record each operation and provide the scripts).

So far the progress is satisfactory. Keep on the good work !

[1] https://localhost:9443/services/RemoteUserStoreManagerService?wsdl
[2] http://tharindue.blogspot.com/2015/04/wso2-identity-server-data-dictionary.html
[3] https://github.com/wso2/carbon-kernel/blob/v4.4.3/core/org.wso2.carbon.user.core/src/main/java/org/wso2/carbon/user/core/common/AbstractUserOperationEventListener.java

Thank you,

TharinduE

On Sat, Jun 4, 2016 at 9:48 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,
 

ok sure i will look forward for that

Thanks,

Asantha

On Fri, Jun 3, 2016 at 2:44 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Shall we have a google hangout tomorrow (Saturday) at 9.00 p.m ? So we can discuss about the issues you are facing and get them resolved.

Regards,

TharinduE

On Fri, Jun 3, 2016 at 11:43 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i fixed the most of the errors appeared in my user store in user management side now it's almost done in user management side now i can add new users ,roles and search roles of users and users of roles only issue i having now is i can't update a user profile of user in user store i getting a exception ,i want to get some advice from you to resolve that error and to implement a logic to commit transaction in mongodb and also another small problem is when i added a new claim where will it save in primary user store ? is there any feature to change the user store  where claims going to save. like it giving an option to select userstore in a dropdown when adding newusers and roles.

all the works i have done so far in my repo[1]

[1] . https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Wed, Jun 1, 2016 at 3:00 AM, Asantha Thilina <[hidden email]> wrote:

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>

Date: Wed, Jun 1, 2016 at 2:59 AM
Subject: Fwd: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: WSO2 Developers' List <[hidden email]>, Tharindu Edirisinghe <[hidden email]>

---------- Forwarded message ----------
From: Asantha Thilina <[hidden email]>
Date: Wed, Jun 1, 2016 at 2:51 AM
Subject: Re: GSOC 2016 - Project 21 : MongoDB Userstore Development
To: Tharindu Edirisinghe <[hidden email]>
Cc: WSO2 Developers' List <[hidden email]>, Lakmal Rupasinghe <[hidden email]>

Hi Tharindu,

I fixed the most of the errors i got now my user store working it is successfully registered and can add users and roles to user store as well still there are some issues in other functions those are query errors i still fixing those as well ,i having some issues  about database transaction point of view and i need some advice from you, in mongodb there is no option to rollback transaction if error occurred how can i manage it? can we arrange a hangout meeting today or any comfortable day for you ? i'm ok with any time
all the updates up to now i pushed to my repository[1]

[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Fri, May 27, 2016 at 1:56 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sure i'll be look forward for that

Thanks,

Asantha

On Thu, May 26, 2016 at 9:37 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Glad to hear the progress... Sure I'll review over the weekend and provide feedback. By next week we'll have a hangout to discuss on how to move forward and the difficulties you have faced.  Keep the momentum going !

Regards,

TharinduE

On Fri, May 27, 2016 at 12:17 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i been able to fix the mongodb user store intitalizing error now it working successfully it will be registered as a secondary user store in IS and when going to add a new users and roles it will show the domain of mongodb userstore still i having small issues in my json query formats for mongodb i fixing those issues at the moment all the progress so far i made updated to my github repository[1] and i recently updated my blog[2] as well,can you review my code and give me a feedback?

[1]https://github.com/asanthamax/mongodbuserstore
[2]http://asanthamax.blogspot.com/

Thanks,

Asantha

On Sun, May 22, 2016 at 3:38 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I have override all the methods in AbstractUserStoreManager in my MongoDBUserStoreManager class after that when i try to add new secondary user store in management console i got following error i attached the screen shot of the error with this mail

i have pushed all the progress upto now to my github repository[1]

can you give me some tips to solve this issue?

any suggestion would be grateful
[1] https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Thu, May 19, 2016 at 11:47 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

sorry for the late reply ,as you advised i remove the scr reference and its corresponding methods from user store now it's works fine but i having another small issue if i wanted to run my mongodb user store as as my primary user store how would i do that?

i added my user store configurations to user-mgt.xml file and comented the ldap userstore properties and i added MongoDBUserStore to tenant-mgt.xml as well but then i get error as datasource cannot be casted to MongoDB db source while deploying thats mean its still referring to Default Activator inside carbon core module how would i set it to my activator to reffer?

any suggestion or help would be grateful

Thanks,

Asantha

On Wed, May 18, 2016 at 10:29 PM, Tharindu Edirisinghe <[hidden email]> wrote:

Hi Asantha,

Great progress so far !

Since the reference is given to realm service in the bundle activator, you should be facing the issue. Can you remove the service reference from the bundle activator class. (You can delete the following from the class).

* @scr.reference name="user.realmservice.default"
 * interface="org.wso2.carbon.user.core.service.RealmService" cardinality="1..1"
 * policy="dynamic" bind="setRealmService"
 * unbind="unsetRealmService"

Also delete the following methods where realm service is used.

getRealmService()
setRealmService(RealmService rlmService)
unsetRealmService(RealmService rlmService)

After that can you try if the userstore manager is working as expected. If you face any issues, please get back.

Thank you,

TharinduE

On Sun, May 15, 2016 at 7:48 PM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

I found the issue after debugging into my code ,my service bundle is not get activated because i referring into mongodb-driver which is not osgi comaptible and i just found another library in here[1] and it is worked for me ,fixed project i pushed into my github repository [2].

i found out that DefaultRealmService which is used in Bundle Activator of JDBCUserstoreManager cannot be used for MongoDB since it used java.sql.DataSource.
Is their having any DataSource class like that for MongoDB? or would i have to create a different logic to implement that scenario in MongoDB?

waiting for your suggestion

[1]. http://central.maven.org/maven2/org/mongodb/mongo-java-driver/3.2.2/
[2]. https://github.com/asanthamax/mongodbuserstore

Thanks,

Asantha

On Sat, May 14, 2016 at 10:53 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

i faced following issue while i developing ,

after adding my new json query support i didn't able to see My MongoDBUserStore in dropdown in create user store option.
for json related activites i used an external json dependency from here[1] and i added it jar to lib folder in IS as well,i have enabled remote debugging and inspect my code and there i found in my MongoDBUserStoreDSComponent  class activate method triggered but execute 1st line of code only. it will not registered the service but in my osgi console window when i run ss mongo command i get monodb.userstore bundle and mongodb java driver bundle are activated successfully i attached my console output and identity server output for your consideration as well

any suggestion would be grateful

[1]. http://mvnrepository.com/artifact/org.json/json/20160212

my github repository
https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

 

On Thu, May 12, 2016 at 2:01 AM, Asantha Thilina <[hidden email]> wrote:

Hi Tharindu,

since mongodb driver not having prepared statement support earlier i developed a logic to support prepared statement but i realized that it has a limitation such as parameters has to be pass as same order it exists in prepared statement then as you suggested to use key value relationship instead of order during our hangout conversation i created a new logic same to prepared statement in sql and it will work according to key value relationship and i created standard query format for mongodb with json to support prepared statements and i rewrite the JDBCRealmConstants class with those supporting json queries instead of sql queries

All of my prepared statement methods contain in MongoDBPreparedStatement interface and its implementation in MongoDBPreparedStatementImpl class

All the changes i have pushed to my repo in here[1]

security of this prepared statements has to be improve 

I'll be grateful if you can provide me some suggestions to improve it and also another small issue i having is can i write a DataSource for MongoDB as in java.sql.DataSource ?

[1] https://github.com/asanthamax/mongodbuserstore/tree/master/mongodb.userstoremanager

Thanks,

Asantha

--

Tharindu Edirisinghe
Software Engineer | WSO2 Inc
Platform Security Team
Blog : tharindue.blogspot.com
mobile : <a href="tel:%2B94%20775181586" value="[phone]" target="_blank">[phone]

--

--

--

--

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev