[CEP] GSoC Project - Siddhi Extension Doc Auto Generation

classic Classic list List threaded Threaded
56 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
  2. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
  3. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Sriskandarajah Suhothayan


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?


Thank you.

On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Sriskandarajah Suhothayan


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nirmal Fernando-3
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: +94715779733
Blog: http://nirmalfdo.blogspot.com/



_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Sriskandarajah Suhothayan
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Fri, May 5, 2017 at 2:12 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Sorry. My mistake. I will resend.

Thank you for pointing out.

Nadun De Silva.

On Fri, May 5, 2017 at 2:07 PM, Nirmal Fernando <[hidden email]> wrote:
Congratulations! Please loop [hidden email] always. 

On Fri, May 5, 2017 at 2:05 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Mon, Apr 3, 2017 at 8:49 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have submitted the final proposal via the GSoC web portal. Thank you very much for all the support given.

On Sat, Apr 1, 2017 at 11:02 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the updates. I will make the necessary changes and submit.

On Fri, Mar 31, 2017 at 2:52 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Sriskandarajah Suhothayan
Hi Nadun, 

Sorry for the late reply 
please find the new @Extension annotation update here, 

This is due such that extension will contain all information to generate the documentation. The changes are not committed to siddhi master.

Can you implement HTML based documentation based on this and share us the output as the first task. 
Currently, you can only try doc gen on siddhi core bundle, developers will fix for the other extensions in following next two weeks. 

We can then work on the improvements on that. 
Let me know if you have any issues 

Regards
Suho

On Fri, May 5, 2017 at 2:14 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Fri, May 5, 2017 at 2:12 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Sorry. My mistake. I will resend.

Thank you for pointing out.

Nadun De Silva.

On Fri, May 5, 2017 at 2:07 PM, Nirmal Fernando <[hidden email]> wrote:
Congratulations! Please loop [hidden email] always. 

On Fri, May 5, 2017 at 2:05 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Mon, Apr 3, 2017 at 8:49 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have submitted the final proposal via the GSoC web portal. Thank you very much for all the support given.

On Sat, Apr 1, 2017 at 11:02 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the updates. I will make the necessary changes and submit.

On Fri, Mar 31, 2017 at 2:52 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
In reply to this post by Nadun De Silva
Hi,

As I mentioned before I have already implemented the core documentation generation aspects like reading annotated metadata and generating simple HTML and markdown files. But there are few things I need to clarify before I proceed further,
  • Templating Engine -
      • Apache FreeMarker [1] -
Pros -
          • Since this is a Java library, we can run it as part of the documentation generation process. Therefore we can generate any type of file.
Cons -
          • This library is still in the Apache incubator.
      • Handlebars JS [2] (Was recommended to me) -
Pros -
          • This is a more mature library.
          • The documentation combination will be much easier since we can simply merge the JSON files. (I am storing the metadata after the documentation generation process in JSON files for this approach.)
Cons -
          • Since this is a JavaScript library, we can use this for HTML. (However, we won't be able to generate markdown with this which was mentioned as one of the requirements.)

I have implemented both methods for now. Should I keep on supporting both methods?
  • Documentation Combining -
Since all the extensions are in different repositories I don't think we can do this with the Maven plugin. In the proposal, I suggested to use a separate server which could listen for GitHub push events and carry out the combination process. Should we stick with this or explore another method for the combination?
  • Extension Store -
Are there any special requirements of the extensions store that should be considered while implementing the documentation generation or the combination? If so how can I get to know those requirements?

Maybe we can clarify these matters during the community bonding period. (Especially documentation combination and the requirements of the extension store)

[2] http://handlebarsjs.com/

Regards,
Nadun De Silva

On Fri, May 5, 2017 at 2:14 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Fri, May 5, 2017 at 2:12 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Sorry. My mistake. I will resend.

Thank you for pointing out.

Nadun De Silva.

On Fri, May 5, 2017 at 2:07 PM, Nirmal Fernando <[hidden email]> wrote:
Congratulations! Please loop [hidden email] always. 

On Fri, May 5, 2017 at 2:05 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Mon, Apr 3, 2017 at 8:49 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have submitted the final proposal via the GSoC web portal. Thank you very much for all the support given.

On Sat, Apr 1, 2017 at 11:02 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the updates. I will make the necessary changes and submit.

On Fri, Mar 31, 2017 at 2:52 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nirmal Fernando-3


On Tue, May 9, 2017 at 6:52 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

As I mentioned before I have already implemented the core documentation generation aspects like reading annotated metadata and generating simple HTML and markdown files. But there are few things I need to clarify before I proceed further,
  • Templating Engine -
      • Apache FreeMarker [1] -
Pros -
          • Since this is a Java library, we can run it as part of the documentation generation process. Therefore we can generate any type of file.
Cons -
          • This library is still in the Apache incubator.
      • Handlebars JS [2] (Was recommended to me) -
Pros -
          • This is a more mature library.
          • The documentation combination will be much easier since we can simply merge the JSON files. (I am storing the metadata after the documentation generation process in JSON files for this approach.)
Cons -
          • Since this is a JavaScript library, we can use this for HTML. (However, we won't be able to generate markdown with this which was mentioned as one of the requirements.)


I have implemented both methods for now. Should I keep on supporting both methods?
  • Documentation Combining -
Since all the extensions are in different repositories I don't think we can do this with the Maven plugin. In the proposal, I suggested to use a separate server which could listen for GitHub push events and carry out the combination process. Should we stick with this or explore another method for the combination?
  • Extension Store -
Are there any special requirements of the extensions store that should be considered while implementing the documentation generation or the combination? If so how can I get to know those requirements?

Maybe we can clarify these matters during the community bonding period. (Especially documentation combination and the requirements of the extension store)

[2] http://handlebarsjs.com/

Regards,
Nadun De Silva

On Fri, May 5, 2017 at 2:14 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Fri, May 5, 2017 at 2:12 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Sorry. My mistake. I will resend.

Thank you for pointing out.

Nadun De Silva.

On Fri, May 5, 2017 at 2:07 PM, Nirmal Fernando <[hidden email]> wrote:
Congratulations! Please loop [hidden email] always. 

On Fri, May 5, 2017 at 2:05 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Mon, Apr 3, 2017 at 8:49 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have submitted the final proposal via the GSoC web portal. Thank you very much for all the support given.

On Sat, Apr 1, 2017 at 11:02 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the updates. I will make the necessary changes and submit.

On Fri, Mar 31, 2017 at 2:52 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: +94715779733
Blog: http://nirmalfdo.blogspot.com/



_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

I researched on that library [1]. That actually enables us to render markdown in a website using Handlebars. I think it might not work in my case since, if I am not mistaken, the siddhi doc auto generator Java program needs to output a markdown file which will be rendered by another markdown renderer. (At this point, I am assuming that I do not have any control over the markdown renderer.)

Markdown notation does support HTML tags. But most markdown renderers such as the one used by GitHub blocks markdown from loading scripts.


Regards,
Nadun De Silva

On Tue, May 9, 2017 at 7:28 PM, Nirmal Fernando <[hidden email]> wrote:


On Tue, May 9, 2017 at 6:52 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

As I mentioned before I have already implemented the core documentation generation aspects like reading annotated metadata and generating simple HTML and markdown files. But there are few things I need to clarify before I proceed further,
  • Templating Engine -
      • Apache FreeMarker [1] -
Pros -
          • Since this is a Java library, we can run it as part of the documentation generation process. Therefore we can generate any type of file.
Cons -
          • This library is still in the Apache incubator.
      • Handlebars JS [2] (Was recommended to me) -
Pros -
          • This is a more mature library.
          • The documentation combination will be much easier since we can simply merge the JSON files. (I am storing the metadata after the documentation generation process in JSON files for this approach.)
Cons -
          • Since this is a JavaScript library, we can use this for HTML. (However, we won't be able to generate markdown with this which was mentioned as one of the requirements.)


I have implemented both methods for now. Should I keep on supporting both methods?
  • Documentation Combining -
Since all the extensions are in different repositories I don't think we can do this with the Maven plugin. In the proposal, I suggested to use a separate server which could listen for GitHub push events and carry out the combination process. Should we stick with this or explore another method for the combination?
  • Extension Store -
Are there any special requirements of the extensions store that should be considered while implementing the documentation generation or the combination? If so how can I get to know those requirements?

Maybe we can clarify these matters during the community bonding period. (Especially documentation combination and the requirements of the extension store)

[2] http://handlebarsjs.com/

Regards,
Nadun De Silva

On Fri, May 5, 2017 at 2:14 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Fri, May 5, 2017 at 2:12 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Sorry. My mistake. I will resend.

Thank you for pointing out.

Nadun De Silva.

On Fri, May 5, 2017 at 2:07 PM, Nirmal Fernando <[hidden email]> wrote:
Congratulations! Please loop [hidden email] always. 

On Fri, May 5, 2017 at 2:05 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Mon, Apr 3, 2017 at 8:49 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have submitted the final proposal via the GSoC web portal. Thank you very much for all the support given.

On Sat, Apr 1, 2017 at 11:02 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the updates. I will make the necessary changes and submit.

On Fri, Mar 31, 2017 at 2:52 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Sriskandarajah Suhothayan
Please look at https://mkdocs.readthedocs.io/en/stable/ as in WSO2 we are using this for Ballerina docs so there is a high change we will also end up using this. 
Can we create MD files that support this? 

Regards
Suho

On Fri, May 12, 2017 at 9:08 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

I researched on that library [1]. That actually enables us to render markdown in a website using Handlebars. I think it might not work in my case since, if I am not mistaken, the siddhi doc auto generator Java program needs to output a markdown file which will be rendered by another markdown renderer. (At this point, I am assuming that I do not have any control over the markdown renderer.)

Markdown notation does support HTML tags. But most markdown renderers such as the one used by GitHub blocks markdown from loading scripts.


Regards,
Nadun De Silva

On Tue, May 9, 2017 at 7:28 PM, Nirmal Fernando <[hidden email]> wrote:


On Tue, May 9, 2017 at 6:52 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

As I mentioned before I have already implemented the core documentation generation aspects like reading annotated metadata and generating simple HTML and markdown files. But there are few things I need to clarify before I proceed further,
  • Templating Engine -
      • Apache FreeMarker [1] -
Pros -
          • Since this is a Java library, we can run it as part of the documentation generation process. Therefore we can generate any type of file.
Cons -
          • This library is still in the Apache incubator.
      • Handlebars JS [2] (Was recommended to me) -
Pros -
          • This is a more mature library.
          • The documentation combination will be much easier since we can simply merge the JSON files. (I am storing the metadata after the documentation generation process in JSON files for this approach.)
Cons -
          • Since this is a JavaScript library, we can use this for HTML. (However, we won't be able to generate markdown with this which was mentioned as one of the requirements.)


I have implemented both methods for now. Should I keep on supporting both methods?
  • Documentation Combining -
Since all the extensions are in different repositories I don't think we can do this with the Maven plugin. In the proposal, I suggested to use a separate server which could listen for GitHub push events and carry out the combination process. Should we stick with this or explore another method for the combination?
  • Extension Store -
Are there any special requirements of the extensions store that should be considered while implementing the documentation generation or the combination? If so how can I get to know those requirements?

Maybe we can clarify these matters during the community bonding period. (Especially documentation combination and the requirements of the extension store)

[2] http://handlebarsjs.com/

Regards,
Nadun De Silva

On Fri, May 5, 2017 at 2:14 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Fri, May 5, 2017 at 2:12 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Sorry. My mistake. I will resend.

Thank you for pointing out.

Nadun De Silva.

On Fri, May 5, 2017 at 2:07 PM, Nirmal Fernando <[hidden email]> wrote:
Congratulations! Please loop [hidden email] always. 

On Fri, May 5, 2017 at 2:05 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Mon, Apr 3, 2017 at 8:49 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have submitted the final proposal via the GSoC web portal. Thank you very much for all the support given.

On Sat, Apr 1, 2017 at 11:02 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the updates. I will make the necessary changes and submit.

On Fri, Mar 31, 2017 at 2:52 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Nadun De Silva
Hi,

Yes, the documentation I generated worked fine on mkdocs. For now, I manually copied the files and edited the configuration file. I think we can automate it as well.

I have included the markdown files generated, as well as screenshots of the rendered markdown files on mkdocs.

Please note that some of the content in the documentation are dummy data used for getting a preview.

Regards,
Nadun De Silva

On Fri, May 12, 2017 at 10:15 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:
Please look at https://mkdocs.readthedocs.io/en/stable/ as in WSO2 we are using this for Ballerina docs so there is a high change we will also end up using this. 
Can we create MD files that support this? 

Regards
Suho

On Fri, May 12, 2017 at 9:08 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

I researched on that library [1]. That actually enables us to render markdown in a website using Handlebars. I think it might not work in my case since, if I am not mistaken, the siddhi doc auto generator Java program needs to output a markdown file which will be rendered by another markdown renderer. (At this point, I am assuming that I do not have any control over the markdown renderer.)

Markdown notation does support HTML tags. But most markdown renderers such as the one used by GitHub blocks markdown from loading scripts.


Regards,
Nadun De Silva

On Tue, May 9, 2017 at 7:28 PM, Nirmal Fernando <[hidden email]> wrote:


On Tue, May 9, 2017 at 6:52 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

As I mentioned before I have already implemented the core documentation generation aspects like reading annotated metadata and generating simple HTML and markdown files. But there are few things I need to clarify before I proceed further,
  • Templating Engine -
      • Apache FreeMarker [1] -
Pros -
          • Since this is a Java library, we can run it as part of the documentation generation process. Therefore we can generate any type of file.
Cons -
          • This library is still in the Apache incubator.
      • Handlebars JS [2] (Was recommended to me) -
Pros -
          • This is a more mature library.
          • The documentation combination will be much easier since we can simply merge the JSON files. (I am storing the metadata after the documentation generation process in JSON files for this approach.)
Cons -
          • Since this is a JavaScript library, we can use this for HTML. (However, we won't be able to generate markdown with this which was mentioned as one of the requirements.)


I have implemented both methods for now. Should I keep on supporting both methods?
  • Documentation Combining -
Since all the extensions are in different repositories I don't think we can do this with the Maven plugin. In the proposal, I suggested to use a separate server which could listen for GitHub push events and carry out the combination process. Should we stick with this or explore another method for the combination?
  • Extension Store -
Are there any special requirements of the extensions store that should be considered while implementing the documentation generation or the combination? If so how can I get to know those requirements?

Maybe we can clarify these matters during the community bonding period. (Especially documentation combination and the requirements of the extension store)

[2] http://handlebarsjs.com/

Regards,
Nadun De Silva

On Fri, May 5, 2017 at 2:14 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Fri, May 5, 2017 at 2:12 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Sorry. My mistake. I will resend.

Thank you for pointing out.

Nadun De Silva.

On Fri, May 5, 2017 at 2:07 PM, Nirmal Fernando <[hidden email]> wrote:
Congratulations! Please loop [hidden email] always. 

On Fri, May 5, 2017 at 2:05 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.


Thanks,
Nadun De Silva

On Mon, Apr 3, 2017 at 8:49 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have submitted the final proposal via the GSoC web portal. Thank you very much for all the support given.

On Sat, Apr 1, 2017 at 11:02 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the updates. I will make the necessary changes and submit.

On Fri, Mar 31, 2017 at 2:52 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:
I have given some update in the proposal please fix and submit. 

On Mon, Mar 27, 2017 at 4:24 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I have prepared a proposal and I have shared it with WSO2 via the GSoC web portal.

In the proposal, I have not gone into detail about the design and implementation of the "combination of documentation" part (deliverable 3) since we have not completely finalised details on how we should do that. Would that be enough for the proposal?

Please let me know if there are any changes required in the proposal. I am hoping to submit the final proposal based any suggestions received from you.

Thank you.

On Fri, Mar 17, 2017 at 7:09 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

For HTML pages we can use Handlebars JS. I'm not sure if we can do so for MD files.

I will research further and try out both template engines.

Thank you.

On Fri, Mar 17, 2017 at 5:48 PM, Nirmal Fernando <[hidden email]> wrote:
You can try handlebars JS as well. https://github.com/wycats/handlebars.js/

On Fri, Mar 17, 2017 at 12:44 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the information.

I have started to create a prototype for this project. My approach is as follows,
  • Loading the Siddhi extension classes using ClassIndex library used by siddhi annotations.
  • Generating a simple HTML skeleton using Apache FreeMarker [1] as the template engine.
  • Running the above logic from a Mojo.
Please correct me if my approach is not what is expected. I have pushed the above implementation to a GitHub repository. [2]

I will also start working on the proposal.


Thank you.

On Wed, Mar 15, 2017 at 1:00 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Tue, Mar 14, 2017 at 7:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for the response. I have researched a bit more and have few more follow-up questions

It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.

Then I think when the Mojo is executed, I should create the HTML and MD files inside the maven project so that with each push, the GitHub repos are updated. Is this what is expected?
Yes 

Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.

Since by the time the combination happens, all the documentation is in GitHub repos, my suggestion would be to get the content of the generated HTML files using the GitHub contents API [1] and then combine them. But this will be a separate program which will be run for the combination alone.

If we decide to use this approach, I would suggest implementing one of the following or any other method for rerunning the combination process.
  • A server listening to GitHub webhooks [2]
  • A scheduled task
What is your opinion about this approach?

We are moving extensions to wso2-extensions repo[5] and going to host all of them in the extension store[6] so each extension will have it's own doc, and siddhi-core might have one with the predefined functions. We might not need to combine multiple repos together at this point, but when there are multiple extsnsions within the same repo they need to be properly organized. If time permits we can explore how we can merge then, but it's not a requirement at this point.

 
I also have a question about the Siddhi annotations. At the moment, all the details are in one annotation called "@Extension" [3] and it does not contain the "return value" of the extension. How can I fetch the return value of functions from the current annotation system?

We have to change the previous implementation to bring all into one annotation as we did some improvements to optimize extension class loading and that needed a single extension annotation.
To identify the return value use the "returnAttributes()":  for functions this will have only the return type and no names, for windows this will return empty, and for stream processors this can have some attributes with names.

Regards
Suho


On Mon, Mar 13, 2017 at 10:04 PM, Sriskandarajah Suhothayan <[hidden email]> wrote:


On Sat, Mar 11, 2017 at 8:45 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

I am an undergraduate at the University of Moratuwa in my final year. I also worked as an intern at WSO2 last year.

I am interested in "Siddhi Extension Doc Auto Generation" GSoC project. I have worked with WSO2 CEP and Siddhi during my internship and I am also familiar with the Siddhi annotations.

I went through the references provided and I would be very grateful if I can get more guidance on how I can learn more details about the project. Some of the questions I have are as follows.
  1. Does the final HTML pages need to be deployed into the current Siddhi documentation and if so is that part of the project scope?
It will not be in the current Siddhi format, we are thinking of a plain HTML and MD files which can be part of Siddhi Docs in Github.  
  1. Does the combined documentation (Deliverable 3) need to be in the same structure the current documentation is in? (If not the combination can maybe be achieved by having separate pages for extension namespaces with proper navigation between them)
Can have multiple pages (one per namespace) and having index and navigation across is good. Please present your suggestion we can discuss and come to a conclusion on this.  
  1. If I understood correctly the project does not cover the inbuilt processors. Please correct me if I'm wrong.
We have now done some improvements, and now the internal functions too support annotations, so they can also be generated with the approach. 

Regards
Suho
 
Thank you.

Nadun De Silva
Undergraduate of Computer Science and Engineering
University of Moratuwa



--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect & Team Lead of WSO2 Complex Event Processor
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead - Analytics Team, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--

Thanks & regards,
Nirmal

Technical Lead, WSO2 Inc.
Mobile: <a href="tel:071%20577%209733" value="+94715779733" target="_blank">+94715779733
Blog: http://nirmalfdo.blogspot.com/





--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]




--
S. Suhothayan
Associate Director / Architect
lean . enterprise . middleware

cell: <a href="tel:077%20975%206757" value="+94779756757" target="_blank">(+94) 779 756 757 | blog: http://suhothayan.blogspot.com/
twitter: http://twitter.com/suhothayan | linked-in: http://lk.linkedin.com/in/suhothayan



--

profile_pic.jpg

Nadun De Silva

Undergraduate of Computer Science and Engineering

University of Moratuwa

GitHub.png LinkedIn.png Facebook.png

Mobile:

(+94) 77 8 222 607

Email:

[hidden email]


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev

core processors sample screenshot.png (113K) Download Attachment
core-documentation.md (61K) Download Attachment
math namespace sample screenshot.png (107K) Download Attachment
math-documentation.md (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [CEP] GSoC Project - Siddhi Extension Doc Auto Generation

Sriskandarajah Suhothayan
For the first cut, it looks good :) 

Next, we need to come up with the format of the docs. 
Can you come up with some formatting for the extensions?
IMO, having them in a tabular format will be better than using bullets.

Share the resulting .md too, we'll finalize the look and feel.

I have created the initial site here

We have to have an index of extensions under http://wso2.github.io/siddhi/extensions/
and then those indexes need to point to the extension docs. 

Regards
Suho









On Fri, May 12, 2017 at 6:38 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Yes, the documentation I generated worked fine on mkdocs. For now, I manually copied the files and edited the configuration file. I think we can automate it as well.

I have included the markdown files generated, as well as screenshots of the rendered markdown files on mkdocs.

Please note that some of the content in the documentation are dummy data used for getting a preview.

Regards,
Nadun De Silva

On Fri, May 12, 2017 at 10:15 AM, Sriskandarajah Suhothayan <[hidden email]> wrote:
Please look at https://mkdocs.readthedocs.io/en/stable/ as in WSO2 we are using this for Ballerina docs so there is a high change we will also end up using this. 
Can we create MD files that support this? 

Regards
Suho

On Fri, May 12, 2017 at 9:08 AM, Nadun De Silva <[hidden email]> wrote:
Hi,

I researched on that library [1]. That actually enables us to render markdown in a website using Handlebars. I think it might not work in my case since, if I am not mistaken, the siddhi doc auto generator Java program needs to output a markdown file which will be rendered by another markdown renderer. (At this point, I am assuming that I do not have any control over the markdown renderer.)

Markdown notation does support HTML tags. But most markdown renderers such as the one used by GitHub blocks markdown from loading scripts.


Regards,
Nadun De Silva

On Tue, May 9, 2017 at 7:28 PM, Nirmal Fernando <[hidden email]> wrote:


On Tue, May 9, 2017 at 6:52 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

As I mentioned before I have already implemented the core documentation generation aspects like reading annotated metadata and generating simple HTML and markdown files. But there are few things I need to clarify before I proceed further,
  • Templating Engine -
      • Apache FreeMarker [1] -
Pros -
          • Since this is a Java library, we can run it as part of the documentation generation process. Therefore we can generate any type of file.
Cons -
          • This library is still in the Apache incubator.
      • Handlebars JS [2] (Was recommended to me) -
Pros -
          • This is a more mature library.
          • The documentation combination will be much easier since we can simply merge the JSON files. (I am storing the metadata after the documentation generation process in JSON files for this approach.)
Cons -
          • Since this is a JavaScript library, we can use this for HTML. (However, we won't be able to generate markdown with this which was mentioned as one of the requirements.)


I have implemented both methods for now. Should I keep on supporting both methods?
  • Documentation Combining -
Since all the extensions are in different repositories I don't think we can do this with the Maven plugin. In the proposal, I suggested to use a separate server which could listen for GitHub push events and carry out the combination process. Should we stick with this or explore another method for the combination?
  • Extension Store -
Are there any special requirements of the extensions store that should be considered while implementing the documentation generation or the combination? If so how can I get to know those requirements?

Maybe we can clarify these matters during the community bonding period. (Especially documentation combination and the requirements of the extension store)

[2] http://handlebarsjs.com/

Regards,
Nadun De Silva

On Fri, May 5, 2017 at 2:14 PM, Nadun De Silva <[hidden email]> wrote:
Hi,

Thank you for accepting my proposal for "Siddhi Extension Doc Auto Generation" GSoC project.

I have already implemented the following in the prototype and it is available in a GitHub repository [1].
  • Retrieving annotated metadata using ClassIndex library
  • Generating documentation using the retrieved metadata
    • Generating HTML and markdown documentation using Apache FreeMarker.
    • Generating HTML documentation file using HandlebarsJS had been implemented as well since it was the recommended templating engine. (However, I don't think we can use this approach for markdown files since Handlebars is a JS library)
As of now, I am saving the generated documentation files inside the "target/" directory and I have only created simple HTML files since I am not sure of the themes that should be used.

Please correct me if my current approach is not what is expected.

Also, I would be grateful if we can elaborate further on our plan for the upcoming months; whether we should stick with my proposal [2] or whether there should be any deviations from it (especially regarding the method for combining multiple documentation into a single documentation. In the proposal, I suggested my initial idea where we could retrieve the documentation files in all the repositories using the GitHub Contents API and combine them using a completely separate program.).

Thank you again for accepting my proposal. Hope to have a great experience in doing the project in the upcoming months.

[1]