[IoT] Real time communication model for IoT server

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[IoT] Real time communication model for IoT server

Waruna Jayaweera
Hi,
In WSO2 IoT server , user (web browser) and device communication happen via central storage where we store operation responses in database. But for some operations like shell execution, logcat view we need to get data from devices in real time and we may not require to save them in database. 
Problem
With current model client requires to wait until device come back and execute operation where devices periodically provide responses based on polling time. Responses will be saved in database so users required to send database calls repeatedly to retrieve the response.

Solution
We need to have real time communication feature which allows device or client(web ui) to directly communicate with another device and retrieve response real time. MQTT based real time communication is proposed to solve above problem.
Please find the design for real time communication of WSO2 IOT server. Initially I am working on proving remote shell command support.


Following components will be used for real time communication.

Web socket client - Client which has full duplex communication with IoT server for request and responses.
Session Manager - This will be used to manage sessions and operations mapping.
Operation Manager - For each real time operation , we will create operation using existing operation manager.
Feature level Push Notification strategy - For each device type feature,users will able to set which push notifications strategy (transport) to use. There will be new configuration in <device-type>.xml in defining features.
MQTT Push notification strategy - This will be used to send request from client to broker via publishing to topic. This can be replaced by any given transport implementation.
Pull notification Subscriber- This will pull responses coming from broker and push to session manager to filter based on operation mappings.

With this model,Any device type can send real time operation with wss://host/realtime/<device_type>/<device_id> secured web socket endpoint. Payload should include the operation code.
Agent can subscribe to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/request" topic and responses can be send to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/response topic.
Agent will receive payload with operation id and required to publish payload with operation id to mapping the response with session operation.

Let me know your concerns.

Thanks,
Waruna

--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: <a href="tel:+94%2071%20325%205198" value="+94713255198" target="_blank">+94713255198


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: [IoT] Real time communication model for IoT server

Waruna Jayaweera
Adding missing diagram.

Inline image 1

On Tue, Jun 20, 2017 at 12:25 AM, Waruna Jayaweera <[hidden email]> wrote:
Hi,
In WSO2 IoT server , user (web browser) and device communication happen via central storage where we store operation responses in database. But for some operations like shell execution, logcat view we need to get data from devices in real time and we may not require to save them in database. 
Problem
With current model client requires to wait until device come back and execute operation where devices periodically provide responses based on polling time. Responses will be saved in database so users required to send database calls repeatedly to retrieve the response.

Solution
We need to have real time communication feature which allows device or client(web ui) to directly communicate with another device and retrieve response real time. MQTT based real time communication is proposed to solve above problem.
Please find the design for real time communication of WSO2 IOT server. Initially I am working on proving remote shell command support.


Following components will be used for real time communication.

Web socket client - Client which has full duplex communication with IoT server for request and responses.
Session Manager - This will be used to manage sessions and operations mapping.
Operation Manager - For each real time operation , we will create operation using existing operation manager.
Feature level Push Notification strategy - For each device type feature,users will able to set which push notifications strategy (transport) to use. There will be new configuration in <device-type>.xml in defining features.
MQTT Push notification strategy - This will be used to send request from client to broker via publishing to topic. This can be replaced by any given transport implementation.
Pull notification Subscriber- This will pull responses coming from broker and push to session manager to filter based on operation mappings.

With this model,Any device type can send real time operation with wss://host/realtime/<device_type>/<device_id> secured web socket endpoint. Payload should include the operation code.
Agent can subscribe to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/request" topic and responses can be send to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/response topic.
Agent will receive payload with operation id and required to publish payload with operation id to mapping the response with session operation.

Let me know your concerns.

Thanks,
Waruna

--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: <a href="tel:+94%2071%20325%205198" value="+94713255198" target="_blank">+94713255198




--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: +94713255198


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: [IoT] Real time communication model for IoT server

Maninda Edirisooriya
Hi Waruna,

What is that shell execution feature and logcat view?
When that type of command is executed on a very large number of devices, isn't there any possibility of failing the system due to overload as persistence is not there similar to the web browser approach?
What happens to the devices that are not online at the moment as topics in MB does not keep the messages unless they are durable topics?

Thanks.


Maninda Edirisooriya

Senior Software Engineer
WSO2, Inc.
lean.enterprise.middleware.


Blog : http://maninda.blogspot.com/
E-mail : [hidden email]
Skype : @manindae
Twitter : @maninda

On Tue, Jun 20, 2017 at 12:32 AM, Waruna Jayaweera <[hidden email]> wrote:
Adding missing diagram.

Inline image 1

On Tue, Jun 20, 2017 at 12:25 AM, Waruna Jayaweera <[hidden email]> wrote:
Hi,
In WSO2 IoT server , user (web browser) and device communication happen via central storage where we store operation responses in database. But for some operations like shell execution, logcat view we need to get data from devices in real time and we may not require to save them in database. 
Problem
With current model client requires to wait until device come back and execute operation where devices periodically provide responses based on polling time. Responses will be saved in database so users required to send database calls repeatedly to retrieve the response.

Solution
We need to have real time communication feature which allows device or client(web ui) to directly communicate with another device and retrieve response real time. MQTT based real time communication is proposed to solve above problem.
Please find the design for real time communication of WSO2 IOT server. Initially I am working on proving remote shell command support.


Following components will be used for real time communication.

Web socket client - Client which has full duplex communication with IoT server for request and responses.
Session Manager - This will be used to manage sessions and operations mapping.
Operation Manager - For each real time operation , we will create operation using existing operation manager.
Feature level Push Notification strategy - For each device type feature,users will able to set which push notifications strategy (transport) to use. There will be new configuration in <device-type>.xml in defining features.
MQTT Push notification strategy - This will be used to send request from client to broker via publishing to topic. This can be replaced by any given transport implementation.
Pull notification Subscriber- This will pull responses coming from broker and push to session manager to filter based on operation mappings.

With this model,Any device type can send real time operation with wss://host/realtime/<device_type>/<device_id> secured web socket endpoint. Payload should include the operation code.
Agent can subscribe to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/request" topic and responses can be send to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/response topic.
Agent will receive payload with operation id and required to publish payload with operation id to mapping the response with session operation.

Let me know your concerns.

Thanks,
Waruna

--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: <a href="tel:+94%2071%20325%205198" value="+94713255198" target="_blank">+94713255198




--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: <a href="tel:071%20325%205198" value="+94713255198" target="_blank">+94713255198


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture



_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
Reply | Threaded
Open this post in threaded view
|

Re: [IoT] Real time communication model for IoT server

Waruna Jayaweera
Hi Maninda,
Please refer my comments inline

On Tue, Jun 20, 2017 at 9:03 AM, Maninda Edirisooriya <[hidden email]> wrote:
Hi Waruna,

What is that shell execution feature and logcat view?

Users will need to directly access device and execute shell command and get the response in same time. ex. current running processes. Logcat provide stack traces,error message which device throws. This feature will provide real time responses for those shell command requests.
When that type of command is executed on a very large number of devices, isn't there any possibility of failing the system due to overload as persistence is not there similar to the web browser approach?
This realtime operation will execute single device for at given time.  user will have session with device to execute operations and share data.
What happens to the devices that are not online at the moment as topics in MB does not keep the messages unless they are durable topics?
If device not in online message will be dropped and client will be informed with device status. Those operations will no need to retry since we need responses at realtime. Something similar to ssh to computer which is shutdown. 

Thanks.


Maninda Edirisooriya

Senior Software Engineer
WSO2, Inc.
lean.enterprise.middleware.


Blog : http://maninda.blogspot.com/
E-mail : [hidden email]
Skype : @manindae
Twitter : @maninda

On Tue, Jun 20, 2017 at 12:32 AM, Waruna Jayaweera <[hidden email]> wrote:
Adding missing diagram.

Inline image 1

On Tue, Jun 20, 2017 at 12:25 AM, Waruna Jayaweera <[hidden email]> wrote:
Hi,
In WSO2 IoT server , user (web browser) and device communication happen via central storage where we store operation responses in database. But for some operations like shell execution, logcat view we need to get data from devices in real time and we may not require to save them in database. 
Problem
With current model client requires to wait until device come back and execute operation where devices periodically provide responses based on polling time. Responses will be saved in database so users required to send database calls repeatedly to retrieve the response.

Solution
We need to have real time communication feature which allows device or client(web ui) to directly communicate with another device and retrieve response real time. MQTT based real time communication is proposed to solve above problem.
Please find the design for real time communication of WSO2 IOT server. Initially I am working on proving remote shell command support.


Following components will be used for real time communication.

Web socket client - Client which has full duplex communication with IoT server for request and responses.
Session Manager - This will be used to manage sessions and operations mapping.
Operation Manager - For each real time operation , we will create operation using existing operation manager.
Feature level Push Notification strategy - For each device type feature,users will able to set which push notifications strategy (transport) to use. There will be new configuration in <device-type>.xml in defining features.
MQTT Push notification strategy - This will be used to send request from client to broker via publishing to topic. This can be replaced by any given transport implementation.
Pull notification Subscriber- This will pull responses coming from broker and push to session manager to filter based on operation mappings.

With this model,Any device type can send real time operation with wss://host/realtime/<device_type>/<device_id> secured web socket endpoint. Payload should include the operation code.
Agent can subscribe to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/request" topic and responses can be send to "<tenant_domain>/<device_type>/<device_id>/realtime/<operation>/response topic.
Agent will receive payload with operation id and required to publish payload with operation id to mapping the response with session operation.

Let me know your concerns.

Thanks,
Waruna

--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: <a href="tel:+94%2071%20325%205198" value="+94713255198" target="_blank">+94713255198




--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: <a href="tel:071%20325%205198" value="+94713255198" target="_blank">+94713255198


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture





--
Regards,

Waruna Lakshitha Jayaweera
Senior Software Engineer
WSO2 Inc; http://wso2.com
phone: <a href="tel:+94%2071%20325%205198" value="+94713255198" target="_blank">+94713255198


_______________________________________________
Architecture mailing list
[hidden email]
https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture