Cluster setup with load balancer

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Cluster setup with load balancer

Dilusha Alphonso
Hi All,

I tried to configure cluster setup with the load balancer. I configured manager +worker (offset 3) and a worker node( offset 4). I followed the step according to this document [1]. But cluster setup with the load balancer is not working.

 Can anyone please help on this or suggest If I am wrong somewhere.


Thanks
Dilusha
--
Best Regards,
Dilusha Alphonso
Software Engineer - Support Team
WSO2

Mobile : +94766837098
http://wso2.com/signature

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Cluster setup with load balancer

Maneesha Wijesekara
Hi Dilusha,

Have you set hostnames to correct ip addresses in /etc/hosts for product nodes as well as nginx node ? 

Regards
Maneesha

On Tue, Dec 5, 2017 at 9:11 PM, Dilusha Alphonso <[hidden email]> wrote:
Hi All,

I tried to configure cluster setup with the load balancer. I configured manager +worker (offset 3) and a worker node( offset 4). I followed the step according to this document [1]. But cluster setup with the load balancer is not working.

 Can anyone please help on this or suggest If I am wrong somewhere.


Thanks
Dilusha
--
Best Regards,
Dilusha Alphonso
Software Engineer - Support Team
WSO2

Mobile : <a href="tel:+94%2076%20683%207098" value="+94766837098" target="_blank">+94766837098
http://wso2.com/signature

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev




--
Maneesha Wijesekara
Software Engineer - QA Team
WSO2 Inc.

Mobile: <a href="tel:%2B94712443119" value="+94774171471" target="_blank">+94712443119

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Cluster setup with load balancer

Pubudu Gunatilaka-2
Hi Dilusha,

Configure the cluster between gateway nodes according to the guide. If you start both gateway nodes, you should see 'Member joined' message in one of the gateway nodes. 

Actually, you need to configure pass-through transports in the gateway for nginx. The ports are 8280 and 8243. We will update the docs.

As you have one manager node, it is not mandatory to configure servlet ports for gateway nodes. 

Thank you!
-- 
Pubudu Gunatilaka
Committer and PMC Member - Apache Stratos
Senior Software Engineer 
WSO2, Inc.: http://wso2.com
mobile : <a href="tel:%2B94772207163" value="+94772207163" style="font-size:x-small;color:rgb(17,85,204)" target="_blank">+94774078049


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Cluster setup with load balancer

Sashika Wijesinghe
Hi Dilusha,

Make sure to import the SSL certificate into the client-truststore.jks of gateway nodes.

For example: keytool -import -trustcacerts -alias server -file server.crt -keystore client-truststore.jks

Regards,
Sashika



On Tue, Dec 5, 2017 at 11:31 PM, Pubudu Gunatilaka <[hidden email]> wrote:
Hi Dilusha,

Configure the cluster between gateway nodes according to the guide. If you start both gateway nodes, you should see 'Member joined' message in one of the gateway nodes. 

Actually, you need to configure pass-through transports in the gateway for nginx. The ports are 8280 and 8243. We will update the docs.

As you have one manager node, it is not mandatory to configure servlet ports for gateway nodes. 

Thank you!
-- 
Pubudu Gunatilaka
Committer and PMC Member - Apache Stratos
Senior Software Engineer 
WSO2, Inc.: http://wso2.com
mobile : <a href="tel:%2B94772207163" value="+94772207163" style="font-size:x-small;color:rgb(17,85,204)" target="_blank">+94774078049


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev




--
Sashika Wijesinghe
Software Engineer - QA Team

Mobile : +94 (0) 774537487

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Cluster setup with load balancer

Dushantha Batuwita
Hi Dilusha 
please refer to the above mail I have mistakenly sent to Sashika

Regards

On Wed, Dec 6, 2017 at 9:52 AM, Dushantha Batuwita <[hidden email]> wrote:
Hi Dilusha

I hope you have coded the nginx related configuration in the document correctly. There the pitfall is the port setup please check if the ports are assigned correctly, further you need to handle the HTTP / HTTPS request to the nginx. Those configurations are handled by “am.http.conf” and “am.https.conf” files located at /etc/nginx/conf.d/<file_name>.

I suppose your setup is established on the local machine since you have not specifically mentioned it.


Further, I expect you have followed the document from top to bottom and below are places to double check.

/etc/nginx/conf.d/am.http.conf


upstream wso2.am.com {

            server <localhost>:9763 +{portoffset_of_GateWayWorker};       

server <localhost>:9763  +{portoffset_of_GateWayManager};

# this is because your manager is also  a Gateway worker

}


server ;

       listen 80;

       server_name am.wso2.com;

       location / {

              proxy_set_header X-Forwarded-Host $host;

              proxy_set_header X-Forwarded-Server $host;

              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

              proxy_set_header Host $http_host;

              proxy_read_timeout 5m;

              proxy_send_timeout 5m;

              proxy_pass http://wso2.am.com;

       }

}


/etc/nginx/conf.d/am.https.conf

upstream ssl.wso2.am.com {

       server <localhost>:9443+{portoffset_of_GateWayWorker};  

   server <localhost>:9443 + {portoffset_of_GateWayManager};

}


server {

listen 443;

   server_name am.wso2.com;

   ssl on;

   ssl_certificate /etc/nginx/ssl/server.crt;

   ssl_certificate_key /etc/nginx/ssl/server.key;

   location / {

              proxy_set_header X-Forwarded-Host $host;

              proxy_set_header X-Forwarded-Server $host;

              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

              proxy_set_header Host $http_host;

              proxy_read_timeout 5m;

              proxy_send_timeout 5m;

       proxy_pass https://ssl.wso2.am.com;

       }

}


/etc/nginx/conf.d/mgt.am.https.conf


server {

   listen 443;

   server_name mgt.am.wso2.com;

   ssl on;

   ssl_certificate /etc/nginx/ssl/server.crt;

   ssl_certificate_key /etc/nginx/ssl/server.key;


   location / {

              proxy_set_header X-Forwarded-Host $host;

              proxy_set_header X-Forwarded-Server $host;

              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

              proxy_set_header Host $http_host;

              proxy_read_timeout 5m;

              proxy_send_timeout 5m;

       proxy_pass https://<localhost>:9443 +{portoffset_of_GateWayManager};

       }

   error_log  /var/log/nginx/mgt-error.log ;

          access_log  /var/log/nginx/mgt-access.log;

}


Server.crt and server.key are the SSL certificate and the key you have to create by following the exact instruction given in the document you have mentioned and please be aware of the location you have to copy the over to.


Then the next critical part is the configuring the Gateway manager section

There the configuring axis2.xml is worth paying attention.

( I have noted the sections where you might go wrong and hope other sections of the context have been straight forward )

d. Specify the host used to communicate cluster messages. This is the IP of the Gateway manager node.

<parameter name="localMemberHost">localhost</parameter>

F. section asks to set up the parameters

<parameter name="properties">

               <!--<property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/>

               <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/> -->


               <property name="backendServerURL" value="am.wso2.com"/>

<!-- backend server url is what exposes to the Gate way worker nodes and it should be the name of the Load balancer fronting the workers -->

               <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/>

<!-- mgtConsoleURL should be the Gateway mangers url as we are accessing through the gateway managers console -->

               <property name="subDomain" value="worker"/>

       </parameter>



Configuring the Configuring the carbon.xml file of the Gateway Manager

The given ports should be kept as it’s they are automatically adjusted

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"

               port="9763"

               proxyPort="80"

--------

/>

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"

               port="9443"

               proxyPort="443"

--------

/>


Rest of the configuration I suppose straightforward till the configuration of Gateway worker.
But if you have any queries you can raise it as always. If you get the idea of the above setting then configuring Gateway worker is also somewhat similar.
Hope this would help you to set up the Gateway cluster.

Regards

On Wed, Dec 6, 2017 at 8:37 AM, Sashika Wijesinghe <[hidden email]> wrote:
Hi Dilusha,

Make sure to import the SSL certificate into the client-truststore.jks of gateway nodes.

For example: keytool -import -trustcacerts -alias server -file server.crt -keystore client-truststore.jks

Regards,
Sashika



On Tue, Dec 5, 2017 at 11:31 PM, Pubudu Gunatilaka <[hidden email]> wrote:
Hi Dilusha,

Configure the cluster between gateway nodes according to the guide. If you start both gateway nodes, you should see 'Member joined' message in one of the gateway nodes. 

Actually, you need to configure pass-through transports in the gateway for nginx. The ports are 8280 and 8243. We will update the docs.

As you have one manager node, it is not mandatory to configure servlet ports for gateway nodes. 

Thank you!
-- 
Pubudu Gunatilaka
Committer and PMC Member - Apache Stratos
Senior Software Engineer 
WSO2, Inc.: http://wso2.com
mobile : <a href="tel:%2B94772207163" value="+94772207163" style="font-size:x-small;color:rgb(17,85,204)" target="_blank">+94774078049


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev




--
Sashika Wijesinghe
Software Engineer - QA Team

Mobile : +94 (0) 774537487

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev




--
Dushantha Nayanajith Chandima Batuwita
Software Engineer Support | WSO2
[hidden email]
mobile : 0094777453010



--
Dushantha Nayanajith Chandima Batuwita
Software Engineer Support | WSO2
[hidden email]
mobile : 0094777453010

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev
Reply | Threaded
Open this post in threaded view
|

Re: Cluster setup with load balancer

Dilusha Alphonso
Hi All,

My Configuration is working with the load balancer.  Thank you for your support.

Thanks
Dilusha





On Wed, Dec 6, 2017 at 9:55 AM, Dushantha Batuwita <[hidden email]> wrote:
Hi Dilusha 
please refer to the above mail I have mistakenly sent to Sashika

Regards

On Wed, Dec 6, 2017 at 9:52 AM, Dushantha Batuwita <[hidden email]> wrote:
Hi Dilusha

I hope you have coded the nginx related configuration in the document correctly. There the pitfall is the port setup please check if the ports are assigned correctly, further you need to handle the HTTP / HTTPS request to the nginx. Those configurations are handled by “am.http.conf” and “am.https.conf” files located at /etc/nginx/conf.d/<file_name>.

I suppose your setup is established on the local machine since you have not specifically mentioned it.


Further, I expect you have followed the document from top to bottom and below are places to double check.

/etc/nginx/conf.d/am.http.conf


upstream wso2.am.com {

            server <localhost>:9763 +{portoffset_of_GateWayWorker};       

server <localhost>:9763  +{portoffset_of_GateWayManager};

# this is because your manager is also  a Gateway worker

}


server ;

       listen 80;

       server_name am.wso2.com;

       location / {

              proxy_set_header X-Forwarded-Host $host;

              proxy_set_header X-Forwarded-Server $host;

              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

              proxy_set_header Host $http_host;

              proxy_read_timeout 5m;

              proxy_send_timeout 5m;

              proxy_pass http://wso2.am.com;

       }

}


/etc/nginx/conf.d/am.https.conf

upstream ssl.wso2.am.com {

       server <localhost>:9443+{portoffset_of_GateWayWorker};  

   server <localhost>:9443 + {portoffset_of_GateWayManager};

}


server {

listen 443;

   server_name am.wso2.com;

   ssl on;

   ssl_certificate /etc/nginx/ssl/server.crt;

   ssl_certificate_key /etc/nginx/ssl/server.key;

   location / {

              proxy_set_header X-Forwarded-Host $host;

              proxy_set_header X-Forwarded-Server $host;

              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

              proxy_set_header Host $http_host;

              proxy_read_timeout 5m;

              proxy_send_timeout 5m;

       proxy_pass https://ssl.wso2.am.com;

       }

}


/etc/nginx/conf.d/mgt.am.https.conf


server {

   listen 443;

   server_name mgt.am.wso2.com;

   ssl on;

   ssl_certificate /etc/nginx/ssl/server.crt;

   ssl_certificate_key /etc/nginx/ssl/server.key;


   location / {

              proxy_set_header X-Forwarded-Host $host;

              proxy_set_header X-Forwarded-Server $host;

              proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

              proxy_set_header Host $http_host;

              proxy_read_timeout 5m;

              proxy_send_timeout 5m;

       proxy_pass https://<localhost>:9443 +{portoffset_of_GateWayManager};

       }

   error_log  /var/log/nginx/mgt-error.log ;

          access_log  /var/log/nginx/mgt-access.log;

}


Server.crt and server.key are the SSL certificate and the key you have to create by following the exact instruction given in the document you have mentioned and please be aware of the location you have to copy the over to.


Then the next critical part is the configuring the Gateway manager section

There the configuring axis2.xml is worth paying attention.

( I have noted the sections where you might go wrong and hope other sections of the context have been straight forward )

d. Specify the host used to communicate cluster messages. This is the IP of the Gateway manager node.

<parameter name="localMemberHost">localhost</parameter>

F. section asks to set up the parameters

<parameter name="properties">

               <!--<property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/>

               <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/> -->


               <property name="backendServerURL" value="am.wso2.com"/>

<!-- backend server url is what exposes to the Gate way worker nodes and it should be the name of the Load balancer fronting the workers -->

               <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/>

<!-- mgtConsoleURL should be the Gateway mangers url as we are accessing through the gateway managers console -->

               <property name="subDomain" value="worker"/>

       </parameter>



Configuring the Configuring the carbon.xml file of the Gateway Manager

The given ports should be kept as it’s they are automatically adjusted

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"

               port="9763"

               proxyPort="80"

--------

/>

<Connector  protocol="org.apache.coyote.http11.Http11NioProtocol"

               port="9443"

               proxyPort="443"

--------

/>


Rest of the configuration I suppose straightforward till the configuration of Gateway worker.
But if you have any queries you can raise it as always. If you get the idea of the above setting then configuring Gateway worker is also somewhat similar.
Hope this would help you to set up the Gateway cluster.

Regards

On Wed, Dec 6, 2017 at 8:37 AM, Sashika Wijesinghe <[hidden email]> wrote:
Hi Dilusha,

Make sure to import the SSL certificate into the client-truststore.jks of gateway nodes.

For example: keytool -import -trustcacerts -alias server -file server.crt -keystore client-truststore.jks

Regards,
Sashika



On Tue, Dec 5, 2017 at 11:31 PM, Pubudu Gunatilaka <[hidden email]> wrote:
Hi Dilusha,

Configure the cluster between gateway nodes according to the guide. If you start both gateway nodes, you should see 'Member joined' message in one of the gateway nodes. 

Actually, you need to configure pass-through transports in the gateway for nginx. The ports are 8280 and 8243. We will update the docs.

As you have one manager node, it is not mandatory to configure servlet ports for gateway nodes. 

Thank you!
-- 
Pubudu Gunatilaka
Committer and PMC Member - Apache Stratos
Senior Software Engineer 
WSO2, Inc.: http://wso2.com
mobile : <a href="tel:%2B94772207163" value="+94772207163" style="font-size:x-small;color:rgb(17,85,204)" target="_blank">+94774078049


_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev




--
Sashika Wijesinghe
Software Engineer - QA Team

Mobile : +94 (0) 774537487

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev




--
Dushantha Nayanajith Chandima Batuwita
Software Engineer Support | WSO2
[hidden email]
mobile : 0094777453010



--
Dushantha Nayanajith Chandima Batuwita
Software Engineer Support | WSO2
[hidden email]
mobile : 0094777453010



--
Best Regards,
Dilusha Alphonso
Software Engineer - Support Team
WSO2

Mobile : +94766837098
http://wso2.com/signature

_______________________________________________
Dev mailing list
[hidden email]
http://wso2.org/cgi-bin/mailman/listinfo/dev