Mantis - Resin
Viewing Issue Advanced Details
3756 minor always 11-12-09 09:37 05-20-11 15:38
ferg  
alex  
normal  
closed 4.0.2  
fixed  
none    
none 4.0.19  
0003756: automated deployment - load balancer integration
((rep by Rob Lockstone, Jonas Kowall)

Easier yet, we call a page for system health, which any loadbalancer can do. That page checks for a couple mountpoints, and that it can connect to the database. When we want to remove something from the LB we just rename that file, makes it very easy to script putting things in an out of the LB.

On Thu, Nov 12, 2009 at 12:14 AM, Rob Lockstone wrote:

    I wrote our deployment system. Our load balancer monitors our servers by sending certain commands to check if the server is responding. If it doesn't respond for 10 seconds, the server is removed from the pool. So our deployment system works by sending a privileged command to each server in each pool telling it to give the "I'm sleeping" response to the load balancer. After the load balancer removes the server, the deployment server (which has already retrieved the files to deploy from our repository) copies the files out and then tells resin to restart (if the particular type of deployment requires a restart, not all do). It then monitors the server to see when it's come back online before moving onto the next sever in the pool.

    The above is a somewhat simplified explanation, but that's the basics. For example, the deployment system is multi-threaded so it can operate on multiple pools at the same time, and even multiple servers in a particular pool depending on the time of day.

    When all is said and done, we can do an entire production deployment to scores of servers in about 10-15 minutes without our users ever knowing that servers have been popping in and out and restarting.


Notes
(0005272)
alex   
05-20-11 15:38   
server/6800
resin adds commands to enable and disable servers: see

bin/resin.sh help enable
bin/resin.sh help disable

Command `disable' takes a server out of a load balance pool at resin web-tier.
- `enable' - puts it back in.