0003756: automated deployment - load balancer integration

Viewing Issue Simple Details [ Jump to Notes ]

[ View Advanced ] [ Issue History ] [ Print ]

Category

Severity

Reproducibility

Date Submitted

Last Update

0003756

[Resin]

minor

always

11-12-09 09:37

05-20-11 15:38

Reporter

ferg

View Status

public

Assigned To

alex

Priority

normal

Resolution

fixed

Status

closed

Product Version

4.0.2

Summary

0003756: automated deployment - load balancer integration

Description

((rep by Rob Lockstone, Jonas Kowall)

Easier yet, we call a page for system health, which any loadbalancer can do. That page checks for a couple mountpoints, and that it can connect to the database. When we want to remove something from the LB we just rename that file, makes it very easy to script putting things in an out of the LB.

On Thu, Nov 12, 2009 at 12:14 AM, Rob Lockstone wrote:

    I wrote our deployment system. Our load balancer monitors our servers by sending certain commands to check if the server is responding. If it doesn't respond for 10 seconds, the server is removed from the pool. So our deployment system works by sending a privileged command to each server in each pool telling it to give the "I'm sleeping" response to the load balancer. After the load balancer removes the server, the deployment server (which has already retrieved the files to deploy from our repository) copies the files out and then tells resin to restart (if the particular type of deployment requires a restart, not all do). It then monitors the server to see when it's come back online before moving onto the next sever in the pool.

    The above is a somewhat simplified explanation, but that's the basics. For example, the deployment system is multi-threaded so it can operate on multiple pools at the same time, and even multiple servers in a particular pool depending on the time of day.

    When all is said and done, we can do an entire production deployment to scores of servers in about 10-15 minutes without our users ever knowing that servers have been popping in and out and restarting.

Additional Information

Attached Files

Relationships

Notes
(0005272) alex 05-20-11 15:38	server/6800 resin adds commands to enable and disable servers: see bin/resin.sh help enable bin/resin.sh help disable Command `disable' takes a server out of a load balance pool at resin web-tier. - `enable' - puts it back in.

Issue History
Date Modified	Username	Field	Change
11-12-09 09:37	ferg	New Issue
02-15-11 17:12	ferg	Summary	automated deployment => automated deployment - load balancer integration
05-20-11 15:36	alex	Status	new => assigned
05-20-11 15:36	alex	Assigned To	=> alex
05-20-11 15:38	alex	Status	assigned => closed
05-20-11 15:38	alex	Note Added: 0005272
05-20-11 15:38	alex	Resolution	open => fixed
05-20-11 15:38	alex	Fixed in Version	=> 4.0.19

Mantis 1.0.0rc3[^]

29 total queries executed.
26 unique queries executed.