Archive | Systems Management

20 August 2010 ~ Comments Off

Tunnel all your traffic on “hostile” networks with OpenVPN

I am often on wireless networks that are unsecured ie. either don’t use encryption or if they are I may not trust they will not tamper with my data (you never know). To protect my traffic on such networks I decided to tunnel nearly all my traffic through an OpenVPN server while I’m on such networks. I will [...]

Continue Reading

19 August 2010 ~ Comments Off

Skipping MySQL replication errors

I was talking to my buddy Jeff Buchbinder and he mentioned that he recently added following to mySQL in order to reduce mySQL replication breakages slave-skip-errors=1062,1053,1146,1051,1050 What this does is not stop replication in case following errors are encountered Error: 1050 SQLSTATE: 42S01 (ER_TABLE_EXISTS_ERROR) Message: Table ‘%s’ already exists Error: 1051 SQLSTATE: 42S02 (ER_BAD_TABLE_ERROR) Message: [...]

Continue Reading

12 August 2010 ~ Comments Off

Deployment rollback

This is a question that often comes up in deployment discussion. How do you rollback in case of a “bad” deploy ? Bad deploy can be any of the following Site completely broken Significant performance degradation Key feature(s) broken There are obviously a number of ways to deal with this issue. You could put up [...]

Continue Reading

20 July 2010 ~ Comments Off

Provision to cloud in 5 minutes using fog

Most recently I have been working on disaster recovery project where we are assembling documentation, processes and code to be able to fire up our whole environment in the cloud in case of a major disaster. At Velocity Conference I met Wesley Beary who is the main developer for fog, a Ruby cloud computing library. [...]

Continue Reading

15 July 2010 ~ Comments Off

CouchDB views creation problems

I have had a frustrating time creating views in CouchDB using curl. Executing following command I would get $ curl -s -X PUT -H “text/plain;charset=utf-8″ -d cronview.json http://localhost:5984/cronologger/_design/cronview {“error”:”bad_request”,”reason”:”invalid UTF-8 JSON”} I checked and rechecked JSON, used the same JSON using CouchDB’s Futon to no avail. Finally I found the answer here http://stackoverflow.com/questions/2461798/error-about-invalid-json-with-couchdb-view-but-the-jsons-fine The -d [...]

Continue Reading

16 June 2010 ~ Comments Off

Non-Dell SSDs/drives not supported until Q2 2011

I am writing up this post so perhaps I can save some poor sysadmin from chasing their own tales. If you ever receive following error message using PERC H700 or H800 controllers Jun 15 14:00:17 db07 Server Administrator: Storage Service EventID: 2335  Controller event log: PD 04(e0x20/s4) is not supported:  Controller 0 (PERC H700 Integrated) [...]

Continue Reading

05 June 2010 ~ Comments Off

Beauty of aggregate line graphs

If you saw a graph like this Would it mean anything to you ? First time I was introduced to it I thought they were pointless since you couldn’t really see much. That was until I saw something like this This was was post release. Can you spot something wrong ? Obviously color scheme is [...]

Continue Reading

27 May 2010 ~ Comments Off

Devops homebrew part deux

This is the second part to the devops homebrew post. I forgot couple things in my first post so here are couple other observations Change is an ongoing process All the changes I talked about in the first post took a long time. It took more than a year to get issues assessed, discussed, designed, [...]

Continue Reading

21 April 2010 ~ Comments Off

Tracking web clients in real time

Most recently I have been working on being able to more quickly identify abusers of our service ie. spammers, crawlers etc. We already have a process that rotates web logs on all web servers hourly then processes them extracting per IP access info. On occasion abusers get quite aggressive and cause some of our alarms [...]

Continue Reading