Mac OS X Lion introduced a new nifty feature called AirDrop. This
allows users on a local network to drag and drop files to each other
with Finder.
While it seems that this would be useful, there are security
implications. After looking through
Google Search Results
on the topic, I found some un-helpful information in a random forum
post (unsurprising). A little more
review of the search results
resulted in finding the actual defaults(1) command to do so:
If you change a class name in your library, do a major version
change! You don’t know who is using your library, even the
undocumented parts.
Background
Recently, we had a post on the
Chef mailing list that using bluepill for
the Chef Server daemon process(es) was broken. Upon further
investigation of the output from
Chef debug output, it appeared to
be an issue with
Bluepill itself, but
after looking into that ticket, the daemons gem had made a change.
Out of curiosity, I was compelled to find out what the source of the
change in the daemons gem was. This lead to a yak shave, as first, I
had to look up the daemons gem on
RubyGems.org to try and find the source. The author of the gem still
uses RubyForge rather than GitHub. That’s fine, but it means I have to
do some link-spelunking to find where the source code lives.
Now I take a look at the change log:
Release 1.1.8: February 7, 2012
rename to daemonization.rb to daemonize.rb (and Daemonization to Daemonize) to ensure compatibility.
Release 1.1.7: February 6, 2012
start_proc: Write out the PID file in the newly created proc to avoid race conditions.
daemonize.rb: remove to simplify licensing (replaced by daemonization.rb).
Release 1.1.6: January 18, 2012
Add the :app_name option for the "call" daemonization mode.
Release 1.1.5: December 19, 2011
Catch the case where the pidfile is empty but not deleted and restart
the app (thanks to Rich Healey)
I then went to the ticket tracker to find out what the source of the
changes might be. Fortunately, there was an open issue that I could
reference.
My question (which I posted to the ticket) is why wouldn’t renaming a
class cause the author to do a new major version? This way other Gems
that rely on this as a dependency could use the paranoia operator,
~> so the broken class name wouldn’t break usage elsewhere.
I’m glad that the daemons gem author did the right thing and yanked
the broken version. Open source worked well here. The process of
finding this was a bit slower than it should have been, and I think
that the bluepill maintainers moved too quickly to “resolve” the
issue, rather than post their concern about the class naming. Kudos to
thuehlinger for fixing his gem, though.
An often over-looked part of Puppet is the fact that you can programmatically
test changes. The Puppet master has a –compile feature, which will work if
you sync the fact YAMLs from your master to the machine you run the –compile
from.
Puppet-cucumber looks to solve this, so I tried to start using it today. I
found that the features I need already work in my massive monolith test
script, and that puppet-cucumber was just not a good fit. Particularly because
it seemed a little obtuse in general, especially if I only wanted to test things
that changes in either one revision or range of revisions. Anyways, this is my
script for testing puppet changes. You will need GNU Parallel, Puppet, and
MySQL, maybe a couple other smaller deps to run it.
My typical usage is:
1
scripts/puppet-test -d -e staging
Before I have deployed. If you look at the script through, it supports revision
ranges and whatnot. -d is just a shortcut to run the tests against the changes
local to you that are not yet deployed.
#!/bin/bash# This script does syntax checks on a working tree of a puppet git repo,# and then utilizes the stored config DB to find a host that belongs to# a changed class, then compile the catalog for that host. Note that# this script is designed to be run in the scripts directory of the puppet# checkout.## In regular mode, your diff is between the current origin. This can be mod-# ified by using the -d argument, which diffs against the deployed revision.## This script can also be used as a git pre-commit hook, by placing it in# your git checkout under .git/hooks/pre-commit, or linking the git pre-# commit file to the script itself.## Note that this script auto-detects whether you are in an svn or git# checkout, and alters its mode of operation depending on the VCS used.## Also note that -r does not currently work right when git sub-modules# are involved. Will eventually be fixed.## Finally, you should make sure you are in the root of the checkout when this# is run otherwise it likely won't work right, esp wrt submodules.#function caterror(){ cat "$1" >&2
}function die(){printf"%s\n""$1"exit$2}function usage(){printf" _ _ _ \n"printf" _ __ _ _ _ __ _ __ ___| |_ | |_ ___ ___| |_ \n"printf"| '_ \| | | | '_ \| '_ \ / _ \ __|____| __/ _ \/ __| __|\n"printf"| |_) | |_| | |_) | |_) | __/ ||_____| || __/\__ \ |_ \n"printf"| .__/ \__,_| .__/| .__/ \___|\__| \__\___||___/\__|\n"printf"|_| |_| |_| \n\n"printf"This script can be used to perform a thorough test of\n"printf"your puppet checkout, including syntax, duplication,\n"printf"and compile-time errors.\n\n"printf"Note that specifying revisions here only applies to the diff.\n"printf"This means that you should not expect this script to check\n"printf"out the given revision to test the actual code from that\n"printf"revision. This type of functionality will be added later.\n\n"printf"Usage:\n"printf" $(basename $0) [-c] [-e environment] [-f] [-F]\n"printf" -d Performs test of deployed rev v. your checkout\n"printf" -e Puppet environment to test against, default production\n"printf" -f Force tests even if there are no changes\n"printf" -F Really force, even for regex node checks\n"printf" -m Sets VCS mode. Can be 'git', 'svn', or 'git-svn'\n"printf" -P Specify the root of your Puppet checkout. Default to ~/working/git/puppet\n"printf" -r Specify revision to diff against, or range of revisions to \n"printf" diff between. Range must be in the form of rev1:rev2. Does not yet\n"printf" work with submodules.\n"printf"\n"exit 0
}check_node_regex=falsecompile_failure_log="$(mktemp /tmp/compile_failure_log.XXXXXX)"deployed="false"environment="production"error_msg=$(mktemp /tmp/error_msg.XXXXXX)force="false"fullforce="false"puppet_masters="puppet.yoursite.com"puppetroot="$HOME/working/git/puppet"storedconfig_password='3287432424jkdhsfkh'syntax_errors=0
vcs=""while getopts de:fFhm:P:r: option; do case"$option" in
d)deployed="true" ;;
e)environment="$OPTARG"if['!' -d "${puppetroot}/${environment}"]; thenprintf"Environment %s does not exist\n.""$environment"exit -1
fi ;;
f)force="true" ;;
F)force="true"fullforce="true" ;;
m)vcs="$OPTARG"if["$vcs" !="git" -a "$vcs" !="svn" -a "$vcs" !="git-svn"]; thenprintf"FATAL: %s is not a supported VCS!\n" >&2
exit -15
fi ;;
P)puppetroot="$OPTARG" ;;
r)rev="$OPTARG" ;;
[?h]) usage
;;
esacdone# Fail if puppetroot is not thereif[ ! -d "$puppetroot"]; thendie "FATAL: Puppet checkout not found at ${puppetroot}" -2
fi# Die if attempt to diff against prod and specify revisionif["$deployed"=="true" -a -n "$rev"]; thendie "-d and -r are mutually exclusive options!""107"fi# Attempt to auto-identify VCS if nothing is passed to -mif[ -z "$vcs"]; then if[ -e "${puppetroot}/.git"]; thenvcs="git"elif[ -e "${puppetroot}/.svn"]; thenvcs="svn"fi if[ -z "$vcs"]; thenprintf"FATAL: Could not determine VCS!\n" >&2
exit -1
elseprintf"Selecting %s as vcs...\n""$vcs"fifi# Move to puppet root and let the games begincd$puppetroot# Determine if given revision or revision range is valid. Die if not.if[ -n "$rev"]; thenexport IFS=:
case"$vcs" in
git)for r in $rev; do if'!' git show --summary ${r} >/dev/null 2>&1; thendie "Git revision ${r} is not valid""69"fi done ;;
git-svn)for r in $rev; do if'!' git svn find-rev r${r} >/dev/null 2>&1; thendie "SVN Revision ${r} does not exit""70"fi done ;;
svn)for r in $rev; do if'!' svn info -r${r} >/dev/null 2>&1; thendie "SVN Revision ${r} is not valid""71"fi done ;;
*) die "Sense is not being made""42" ;;
esacfi# It would be a little weird to try and add conditionals to save some lines# here, so we just do separate behavior for each VCS.case$vcs in
git)current_branch=$(git branch | egrep '^\*' | cut -d' ' -f2)if["$current_branch"=="master"]; thengit pull --all >/dev/null 2>&1
git submodule update --merge >/dev/null 2>&1
if[$? -ne 0 ]; thenprintf"Unspecified error when running git pull and/or submodule update on your checkout!\n" >&2
exit -1
fi fi if["$deployed"="true"]; thenoldrev=$(ssh $(echo$puppet_masters | cut -d' ' -f1)"sudo cat /etc/puppet/REVISION")diffcmd="git diff ${oldrev}"elif[ -n "$rev"]; thendiffcmd="git diff $(echo $rev | tr ':' ' ')"elsediffcmd="git diff origin"fiunset IFS
# Get list of submodulessubmodules=$(git submodule | cut -c2-)if["$submodules"==""]; thenchanges=$(\eval${diffcmd}\ | grep '^+++'\ | grep -v '/dev/null$'\ | cut -d'/' -f2- \ | sort -u \)else if["$deployed"=="true"]; thenchanges="$($puppetroot/scripts/uberdiff.rb "${oldrev}")"elif[ -n "$rev"]; thenchanges="$($puppetroot/scripts/uberdiff.rb \"$(echo $rev | tr ':' ' ')\")"elsechanges="$($puppetroot/scripts/uberdiff.rb HEAD)"fi fi ;;
git-svn)if["$deployed"="true"]; thenoldsvnrev=$(ssh $(echo$puppet_masters | cut -d' ' -f1)"sudo cat /etc/puppet/REVISION")oldrev=$(git svn find-rev r${oldsvnrev})diffcmd="git diff ${oldrev}"elif[ -n "$rev"]; then for r in $rev; dogitrev="$gitrev $(git svn find-rev r${r})"donediffcmd="git diff $gitrev"elsediffcmd="git diff git-svn"fiunset IFS
changes=$(\eval${diffcmd}\ | grep '^+++'\ | grep -v '/dev/null$'\ | cut -d'/' -f2- \ | sort -u \) ;;
svn) svn up >/dev/null 2>&1
if[$? -ne 0 ]; thenprintf"Unspecified error when running svn up on your checkout!\n" >&2
exit -1
fi if["$deployed"=='true']; thenoldrev=$(ssh $(echo$puppet_masters | cut -d' ' -f1)"sudo cat /etc/puppet/REVISION")diffcmd="svn diff -r${oldrev}"elif[ -n "$rev"]; thendiffcmd="svn diff -r${rev}"elsediffcmd="svn diff"fiunset IFS
changes=$(\eval${diffcmd}\ | grep '^+++'\ | cut -d' ' -f2- \ | sed 's/[[:space:]]\+(working copy)//g'\ | sort -u \) ;;
*)printf"FATAL: Pigs have flown\n" >&2
exit -42
;;
esac# If no changes and not in force mode, end.if[ -z "$changes" -a "$force"=="false"]; thenprintf"No changes made.\n" rm -f $error_msgexit 0
fi# If full force mode is on, set check_node_regex to truecheck_node_regex="$fullforce"# If full force is off, but there are node changes, set check_node_regex trueif echo$changes | grep -q 'manifests/nodes'; thencheck_node_regex=truefi# The syntax checks also check for hard tabs in Ruby, ERB, shell, and puppet# code. It is fairly stupid and only uses file extensions to do this.printf"Checking syntax on changes in working tree... "for change in $changes; do case$change in
*.erb)# Check ERB template syntax cat $change\ | erb -P -x -T - \ | ruby -cw 2> $error_msg > /dev/null
if["$?" -ne 0 ]; thenprintf'FAIL!\nERB Parse Failure:\n' >&2
printf"$change: " >&2
caterror $error_msgsyntax_errors=$((syntax_errors +1))fi if cat $change | grep -q ' '; thenprintf'FAIL!\nHard tabs found in %s!\n'"$change" >&2
syntax_errors=$((syntax_errors +1))fi ;;
*.pp)# Check puppet manifest syntax puppet parser validate --color=false --ignoreimport $change > $error_msg 2>&1
if["$?" -ne 0 ]; thenprintf'FAIL!\nPuppet Parse Failure:\n' >&2
printf"$change: " >&2
caterror $error_msgsyntax_errors=$((syntax_errors +1))fi if cat $change | grep -q ' '; thenprintf'FAIL!\nHard tabs found in %s!\n'"$change" >&2
syntax_errors=$((syntax_errors +1))fi ;;
*.rb)# Check Ruby template syntax cat $change\ | ruby -cw > /dev/null
if["$?" -ne 0 ]; thenprintf'FAIL!\nRuby Parse Failure:\n' >&2
printf"$change: " >&2
caterror $error_msgsyntax_errors=$((syntax_errors +1))fi if cat $change | grep -q ' '; thenprintf'FAIL!\nHard tabs found in %s!\n'"$change" >&2
syntax_errors=$((syntax_errors +1))fi ;;
*.sh)# Shell scripts cat $change\ | bash -n
if["$?" -ne 0 ]; thenprintf'FAIL!\nParse Failure:\n' >&2
printf"$change: " >&2
caterror $error_msgsyntax_errors=$((syntax_errors +1))fi if cat $change | grep -q ' '; thenprintf'FAIL!\nHard tabs found in %s!\n'"$change" >&2
syntax_errors=$((syntax_errors +1))fi ;;
esacrm -f $error_msgif["$syntax_errors" -ne 0 ]; thenprintf\'%s syntax or style errors found!\n'\"$syntax_errors" >&2
exit 1
fidoneprintf"SUCCESS!\n"# Now let's check for duplicate defined classesprintf"Checking for duplicate class definitions... "IFS=$'\n'sorted_classlist=($(find $puppetroot/$environment -type f -name '*.pp' -a ! -path $puppetroot/dist\* -exec egrep -h 'class[[:space:]]+([[:alnum:]]|[-_:])+[[:space:]]+(inherits[[:space:]]+([[:alnum:]]|[-_:])+|.*)*{'{}\; | sort))unset IFS
duplicate_classlist=$(\for((i= 0; i < ${#sorted_classlist[*]}; i++)); doecho${sorted_classlist[$i]}done | uniq -d | awk '{ print $2 }'\)if[ -n "$duplicate_classlist"]; thenprintf"FAIL!\n\nIt appears that the following classes have duplicates:\n" >&2
for((i= 0; i < ${#duplicate_classlist[*]}; i++)); doprintf"%s\n""${duplicate_classlist[$i]}" >&2
doneprintf\"Please locate these files using "egrep -R 'class classname {' *" from theroot of the puppet checkout.\n"exit -1
fiprintf'SUCCESS!\n'# Check for duplicate node regexes if nodes files have changed, or full force# mode is on.if$check_node_regex; thenprintf"Checking for duplicate node regexes... "for master in $puppet_masters; donodelist="$nodelist $(\ ssh $master \"\ sudo mysql -s \ -D puppet \ -h $master \ -e \"select name from hosts;\" \ | cat \ \")"done if[ -z "$nodelist"]; thenprintf'FAIL! Unspecified Error retrieving nodes from database!\n' >&2
exit -1
fi# Sort nodes to remove duplicates, which can happen if the databases are# not maintained.sorted_nodelist=$(echo$nodelist | sort -u)for node_manifest in $puppetroot/$environment/manifests/nodes*; donode_regex_list="$node_regex_list $(egrep -h '^node /' $node_manifest | cut -d'/' -f2)"done# This variable is so we can check how many total dupes there wereglobal_duplicates=0
for node in $sorted_nodelist; domatches=""matchcount=0
for node_regex in $node_regex_list; do if[["$node"=~ $node_regex]]; thenmatches="$matches $node_regex"matchcount=$((matchcount+1))fi done if[$matchcount -gt 1 ]; thenglobal_duplicates=$((global_duplicates+1))printf"WARNING: %s matches multiple regexes:\n""$node" >&2
for regex in $matches; doprintf"%s\n""$regex"done fi done if[$global_duplicates -gt 0 ]; thenprintf"FAIL!\n%d duplicate node regexes found.\n"$global_duplicates >&2
exit -2
elseprintf"SUCCESS!\n"fifi# Now we compile for compile-time errors# Cat changes files and look for classesprintf'Checking for compilation errors...\n'# Get changes that are applicable (puppet manifest changes only)pp_changes=$(\for change in $changes; do if[[$change=~ .pp$ ]]; thenprintf"%s ""$change"fi done)if[ -z "${pp_changes}"]; thenprintf"No puppet manifest changes to test for compilation errors, exiting!\n"exit 0
ficlasses=$(\ cat $pp_changes\ | egrep -v '^[[:space:]]+?#'\ | egrep '^[[:space:]]+?class[[:space:]]+([[:alnum:]]|[-_:])+[[:space:]]+(inherits[[:space:]]+([[:alnum:]]|[-_:])+|.*)*({|\()'\ | awk -F' ''{ print $2 }'\)# Should have at least one server for a non-specific compile testservers="puppet.yoursite.com"# Ensure fake ssldir is set upsudo mkdir -p /var/lib/puppet-test-$USERsudo rsync -r --delete $puppetroot/ssl/ /var/lib/puppet-test-$USER/ssl/
sudo chown -R $USER /var/lib/puppet-test-$USER# Now we will find one host for each changed class to add to the list of# servers for testing compiled catalogs.for master in $puppet_masters; do# First we sync facts as they are needed for catalog compilation testing ssh ${master}"sudo mkdir /tmp/puppetyaml-$USER 2>/dev/null; sudo rsync -r /var/lib/puppet/yaml/ /tmp/puppetyaml-$USER/; sudo chmod -R 755 /tmp/puppetyaml-$USER" sudo -E rsync -r ${USER}@${master}:/tmp/puppetyaml-$USER/ /var/lib/puppet-test-$USER/yaml/
ssh ${master}"sudo rm -rf /tmp/puppetyaml-$USER/*"for class in $classes; do# Get host id of a node that includes $classhost_id=$(\ ssh $master"\ sudo mysql -s \ -D puppet \ -h $master \ -e \"select host_id from resources \ where title = '$class' \ and restype = 'class';\" \ | cat \ | head -1 \ ")# Continue with next class if the host_id isn't valid (such as empty)if[[$host_id=~ [[:digit:]]+ ]]; then :
else continue fi# Get host_name from host_idhost_name=$(\ ssh $master"\ sudo mysql -s \ -D puppet \ -e \"select name from hosts where id = '$host_id';\" \ | cat \ ")# If the server is already in the list to compile, don't add.if echo$servers | grep -q $host_name; then continue fiservers="$servers $host_name"donedonesudo parallel -P 4 \"puppet master \ --config_version='cd $HOME/working/git/puppet; git rev-parse HEAD' \ --color=false \ --no-daemonize \ -l console \ --confdir=$puppetroot \ --manifest=$puppetroot/$environment/manifests/site.pp \ --modulepath=$puppetroot/$environment/grumps-modules:$puppetroot/$environment/yoursite-modules \ --ssldir=/var/lib/puppet-test-$USER/ssl \ --vardir=/var/lib/puppet-test-$USER \ --compile {} \ | egrep \"^(err|notice|warning|Fail)\""\ ::: $servers\| tee $compile_failure_logif egrep -q '^(err|Fail)'"$compile_failure_log"; thenprintf'Compilation errors, please fix.\n' >&2
rm -f "$compile_failure_log"exit -1
elseprintf"Changed catalogs compiled successfully.\n" rm -f "$compile_failure_log"exit 0
fi#vim: set expandtab ts=3 sw=3:
If you end up using Git submodules like I have outlined in my previous post,
you will need the uberdiff.rb script:
#!/usr/bin/env ruby# This is a script that takes a revision or revision range from a super-# project, and spits out all the files under submodules that have changed.# Note that this will not output removed files.rev=ARGV[0]submodules=[]ifDir.pwd.split("/")[-1]!="puppet"STDERR.puts"You must run this script from the root of the super-project."exit-1end`git diff #{rev} --submodule='log' | egrep '^Submodule (staging|production)'`.each_linedo|submodule|ifsubmodule.split(" ")[2].gsub(/:$/,"")=="contains"nextendsubmodules<<{:modulename=>submodule.split(" ")[1],:revrange=>submodule.split(" ")[2].gsub(/:$/,""),:changes=>[]}endsubmodules.eachdo|submodule|submodule[:changes]=`git submodule --quiet foreach 'if [ $path == "#{submodule[:modulename]}" ]; then git diff --diff-filter=ACM --name-only #{submodule[:revrange]}; fi'`.split("\n")submodule[:changes].map!{|change|submodule[:modulename]+File::SEPARATOR+change}endsubmodules.eachdo|submodule|putssubmodule[:changes]end#vim: set expandtab ts=3 sw=3:
Often, in Puppet manifests you will find you need some arbitrary information
about machines in your infrastructure. The stored configuration database, for
all its blemishes, can be an easy way to achieve this.
require'puppet/rails'require'puppet/rails/fact_value'modulePuppet::Parser::Functionsnewfunction(:getinfo,:type=>:rvalue,:doc=>"This is a function that will return an array of information based on what you ask for. For example this call: getinfo('Class', 'httpd', 'ec2_public_ipv4') will return a list of the ec2_public_ipv4 fact values for any node that has the resource Class['httpd']. If you leave off the third argument, fqdn will be assumed. Also note you can pass an environment to this function, but likely you won't need to because by default it will find the node's environment and use that. ")do|args|resource_type=args[0]resource_title=args[1]return_info=args[2]environment=args[3]return_info||="fqdn"environment||=lookupvar("::environment")raisePuppet::ParseError,("getinfo(): wrong number of arguments (#{args.length}; must be >= 2)")ifargs.length<2info=[]info=Puppet::Rails::FactValue.find_by_sql(" SELECT fv.value from fact_values fv, fact_names fn, resources r, hosts h WHERE fn.name = '#{return_info}' AND h.environment = '#{environment}' AND h.id = r.host_id AND r.restype = '#{resource_type}' AND r.title = '#{resource_title}' AND fv.host_id = h.id AND fv.fact_name_id = fn.id; ")info.map!{|i|i.value}infoendend
Put this in a module and you can start using it in manifests:
This will make the nodenets parameter to be an array of ec2 public IPs from
any nodes that have the httpd class.
Here is an example of using it in an Apache load balancer config:
12345678
<Location ~ "^/(SYSTEM|system)"> Order allow,deny Allow from 127.0.0.1<%scope.function_getinfo('Class','moodle','ipaddress').eachdo|ip|-%> Allow from <%=ip%><%end-%> Satisfy Any </Location>
The third parameter makes it so you can get any arbitrary information. You
could for instance compile more complex configuration files that use hostname
and something like architecture. This would take two calls to getinfo(), but
I think you get the picture. Eventually I think I’d like to take an array of
resources and write a query builder method to keep the amount of SQL queries
to a minimum. For now this is working fine for me to solve some real problems.
Git submodules are somewhat of an ‘advanced’ git feature, akin to Subversion
externals for those of us unlucky enough to have the pleasure of knowing svn.
The most common usage is pulling in third party libraries to your project. You
can think of git submodules as git checkouts within checkouts. The ‘parent’
checkout or ‘super-project’ as I call it, knows that there are submodules, and
knows which SHA each submodule is at. Since most of my work as both a Systems
Engineer at $dayjob and a general IT consultant part-time, I needed to use
submodules slightly differently than the most common use-case.
During some time off during 2011 from having a dayjob, I developed a lot of
Puppet code that used what was at the time new features like Parameterized
classes, hashes, etc. I also pretty painstakingly made sure the code would
work on OpenBSD, Debian, Ubuntu, Red Hat, and CentOS. The plan was to keep
this code private and charge people for the service of supporting this set of
modules and develop new ones etc. So, I needed a way to let people pull this
code into their Puppet repo and use it. Submodules are the only easy way to do
this. This code, along with some scripts to set up an environment utilizing
this code was what I called the GRand Unified Modular Puppet System (GRUMPS).
First, not to go too far off track, but if you are pulling puppet modules
on the net, you should already be aware of git submodules. If you are down
loading modules then just copying them into your setup you’re doing it wrong.
Puppet code is code. You need to treat improvements to public modules the
same way you treat any open-source project. You have your own local branch,
submit fixes to upstream etc.
Now let’s talk about Puppet environments. You should at the very least have
two environments. The name of the one that isn’t production probably doesn’t
matter much, but we’ll call this staging, since that’s what I use in the code
snippets later on. Many people will make these git checkouts under /etc/puppet
that they run git pull on to update. This is an atrocious update mechanism, and
a poor development layout.
Instead, what I’ve done is to make each sub-directory of each Puppet env correspond
to specific branches in various sub-modules. Example:
“Develop” was a pre-existing convention I had for branch naming, but that’s
really irrelevant, make it what you want. Now this shows I have 8 submodules
for my two environments. I actually have one more for the SSL dir, but that
is off-topic really. Some of the code samples may reference develop or master.
To get all this setup you will need to run git submodule add from the root of
your checkout for each submodule, and you will also need to run git submodule
init, which adds the added submodules to the .gitmodules file. Totally not
confusing :-).
The 1000-foot view of the workflow after setup is complete:
Team members committing like it’s April 29 1992 on develop
Everybody’s seeing each other’s work, reviews it
Merge every sub-module’s develop branch to merge
Update super-project
Push and Deploy
Another, more common workflow for me is:
Committing like fire on develop
Reviewing work
Some guy runs up to your desk and needs XXX fixed in production
O NO! you have about 20 commits that can’t go to prod
Relax, we use modern version control, Cherry pick change(s)
Update super-project
Push and Deploy
You’re thinking “this is just a bunch of overhead”. Well yea if I didn’t
script all this monotony away from me I would say the same thing. But one
thing I’ve learned from heavy git sub-module usage, is that if you don’t
script all these things, you will forget a step and break something. You’re
also constantly repeating yourself, since you always require N+1 changes for
N changes (you must update the git super-project, which updates the SHA’s
found in .gitmodules).
So, here is the script for committing a single change, which may be across
several submodules. Please note that I have had to on one system change the
git-submodule script that comes with git to use bash in the shebang. This is
because I use shell regex in the foreach commands, often:
123456789101112131415161718192021222324
#!/usr/bin/env bash# This is a silly QnD that lets you develop in the staging sub modules, then# run this to go into each one to commit and push. Saves you some keystrokes.# Force needs to be used sometimes to really push things. What force does is# just change the command chain to use ; instead of &&.export COMMIT_MESSAGE="$1"if[ -z "$COMMIT_MESSAGE"]; thenprintf"You must enter a commit message as the only argument to this script!\n" >&2
exit -1
fiif["$2"=="force"]; thenprintf"Forcing the operation...\n" git submodule foreach 'if [[ $path =~ ^staging.* ]]; then git checkout develop; git add .; git commit -am "$COMMIT_MESSAGE"; git push -u origin develop; fi'elsegit submodule foreach 'if [[ $path =~ ^staging.* ]]; then git checkout develop && git add . && git commit -am "$COMMIT_MESSAGE" && git push -u origin develop; fi'figit commit -am "Updated submodules: $COMMIT_MESSAGE"git push
#vim: set expandtab ts=3 sw=3:
So, this needs to be run from the root of the checkout. My convention for all
code checkouts is $HOME/working/$vcs/$project, so I do:
1234
[~/working/git/puppet]> vim staging/grumps-modules/common/manifests/debuntu.pp
[~/working/git/puppet]> vim staging/manifests/node_templates.pp
[~/working/git/puppet]> scripts/stagcom.sh "Wrote debuntu class for Debian and Ubuntu machines, made sure basenode includes it"
[~/working/git/puppet]> cap deploy
This is all I need to do to get commits in staging out to the master. The
snippet which shows the capistrano stuff I will show last. Promotion is as
easy as:
12
[~/working/git/puppet]> scripts/promote.sh
[~/working/git/puppet]> cap deploy
Or to promote a single commit:
12
[~/working/git/puppet]> scripts/promote.sh -c 73acef8 -m grumps-modules
[~/working/git/puppet]> cap deploy
Now let’s talk about promotion of changes. This is somewhat dependent on your
CR process (if you have one), but here is the script I use for full-env pro
motion as well as cherry-pick promotions:
#!/usr/bin/env bash# == Synopsis# This is a small script that will promote the develop branches in each git# submodule in the Puppet staging environment to master, then then pull in# the changes to the production environment. Note that you can promote single# commits by using the cherry-pick (-c) functionality. If you use cherry pick# you must pass a modulename, ie 'grumps-modules' or 'manifests', with the -m# switch## == Usage# See usage() function below## == Notes# This script does serious changes and pushes to master. Don't run it all# willy nilly.## == Authors# Joe McDonagh <jmcdonagh@thesilentpenguin.com>## == Copyright# 2012 The Silent Penguin LLC## == License# Licensed under The Silent Penguin Proprietary License#puppetroot="$HOME/working/git/puppet"# Useful helper function, shorthand for cat'ing files that have error outputfunction caterror(){ cat "$1" >&2
}# Like caterror, shorthand for sending strings to stderrfunction perror(){printf"%s\n""$1" >&2
}# Print a fatal error and exit with exit code $2function die(){ perror "$1"exit$2}# This will go through all submodules and run the appropriate git log command# to show what commits master needs to be on track with develop.function list_pending_commits(){pushd$puppetroot >/dev/null 2>&1
git submodule --quiet foreach 'if [[ $path =~ ^production.* ]]; then printf "%s:\n" "$(basename $name)"; changes=$(git log master..develop); if [ -z "$changes" ]; then printf "No pending commits.\n"; else printf "%s\n" "$changes"; fi; fi'popd$puppetroot >/dev/null 2>&1
}# Use this function to print out usage information, and exit with code of $2.# This is useful to exit with a non-zero code due to an error in argument# processing.function usage(){printf"Usage:\n"printf" %s [-c commit -m modulename] [-h]\n""$(basename $0)"printf" -l List all commits that master needs in all submodules. Make sure\n"printf" your checkout is up to date if you run this, otherwise results may\n"printf" be inaccurate.\n"printf" -c Cherry pick the commit given as the argument to this switch\n"printf" -f Force mode- this will bypass the prompt when promoting all of staging\n"printf" -h Print this message\n"printf" -m This is required if you use -c; it is the submodule directory name\n"printf" -M This overrides the default commit message with whatever you specify\n"printf"\n"printf"Example:\n"printf" %s -c d4cb267 -m manifests\n\n""$(basename $0)"printf"This will cherry-pick commit d4cb267 from the manifests submodule.\n"printf"\n"printf"Passing no arguments to this script will promote the entire staging env to\n"printf"production.\n"exit$1}while getopts c:lfm:M:h option; do case"$option" in
c)export COMMIT="$OPTARG" ;;
f)force="true" ;;
h) usage 0
;;
l) list_pending_commits
exit 0
;;
m)export MODULE="$OPTARG" ;;
M)export MESSAGE="$OPTARG" ;;
*) perror "Passing bad arguments" usage -1
esacdoneif[ -n "$MODULE" -a -z "$COMMIT"]; thenperror "You passed -m but did not pass -c, you need both." usage -10
fiif[ -z "$MODULE" -a -n "$COMMIT"]; thenperror "You passed -c but did not pass -m, you need both." usage -20
fiif[ ! -e "production/$MODULE"]; thendie "The module $MODULE does not exist!"fi# Verify whether or not commit given actually exists in develop branchif[ -n "$COMMIT"]; thenpushd staging/$MODULE >/dev/null 2>&1
if git branch --contains "$COMMIT" 2>/dev/null | grep -q develop; thencommit_exists="true"elsecommit_exists="false"fipopd >/dev/null 2>&1
if["$commit_exists"=="false"]; thendie "It appears commit $COMMIT does not exist in the develop branch of module $MODULE!"fifi# Force mode in case the script is being used in a batch fashion.if["$force" !="true" -a -z "$COMMIT"]; thenread -p "Are you sure you want to promote the entire staging environment? (y/N) " answer
if["$answer" !="y" -a "$answer" !="Y"]; thendie "Did not confirm full promotion, exiting." -5
fifi# Set commit message, dependent on whether message is passed and if cherry# picking or promoting the whole environment.if[ -z "$MESSAGE" -a -n "$COMMIT"]; thenMESSAGE="Promote commit $COMMIT in submodule $MODULE from staging to production"fiif[ -z "$MESSAGE" -a -z "$COMMIT"]; thenMESSAGE="Promote all of staging to production."fi# Change dir into puppetroot with pushd so we can keep track of where we werepushd$puppetroot >/dev/null 2>&1
# Make sure everything is up to date.scripts/updatecheckout.sh
# Do the actual merging or cherry-picking into production and pushif[ -z "$COMMIT"]; thengit submodule foreach --quiet 'if [[ $path =~ ^production.* ]]; then git checkout develop && git pull && git checkout master && git merge develop && git push; fi'elsegit submodule foreach --quiet 'if [ "$name" == "production/$MODULE" ]; then git checkout develop && git pull && git checkout master && git cherry-pick $COMMIT && git push; fi'figit commit -am "$MESSAGE"git push
popd >/dev/null 2>&1
#vim: set expandtab ts=3 sw=3:
One major point I want to drive home is that your commits in git should fix
the smallest error or add the smallest feature as possible. This is just a
version control best practice, and makes cherry picking possible. If you’re doing
giant commits that change a million things, cherry pick promotions aren’t
going to work for you.
You may have noticed that I call a script named updatecheckout.sh above. This
is the script for updating all the submodules because it’s a serious PITA to
do anything with sub-modules that isn’t scripted. That includes just keeping
your checkout up to date and on the proper branches:
#!/usr/bin/env bash# == Synopsis# This is a small script that will update your whole checkout.## == Usage# See usage() function below.## == Notes# Don't eat the yellow snow.## == Authors# Joe McDonagh <jmcdonagh@thesilentpenguin.com>## == Copyright# 2012 The Silent Penguin LLC## == License# Licensed under The Silent Penguin Proprietary License#puppetroot="$HOME/working/git/puppet"# Useful helper function, shorthand for cat'ing files that have error outputfunction caterror(){ cat "$1" >&2
}# Like caterror, shorthand for sending strings to stderrfunction perror(){printf"%s\n""$1" >&2
}# Print a fatal error and exit with exit code $2function die(){ perror "$1"exit$2}# Change dir into puppetroot with pushd so we can keep track of where we werepushd$puppetroot >/dev/null 2>&1
# Make sure everything is up to date.git pull --all
git submodule update --merge
git submodule foreach --quiet 'if [[ $path =~ ^staging.* ]]; then git checkout develop && git pull; fi'git submodule foreach --quiet 'if [[ $path =~ ^production.* ]]; then git checkout master && git pull && git checkout develop && git pull && git checkout master; fi'popd >/dev/null 2>&1
#vim: set expandtab ts=3 sw=3:
Last but not least, you’ll need some capistrano action. I use capistrano to
deploy, with the railsless-deploy.rb floating around the net. I have one small
modification to the railsless-deploy.rb which will ensure that the submodules
are on the proper branch in the cached-checkout:
# Override deploy! to do some submodule branch stuffmoduleCapistranomoduleDeploymoduleStrategyclassRemoteCache<Remotedefdeploy!update_repository_cache# This will make sure the proper branches are used in the submoduleslogger.info"Ensuring staging sub-modules are using proper develop branches..."devmodules=capture("ls -d #{shared_path}/cached-copy/staging/*")devmodules=devmodules.split("\n")devmodules.eachdo|devmod|run"cd #{devmod}; git checkout develop; git pull; cd -"endlogger.info"Ensuring production sub-modules are using proper master branches..."prodmodules=capture("ls -d #{shared_path}/cached-copy/production/*")prodmodules=prodmodules.split("\n")prodmodules.eachdo|prodmod|run"cd #{prodmod}; git checkout master; git pull; cd -"endcopy_repository_cacheendendendendend
This is done for posterity and to avoid confusion. If you are familiar with
git submodules you know they are typically in a detached state. This means
they are not on any branch in particular. The super-project simply knows what
SHA a given submodule is at. That SHA may correspond to the tip of a branch,
but git submodule isn’t really aware of that. So, the enable submodule var
of Capistrano should be enough, but I like to make sure the cached-checkout is an
exact mirror of what Puppet developers have locally. You also don’t want to
ever be developing in a detached state cause you want your commits to stay
on the proper branch. In fact you probably get an error when trying to commit in
a detached state. Haven’t had to deal with those little mistakes since I wrote
this set of scripts. Here is the actual deploy.rb:
# Configuration Variablesset:admin_group,"webops"set:admin_group_gid,"1337"set:app_user,"puppet"set:application,"puppet"set:copy_exclude,[".git"]set:deploy_lockfile,"/tmp/puppet_being_deployed"set:deploy_to,"/usr/local/puppet"set:deploy_via,"remote_cache"set:git_enable_submodules,1set:keep_releases,5set:local_checkout,"#{ENV['HOME']}/working/git/puppet"set:notification_email,"webops@yourorg.com"set:repository,"ssh://git@github.com/yourorg/puppet"set:scm,"git"set:storeconfig_password,"oFzx.218.jkshfkh82."set:use_sudo,"true"set:use_storeconfigs,"true"# SSH Config Variablesssh_options[:forward_agent]=true# Role for deploymentrole:puppet_masters,"puppet.yourorg.com"# Task to run after everything is done. This runs every time.task:afterpartydorun"if [ ! -d /etc/#{application} ]; then sudo -u root mkdir /etc/#{application}; fi"run"sudo -u root rsync --exclude='tagmail.conf' --delete -cvrP --no-o --no-g --no-p #{deploy_to}/current/ /etc/#{application}/"run"sudo -u root chown -R root:#{app_user} /etc/#{application}; sudo -u root find /etc/#{application} -type d -name 'lib' -prune -o -type d -exec chmod 750 {} \\; && sudo chmod -R g+r /etc/#{application}"run"sudo -u root find /etc/#{application} -type d -name 'lib/*' -exec chmod -R 755 {} \\;"run"sudo -u root chmod -R g+w #{deploy_to}/shared/cached-copy"run"sudo -u root /etc/init.d/apache2 restart"end# Generate puppet docstask:gendocs,:on_error=>:continuedorun"sudo rm -rf /var/www/puppetdocs/staging /var/www/puppetdocs/production"run"sudo /usr/bin/puppet doc -a -m rdoc --outputdir /var/www/puppetdocs/staging/ --manifestdir /etc/puppet/staging/manifests --modulepath '/etc/puppet/staging/grumps-modules:/etc/puppet/staging/yourorg-modules'"run"sudo /usr/bin/puppet doc -a -m rdoc --outputdir /var/www/puppetdocs/production/ --manifestdir /etc/puppet/production/manifests --modulepath '/etc/puppet/production/grumps-modules:/etc/puppet/production/yourorg-modules'"end# This task won't do much to repair a broken git repotask:fix_deploys,:on_error=>:continuedologger.info"Make cached check out group-writeable by #{admin_group}..."run"sudo -u root chgrp -R #{admin_group}#{deploy_to}/shared/cached-copy"run"sudo -u root chmod -R g+w #{deploy_to}/shared/cached-copy"logger.info"Get repo to pristine state..."run"git checkout #{deploy_to}/shared/cached-copy"logger.info"Force permission fixes..."run"sudo -u root find #{deploy_to}/shared/cached-copy -type d -exec chmod 2770 {} \\;"run"sudo -u root find #{deploy_to}/shared/cached-copy -type f -exec chmod 660 {} \\;"logger.info"Unlocking deploys..."run"sudo -u root rm -f #{deploy_lockfile}"endtask:notifydorequire'etc'require'rubygems'require'action_mailer'ActionMailer::Base.delivery_method=:sendmailActionMailer::Base.sendmail_settings={:location=>'/usr/sbin/sendmail',:arguments=>'-i -t'}classNotificationMailer<ActionMailer::Basedefdeployment(application,message,notification_email)mail(:from=>"#{Etc.getpwnam(ENV['USER']).gecos} <#{ENV['USER']}@yourorg.com>",:to=>notification_email,:subject=>"Puppet Deployment - #{Time.now.to_s}",:body=>message)endendmessage="This is a notification of deployment of a Puppet update.\n\n"message<<"Deployed at: #{Time.now.to_s}\n"message<<"Revision: #{real_revision}\n\n"# if the revision has not changed then don't look for logs, also if #SEC# is in the commit message, a full diff is not displayed for security# reasons.beginifprevious_revision!=real_revisionmessage<<"SCM Revisions Deployed\n"gitlog=`#{source.local.log(latest_revision,real_revision)} -v --oneline`ifgitlog.include?'#SEC'message<<gitlogelsemessage<<`#{source.local.log(latest_revision,real_revision)} --submodule=log -v --patch-with-stat`endendrescuemessage<<"SCM Revisions Deployed\n"message<<'Previous revision not available'endmail=NotificationMailer.deployment(application,message,notification_email)mail.deliverend# Task that cats out what revision is deployedtask:getrevdorun"sudo -u root cat /etc/puppet/REVISION"end# Task to add a lock file and bail deploys with a message if one existstask:lock_deploysdorequire'etc'logger.info"Locking deploys..."ifENV.has_key?('lock_reason')lock_reason=ENV['lock_reason']elselock_reason="Deployment"enddata=capture("cat #{deploy_lockfile} 2>/dev/null; echo").to_s.stripif!data.empty?logger.info"\e[0;31;1mATTENTION:\e[0m #{data}"abort"Deploys are locked."endtimestamp=Time.now.strftime("%m/%d/%Y %H:%M:%S %Z")lock_message="Deploys locked by #{Etc.getpwnam(ENV['USER']).gecos} (#{ENV['USER']}) at #{timestamp} for #{lock_reason}"putlock_message,"#{deploy_lockfile}",:mode=>0644endtask:unlock_deploysdologger.info"Unlocking deploys..."run"rm -f #{deploy_lockfile}"end# Before and After hooksafter"deploy:symlink",:afterpartyafter"deploy:rollback",:afterpartybefore"deploy","deploy:cleanup"before"deploy:cleanup",:lock_deploysafter"deploy",:notifybefore"notify",:unlock_deploysafter"notify",:gendocs#vim: set expandtab ts=3 sw=3:
I take no responsibility whatsoever for how you use these scripts. This is
mostly just a demonstration of a workflow that works for me, and keeps everything
clearly separated. I might be able to help people who try to do this, or I
might not (unless you are a paying customer).
I primarily use this to manage the VMware Fusion virtual machines I
use for testing Opscode’s Chef Cookbooks.
This post is rather light on specific details about things that are
either “common” knowledge or documented elsewhere. Particularly, I
won’t tell you how to set up the virtual machines, other than a few
notes that I think make it easier to manage virtual machines in this
way. In other words, you’re smart and can figure them out.
Install and Configure
The first step to use the fission RubyGem is to install it. If you
don’t like RubyGems, then create a package or grab the source from the
GitHub link, above.
gem install fission
Test that it is detecting your VMware Fusion VMs:
fission status
Fission has a configuration file, ~/.fissionrc, which is yaml
format. If the status command fails, you may need to configure fission
to find the vmrun command. Here’s the example from my system:
If you’re reading this, I presume you know how to install an OS in a
VMware virtual machine. I do a number of tasks during the installation
to make it easy and consistent to work with all my test VMs.
Use bridged networking with DHCP
This usually results in the least amount of hassle for connecting to
the VM without any tunneling or port forwarding tomfoolery.
Give it a simple VM name with alphanumeric characters only.
Use the same hostname during installation as the VM name
In the examples below, I use my “guineapig” system. I also have other
systems like “ubuntu1110”, “freebsd82” and “centos6”. This is the name
you’ll use to refer to the VM with fission, so it should be short,
easy to type and clearly identifiable.
Set the root password, even if the OS doesn’t use the root account
Make sure SSH as root is enabled
Some Linux distributions such as Ubuntu do not enable the root login.
This is for testing, so I really don’t care, and I can always write a
Chef recipe to lock things down (as I would in production) if
required.
I also set a simple password that I can use with -P to knife
bootstrap without the shell doing anything with special characters.
Use NTP
Install the NTP package for your operating system. The workflow here
(and the whole point really) is to make heavy use of VMware Fusion
snapshots and rollback, so it is important that the system time is
correct. I customized the bootstrap templates I use to add ntpdate.
Using Fission
And now the moment you’ve been waiting for. First, see the fission
README for full detail on the commands available. I’m going to focus
here on how I use it.
After the install and post install tasks are done, I create a
new snapshot for the VM.
% fission status
guineapig [running]
% fission snapshot create guineapig base
The name is “base” because thats a good name for a baseline. It can be
useful to create specifically named snapshots for particular purposes.
I use Opscode Hosted Chef as my server and I already have my local
workstation set up with the validation key, a Chef repository and have
uploaded the cookbook(s) I use for testing. I’ll use “knife
bootstrap” to kick off a run on my VM:
% knife bootstrap 10.1.1.129 -x root -Pvanilla -r ‘recipe[apache2]’
…
INFO: service[apache2] restarted
INFO: Chef Run complete in 44.324473 seconds
Sweet, it worked. However, if the Chef run fails, I can log in as
root, fix the bug and rerun, or whatever else may need to be done.
Then once I’m ready to reset the VM, fission comes back to play.
% fission snapshot revert guineapig base
Reverting to snapshot ‘base’
Reverted to snapshot ‘base’
Note that fission will poweroff the VM when reverting the snapshot.
Turn it on again with the start command.
% fission status
guineapig [not running]
% fission start guineapig
Starting ‘guineapig’
VM ‘guineapig’ started
% fission status
guineapig [running]
And after logging in, we can see that apache2 is not installed as it
should not be after the snapshot is restored.
% ssh root@guineapig
root@guineapig’s password:
root@guineapig:~# dpkg -l apache2
No packages found matching apache2.
The VM is now ready to do my bidding once again.
Cleanup
Note that reverting the snapshot doesn’t delete the Chef node or
client objects. Since fission is a Ruby library, a simple knife plugin
can wrap up all the fission revert, restart and Chef cleanup, though.
I called mine nukular.
% knife nukular guineapig base guineapig.int.example.com
And here’s the plugin I’m using:
12345678910111213141516171819202122
require'chef/knife'moduleKnifePluginsclassNukular<Chef::Knifedepsdorequire'fission'require'chef/node'require'chef/api_client'endbanner"knife nukular VM SNAPSHOT [NODE]"defrunvm,snapshot=@name_argsnode=@name_args[2].nil??vm:@name_args[2]Fission::Command::SnapshotRevert.new(args=[vm,snapshot]).executeFission::Command::Start.new(args=[vm]).executeChef::Node.load(node).destroyChef::ApiClient.load(node).destroyendendend
The command-line usage takes 2 or 3 arguments. The first two must be
the VM name and the snapshot name, e.g. guineapig and base. If the
node name is different than the VM name, then specify it.
% knife nukular guineapig base guineapig.int.example.com
Note that the plugin has zero error handling or any other sensible
things. You may want to modify it before you use it. Or not, these are
just test systems after all.
Full example
Minus the output from the commands, here is the full output of testing
the Opscode apache2 cookbook on my guinea pig. Assume that all the
required things from my Chef Repository have been uploaded to the Chef
Server and the knife configuration is correct. Also, the chef-full
bootstrap template specified here is a customized version of the
template in the
Chef source master branch template
that has ntpdate -u pool.ntp.org in it.
"An expert is a man who has made all the mistakes which can be made, in a narrow field."
Niels Bohr
When I setup Logstash for the very first time I got bitten by an empty search, aparently no logs were indexed. Reading the log files indeed told me about
WARN: org.elasticsearch.discovery.zen.ping.unicast: [Blaire, Allison] failed to send ping to [[#zen_unicast_1#][inet[/127.0.0.1:9300]]]
The above is the typical error when the ElasticSearch version you are using externally is not in sync with the one Logstash is using, yes those versions need to match.
Fast forward a couple of weeks.. and I`m upgrading Logstash and therefore also ElasticSearch .. I have a Vagrant setup to play with so all of the components are running on 1 node.
I kept running into a similar problem, this time however I saw log entries being indexed, I could get data from my ElasticSearch setup using
wget -q -S -O - http://localhost:9200/_status?pretty=true
But the web interface kept showing no results ;(
While nagging about it on irc .. Jordan gave me the insight :
2012-01-31.194347+0100CET.txt:(07:55:36 PM) whack: slight caveat that elasticsearch clients also join the cluster, so if you point everyone at 127.0.0.1:9300, that :9300 could be one of your clients, not the server
Indeed when you by accident start any of the logstash instances (server/shipper/web) before you start your ElasticSearch instance you can be in trouble.
Ordering really matters , you really need to start ElasticSearch before you start the clients.
Obviously is you don't use the unicast setup you don't run into this problem ..
Kent Skaar pinged me last week , asking for feedback on Lisa'11 and input for Lisa 2012.
Thought I should share my advise to him with the rest of the world
So If I were to host an event similar to Lisa I'd had either
Jordan Sissel or Mitchell Hashimoto give the keynote because over the past 24 months those people have written more relevant tools for me than anyone else :)
I'd have someone talk about Kanban for Operations, There's 2 names that pop up Dominica DeGrandis and Mattias Skarin
I'd have the Ubuntu folks talk about JuJu and I'd have RI Pienaar talk about MCollective .. while you have RI have him talk about Hiera too. Have Dean Wilson carry RI's bags and put him unknowingly on a panel. (Masquerade it as a Pub with hidden cameras)
Obviously as #monitoringsucks you want to hear about new monitoring tools initiatives and how people are dealing with them , so you want people talking about Graphite, Collectd, Statsd, Sensu , Icinga-MQ And how people are reviving Ganglia and using that in large scale environments.
You want someone to demistify Queues, I mean .. who still knows about the differences between Active, Rabbit , Zero, Hornet and many other Q's ?
You want people talking about how they deal with logs, so talks about Logstash and Graylog2.
You want to cover Test Driven Infrastructure How do you test your infrastructure , someone to demystify Cucumber and Webrat , and talk about testing Charms, Modules, and Cookbooks.
Oh and Filesystems , distributed ones the Ceph, FraunhoverFS, Moose, KosmosFS, Glusters, Swifts of this world ... you want people to talk about their experiences , good and bad with any of the above, someone who can actually compare those rather than heresay stuff. :) With recent updates on what's going on in these projects.
Now someone please organise this for me :) In a warm and sunny place ... preferably with 27 holes next door , and daycare for my kids :)
PS. Yes the absence of any openstack related topic is on purpose .. that's for 2013 :)