DarkMatter in Cyberspace
  • Home
  • Categories
  • Tags
  • Archives

Task Management in Fabric


I am developing a environment detect application with Fabric. Basically it run many tests on many hosts, then produce a summary report, optionally sending a email. In the codes below for example, function "detect" should run on every host, but "report" and "sendmail" should run only once.

So there are 2 strategies:

  • Define global hosts, when detections over, modify the hosts to none, so report and send email will only run once;

  • Use "report" as entrance, invoke detection within "report" task.

Global hosts style

fabfile.py:

from fabric.api import run, env, hosts
import re

env.hosts = ['chad@10.0.2.47', 'chad@10.0.2.49']
env.password = 'mypasswd'
env['report'] = {}

def checkJDK():
    res = run('java -version')
    cre = 'java version .*1.6.0.*Java\(TM\)'
    return len(re.findall(cre, res, re.S)) > 0

def checkMem(str):
    re.findall('.*buffers/cache.*\d+\s+(\d+)', str)

def checkOS():
    res = run('uname -a')
    return res.split()[0]

def detect():
    host_report = {}
    host_report['os'] = checkOS()
    host_report['jdk'] = checkJDK()
    env['report'][env.host] = host_report
    env.hosts = []    # make following tasks run once

def report():
    print('Test Report:')
    print(env.report)

def sendmail():
    print('send mail...')

Run it:

$ fab --hide=everything detect report sendmail
Test Report:
{'10.0.2.49': {'os': 'Linux', 'jdk': True}, '10.0.2.47': {'os': 'Linux', 'jdk': False}}
send mail...

"execute" style

fab2.py:

from fabric.api import run, env, task, execute
import re

def checkJDK():
    res = run('java -version')
    cre = 'java version .*1.6.0.*Java\(TM\)'
    return len(re.findall(cre, res, re.S)) > 0

def checkMem(str):
    re.findall('.*buffers/cache.*\d+\s+(\d+)', str)

def checkOS():
    res = run('uname -a')
    return res.split()[0]

def detect():
    env.password = env.loginfo[env.host_string]
    host_report = {}
    host_report['os'] = checkOS()
    host_report['jdk'] = checkJDK()
    return host_report

@task
def report():
    env.hosts= ['gcp@10.0.2.48', 'gcp@10.0.2.49', 'gcp@10.0.7.141', 'gcp@10.0.7.142', 'gcp@10.0.7.143:22']
    pwds = ['gcp', 'gcp', 'gcp@123', 'gcp@123', 'gcp@123']
    env.loginfo = dict(zip(env.hosts, pwds))
    res = execute(detect, hosts=host_list)
    print('Test Report:')
    print(res)

@task
def sendmail():
    print('send mail...')

Run it:

$ fab -f fab2.py --hide=everything report sendmail
Test Report:
{'gcp@10.0.2.47': {'os': 'Linux', 'jdk': False}, 'gcp@10.0.2.49': {'os': 'Linux', 'jdk': True}, ... }
send mail...

You can see there is no definition and modification of global hosts, no need to maintain global variable env['report'] manually in "report" style, and the "detect" function is hide from "fab" command line with the help of "@task" decorator. If you run "fab detect", it will complain "Command(s) not found", which provides more accurate visibility control. So I think it's better than the "global hosts" style.

Notes:

  1. the return value of "run" is the stdout and stderr of the remote command.

  2. Yuo can add attributes to "env" directly like "env.loginfo = ..." above. And use this attribute with "env.loginfo[...]";

  3. The format of host string is "user@ip:port", or "user@ip" if port is 22;

  4. The function "execute" returns a dict, the key is the host_string, the value is the return value of the function executed ("detect" in this case). The return value of "detect" is a dict, so the "execute" function returns a nested dict.

  5. for fabric 1.11, you can use run_once decorator.



Published

Mar 28, 2014

Last Updated

Mar 28, 2014

Category

Tech

Tags

  • Fabric 8

Contact

  • Powered by Pelican. Theme: Elegant by Talha Mansoor