Package plumbum.commands

plumbum.commands.base.iter_lines(proc, retcode=0, timeout=None, linesize=-1, line_timeout=None, buffer_size=None, mode=None, _iter_lines=<function _iter_lines_posix>)[source]

Runs the given process (equivalent to run_proc()) and yields a tuples of (out, err) line pairs. If the exit code of the process does not match the expected one, ProcessExecutionError is raised.

Parameters:
  • retcode – The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass None. It may also be a tuple (or any iterable) of expected exit codes.

  • timeout – The maximal amount of time (in seconds) to allow the process to run. None means no timeout is imposed; otherwise, if the process hasn’t terminated after that many seconds, the process will be forcefully terminated an exception will be raised

  • linesize – Maximum number of characters to read from stdout/stderr at each iteration. -1 (default) reads until a b’n’ is encountered.

  • line_timeout – The maximal amount of time (in seconds) to allow between consecutive lines in either stream. Raise an ProcessLineTimedOut if the timeout has been reached. None means no timeout is imposed.

  • buffer_size – Maximum number of lines to keep in the stdout/stderr buffers, in case of a ProcessExecutionError. Default is None, which defaults to DEFAULT_BUFFER_SIZE (which is infinite by default). 0 will disable bufferring completely.

  • mode – Controls what the generator yields. Defaults to DEFAULT_ITER_LINES_MODE (which is BY_POSITION by default) - BY_POSITION (default): yields (out, err) line tuples, where either item may be None - BY_TYPE: yields (fd, line) tuples, where fd is 1 (stdout) or 2 (stderr)

Returns:

An iterator of (out, err) line tuples.

plumbum.commands.base.run_proc(proc, retcode, timeout=None)[source]

Waits for the given process to terminate, with the expected exit code

Parameters:
  • proc – a running Popen-like object, with all the expected methods.

  • retcode – the expected return (exit) code of the process. It defaults to 0 (the convention for success). If None, the return code is ignored. It may also be a tuple (or any object that supports __contains__) of expected return codes.

  • timeout – the number of seconds (a float) to allow the process to run, before forcefully terminating it. If None, not timeout is imposed; otherwise the process is expected to terminate within that timeout value, or it will be killed and ProcessTimedOut will be raised

Returns:

A tuple of (return code, stdout, stderr)

plumbum.commands.base.shquote(text)[source]

Quotes the given text with shell escaping (assumes as syntax similar to sh)

exception plumbum.commands.base.RedirectionError[source]

Raised when an attempt is made to redirect an process’ standard handle, which was already redirected to/from a file

__weakref__

list of weak references to the object (if defined)

class plumbum.commands.base.BaseCommand[source]

Base of all command objects

__str__()[source]

Return str(self).

__or__(other)[source]

Creates a pipe with the other command

__gt__(file)[source]

Redirects the process’ stdout to the given file

__rshift__(file)[source]

Redirects the process’ stdout to the given file (appending)

__ge__(file)[source]

Redirects the process’ stderr to the given file

__lt__(file)[source]

Redirects the given file into the process’ stdin

__lshift__(data)[source]

Redirects the given data into the process’ stdin

__getitem__(args)[source]

Creates a bound-command with the given arguments. Shortcut for bound_command.

bound_command(*args)[source]

Creates a bound-command with the given arguments

__call__(*args, **kwargs)[source]

A shortcut for run(args), returning only the process’ stdout

with_env(**env)[source]

Returns a BoundEnvCommand with the given environment variables

with_cwd(path)[source]

Returns a BoundEnvCommand with the specified working directory. This overrides a cwd specified in a wrapping machine.cwd() context manager.

setenv(**env)

Returns a BoundEnvCommand with the given environment variables

formulate(level=0, args=())[source]

Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by Popen or shells.

Parameters:
  • level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed

  • args – The arguments passed to this command (a tuple)

Returns:

A list of strings

popen(args=(), **kwargs)[source]

Spawns the given command, returning a Popen-like object.

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A Popen-like object

nohup(cwd='.', stdout='nohup.out', stderr=None, append=True)[source]

Runs a command detached.

bgrun(args=(), **kwargs)[source]

Runs the given command as a context manager, allowing you to create a pipeline (not in the UNIX sense) of programs, parallelizing their work. In other words, instead of running programs one after the other, you can start all of them at the same time and wait for them to finish. For a more thorough review, see Lightweight Asynchronism.

Example:

from plumbum.cmd import mkfs

with mkfs["-t", "ext3", "/dev/sda1"] as p1:
    with mkfs["-t", "ext3", "/dev/sdb1"] as p2:
        pass

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

For the arguments, see run.

Returns:

A Popen object, augmented with a .run() method, which returns a tuple of (return code, stdout, stderr)

run(args=(), **kwargs)[source]

Runs the given command (equivalent to popen() followed by run_proc). If the exit code of the process does not match the expected one, ProcessExecutionError is raised.

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • retcode

    The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass None. It may also be a tuple (or any iterable) of expected exit codes.

    Note

    this argument must be passed as a keyword argument.

  • timeout

    The maximal amount of time (in seconds) to allow the process to run. None means no timeout is imposed; otherwise, if the process hasn’t terminated after that many seconds, the process will be forcefully terminated an exception will be raised

    Note

    this argument must be passed as a keyword argument.

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A tuple of (return code, stdout, stderr)

run_bg(**kwargs)[source]

Run this command in the background. Uses all arguments from the BG construct :py:class: plumbum.commands.modifiers.BG

run_fg(**kwargs)[source]

Run this command in the foreground. Uses all arguments from the FG construct :py:class: plumbum.commands.modifiers.FG

run_tee(**kwargs)[source]

Run this command using the TEE construct. Inherits all arguments from TEE :py:class: plumbum.commands.modifiers.TEE

run_tf(**kwargs)[source]

Run this command using the TF construct. Inherits all arguments from TF :py:class: plumbum.commands.modifiers.TF

run_retcode(**kwargs)[source]

Run this command using the RETCODE construct. Inherits all arguments from RETCODE :py:class: plumbum.commands.modifiers.RETCODE

run_nohup(**kwargs)[source]

Run this command using the NOHUP construct. Inherits all arguments from NOHUP :py:class: plumbum.commands.modifiers.NOHUP

class plumbum.commands.base.Pipeline(srccmd, dstcmd)[source]
__init__(srccmd, dstcmd)[source]
__repr__()[source]

Return repr(self).

formulate(level=0, args=())[source]

Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by Popen or shells.

Parameters:
  • level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed

  • args – The arguments passed to this command (a tuple)

Returns:

A list of strings

popen(args=(), **kwargs)[source]

Spawns the given command, returning a Popen-like object.

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A Popen-like object

class plumbum.commands.base.BaseRedirection(cmd, file)[source]
__init__(cmd, file)[source]
__repr__()[source]

Return repr(self).

formulate(level=0, args=())[source]

Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by Popen or shells.

Parameters:
  • level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed

  • args – The arguments passed to this command (a tuple)

Returns:

A list of strings

popen(args=(), **kwargs)[source]

Spawns the given command, returning a Popen-like object.

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A Popen-like object

class plumbum.commands.base.BoundCommand(cmd, args)[source]
__init__(cmd, args)[source]
__repr__()[source]

Return repr(self).

formulate(level=0, args=())[source]

Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by Popen or shells.

Parameters:
  • level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed

  • args – The arguments passed to this command (a tuple)

Returns:

A list of strings

popen(args=(), **kwargs)[source]

Spawns the given command, returning a Popen-like object.

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A Popen-like object

class plumbum.commands.base.BoundEnvCommand(cmd, env=None, cwd=None)[source]
__init__(cmd, env=None, cwd=None)[source]
__repr__()[source]

Return repr(self).

formulate(level=0, args=())[source]

Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by Popen or shells.

Parameters:
  • level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed

  • args – The arguments passed to this command (a tuple)

Returns:

A list of strings

popen(args=(), cwd=None, env=None, **kwargs)[source]

Spawns the given command, returning a Popen-like object.

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A Popen-like object

class plumbum.commands.base.ConcreteCommand(executable, encoding)[source]
__init__(executable, encoding)[source]
__str__()[source]

Return str(self).

__repr__()[source]

Return repr(self).

formulate(level=0, args=())[source]

Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by Popen or shells.

Parameters:
  • level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed

  • args – The arguments passed to this command (a tuple)

Returns:

A list of strings

popen(args=(), **kwargs)[source]

Spawns the given command, returning a Popen-like object.

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A Popen-like object

class plumbum.commands.base.StdinRedirection(cmd, file)[source]
class plumbum.commands.base.StdoutRedirection(cmd, file)[source]
class plumbum.commands.base.StderrRedirection(cmd, file)[source]
class plumbum.commands.base.AppendingStdoutRedirection(cmd, file)[source]
class plumbum.commands.base.StdinDataRedirection(cmd, data)[source]
__init__(cmd, data)[source]
formulate(level=0, args=())[source]

Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by Popen or shells.

Parameters:
  • level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed

  • args – The arguments passed to this command (a tuple)

Returns:

A list of strings

popen(args=(), **kwargs)[source]

Spawns the given command, returning a Popen-like object.

Note

When processes run in the background (either via popen or & BG), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48

Parameters:
  • args – Any arguments to be passed to the process (a tuple)

  • kwargs – Any keyword-arguments to be passed to the Popen constructor

Returns:

A Popen-like object

class plumbum.commands.modifiers.Future(proc, expected_retcode, timeout=None)[source]

Represents a “future result” of a running process. It basically wraps a Popen object and the expected exit code, and provides poll(), wait(), returncode, stdout, and stderr.

__init__(proc, expected_retcode, timeout=None)[source]
__repr__()[source]

Return repr(self).

poll()[source]

Polls the underlying process for termination; returns False if still running, or True if terminated

ready()

Polls the underlying process for termination; returns False if still running, or True if terminated

wait()[source]

Waits for the process to terminate; will raise a plumbum.commands.ProcessExecutionError in case of failure

property stdout

The process’ stdout; accessing this property will wait for the process to finish

property stderr

The process’ stderr; accessing this property will wait for the process to finish

property returncode

The process’ returncode; accessing this property will wait for the process to finish

__weakref__

list of weak references to the object (if defined)

class plumbum.commands.modifiers.PipeToLoggerMixin[source]

This mixin allows piping plumbum commands’ output into a logger. The logger must implement a log(level, msg) method, as in logging.Logger

Example:

class MyLogger(logging.Logger, PipeToLoggerMixin):
    pass

logger = MyLogger("example.app")

Here we send the output of an install.sh script into our log:

local['./install.sh'] & logger

We can choose the log-level for each stream:

local['./install.sh'] & logger.pipe(out_level=logging.DEBUG, err_level=logging.DEBUG)

Or use a convenience method for it:

local['./install.sh'] & logger.pipe_debug()

A prefix can be added to each line:

local['./install.sh'] & logger.pipe(prefix="install.sh: ")

If the command fails, an exception is raised as usual. This can be modified:

local['install.sh'] & logger.pipe_debug(retcode=None)

An exception is also raised if too much time (DEFAULT_LINE_TIMEOUT) passed between lines in the stream, This can also be modified:

local['install.sh'] & logger.pipe(line_timeout=10)

If we happen to use logbook:

class MyLogger(logbook.Logger, PipeToLoggerMixin):
    from logbook import DEBUG, INFO  # hook up with logbook's levels
pipe(out_level=None, err_level=None, prefix=None, line_timeout=None, **kw)[source]

Pipe a command’s stdout and stderr lines into this logger.

Parameters:
  • out_level – the log level for lines coming from stdout

  • err_level – the log level for lines coming from stderr

Optionally use prefix for each line.

pipe_info(prefix=None, **kw)[source]

Pipe a command’s stdout and stderr lines into this logger (both at level INFO)

pipe_debug(prefix=None, **kw)[source]

Pipe a command’s stdout and stderr lines into this logger (both at level DEBUG)

__rand__(cmd)[source]

Pipe a command’s stdout and stderr lines into this logger. Log levels for each stream are determined by DEFAULT_STDOUT and DEFAULT_STDERR.

__weakref__

list of weak references to the object (if defined)

exception plumbum.commands.processes.ProcessExecutionError(argv, retcode, stdout, stderr, message=None, *, host=None)[source]

Represents the failure of a process. When the exit code of a terminated process does not match the expected result, this exception is raised by run_proc. It contains the process’ return code, stdout, and stderr, as well as the command line used to create the process (argv)

__init__(argv, retcode, stdout, stderr, message=None, *, host=None)[source]
__str__()[source]

Return str(self).

__weakref__

list of weak references to the object (if defined)

exception plumbum.commands.processes.ProcessTimedOut(msg, argv)[source]

Raises by run_proc when a timeout has been specified and it has elapsed before the process terminated

__init__(msg, argv)[source]
__weakref__

list of weak references to the object (if defined)

exception plumbum.commands.processes.ProcessLineTimedOut(msg, argv, machine)[source]

Raises by iter_lines when a line_timeout has been specified and it has elapsed before the process yielded another line

__init__(msg, argv, machine)[source]
__weakref__

list of weak references to the object (if defined)

exception plumbum.commands.processes.CommandNotFound(program, path)[source]

Raised by local.which and RemoteMachine.which when a command was not found in the system’s PATH

__init__(program, path)[source]
__weakref__

list of weak references to the object (if defined)

plumbum.commands.processes.run_proc(proc, retcode, timeout=None)[source]

Waits for the given process to terminate, with the expected exit code

Parameters:
  • proc – a running Popen-like object, with all the expected methods.

  • retcode – the expected return (exit) code of the process. It defaults to 0 (the convention for success). If None, the return code is ignored. It may also be a tuple (or any object that supports __contains__) of expected return codes.

  • timeout – the number of seconds (a float) to allow the process to run, before forcefully terminating it. If None, not timeout is imposed; otherwise the process is expected to terminate within that timeout value, or it will be killed and ProcessTimedOut will be raised

Returns:

A tuple of (return code, stdout, stderr)

plumbum.commands.processes.iter_lines(proc, retcode=0, timeout=None, linesize=-1, line_timeout=None, buffer_size=None, mode=None, _iter_lines=<function _iter_lines_posix>)[source]

Runs the given process (equivalent to run_proc()) and yields a tuples of (out, err) line pairs. If the exit code of the process does not match the expected one, ProcessExecutionError is raised.

Parameters:
  • retcode – The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass None. It may also be a tuple (or any iterable) of expected exit codes.

  • timeout – The maximal amount of time (in seconds) to allow the process to run. None means no timeout is imposed; otherwise, if the process hasn’t terminated after that many seconds, the process will be forcefully terminated an exception will be raised

  • linesize – Maximum number of characters to read from stdout/stderr at each iteration. -1 (default) reads until a b’n’ is encountered.

  • line_timeout – The maximal amount of time (in seconds) to allow between consecutive lines in either stream. Raise an ProcessLineTimedOut if the timeout has been reached. None means no timeout is imposed.

  • buffer_size – Maximum number of lines to keep in the stdout/stderr buffers, in case of a ProcessExecutionError. Default is None, which defaults to DEFAULT_BUFFER_SIZE (which is infinite by default). 0 will disable bufferring completely.

  • mode – Controls what the generator yields. Defaults to DEFAULT_ITER_LINES_MODE (which is BY_POSITION by default) - BY_POSITION (default): yields (out, err) line tuples, where either item may be None - BY_TYPE: yields (fd, line) tuples, where fd is 1 (stdout) or 2 (stderr)

Returns:

An iterator of (out, err) line tuples.