Package plumbum.commands¶
- plumbum.commands.base.iter_lines(proc, retcode=0, timeout=None, linesize=-1, line_timeout=None, buffer_size=None, mode=None, _iter_lines=<function _iter_lines_posix>)[source]¶
Runs the given process (equivalent to run_proc()) and yields a tuples of (out, err) line pairs. If the exit code of the process does not match the expected one,
ProcessExecutionError
is raised.- Parameters:
retcode – The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass
None
. It may also be a tuple (or any iterable) of expected exit codes.timeout – The maximal amount of time (in seconds) to allow the process to run.
None
means no timeout is imposed; otherwise, if the process hasn’t terminated after that many seconds, the process will be forcefully terminated an exception will be raisedlinesize – Maximum number of characters to read from stdout/stderr at each iteration.
-1
(default) reads until a b’n’ is encountered.line_timeout – The maximal amount of time (in seconds) to allow between consecutive lines in either stream. Raise an
ProcessLineTimedOut
if the timeout has been reached.None
means no timeout is imposed.buffer_size – Maximum number of lines to keep in the stdout/stderr buffers, in case of a ProcessExecutionError. Default is
None
, which defaults to DEFAULT_BUFFER_SIZE (which is infinite by default).0
will disable bufferring completely.mode – Controls what the generator yields. Defaults to DEFAULT_ITER_LINES_MODE (which is BY_POSITION by default) - BY_POSITION (default): yields
(out, err)
line tuples, where either item may beNone
- BY_TYPE: yields(fd, line)
tuples, wherefd
is 1 (stdout) or 2 (stderr)
- Returns:
An iterator of (out, err) line tuples.
- plumbum.commands.base.run_proc(proc, retcode, timeout=None)[source]¶
Waits for the given process to terminate, with the expected exit code
- Parameters:
proc – a running Popen-like object, with all the expected methods.
retcode – the expected return (exit) code of the process. It defaults to 0 (the convention for success). If
None
, the return code is ignored. It may also be a tuple (or any object that supports__contains__
) of expected return codes.timeout – the number of seconds (a
float
) to allow the process to run, before forcefully terminating it. IfNone
, not timeout is imposed; otherwise the process is expected to terminate within that timeout value, or it will be killed andProcessTimedOut
will be raised
- Returns:
A tuple of (return code, stdout, stderr)
- plumbum.commands.base.shquote(text)[source]¶
Quotes the given text with shell escaping (assumes as syntax similar to
sh
)
- exception plumbum.commands.base.RedirectionError[source]¶
Raised when an attempt is made to redirect an process’ standard handle, which was already redirected to/from a file
- __weakref__¶
list of weak references to the object (if defined)
- class plumbum.commands.base.BaseCommand[source]¶
Base of all command objects
- __getitem__(args)[source]¶
Creates a bound-command with the given arguments. Shortcut for bound_command.
- with_cwd(path)[source]¶
Returns a BoundEnvCommand with the specified working directory. This overrides a cwd specified in a wrapping machine.cwd() context manager.
- setenv(**env)¶
Returns a BoundEnvCommand with the given environment variables
- formulate(level=0, args=())[source]¶
Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by
Popen
or shells.- Parameters:
level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed
args – The arguments passed to this command (a tuple)
- Returns:
A list of strings
- popen(args=(), **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- bgrun(args=(), **kwargs)[source]¶
Runs the given command as a context manager, allowing you to create a pipeline (not in the UNIX sense) of programs, parallelizing their work. In other words, instead of running programs one after the other, you can start all of them at the same time and wait for them to finish. For a more thorough review, see Lightweight Asynchronism.
Example:
from plumbum.cmd import mkfs with mkfs["-t", "ext3", "/dev/sda1"] as p1: with mkfs["-t", "ext3", "/dev/sdb1"] as p2: pass
Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48For the arguments, see
run
.- Returns:
A Popen object, augmented with a
.run()
method, which returns a tuple of (return code, stdout, stderr)
- run(args=(), **kwargs)[source]¶
Runs the given command (equivalent to popen() followed by
run_proc
). If the exit code of the process does not match the expected one,ProcessExecutionError
is raised.- Parameters:
args – Any arguments to be passed to the process (a tuple)
retcode –
The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass
None
. It may also be a tuple (or any iterable) of expected exit codes.Note
this argument must be passed as a keyword argument.
timeout –
The maximal amount of time (in seconds) to allow the process to run.
None
means no timeout is imposed; otherwise, if the process hasn’t terminated after that many seconds, the process will be forcefully terminated an exception will be raisedNote
this argument must be passed as a keyword argument.
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A tuple of (return code, stdout, stderr)
- run_bg(**kwargs)[source]¶
Run this command in the background. Uses all arguments from the BG construct :py:class: plumbum.commands.modifiers.BG
- run_fg(**kwargs)[source]¶
Run this command in the foreground. Uses all arguments from the FG construct :py:class: plumbum.commands.modifiers.FG
- run_tee(**kwargs)[source]¶
Run this command using the TEE construct. Inherits all arguments from TEE :py:class: plumbum.commands.modifiers.TEE
- run_tf(**kwargs)[source]¶
Run this command using the TF construct. Inherits all arguments from TF :py:class: plumbum.commands.modifiers.TF
- class plumbum.commands.base.Pipeline(srccmd, dstcmd)[source]¶
-
- formulate(level=0, args=())[source]¶
Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by
Popen
or shells.- Parameters:
level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed
args – The arguments passed to this command (a tuple)
- Returns:
A list of strings
- popen(args=(), **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- class plumbum.commands.base.BaseRedirection(cmd, file)[source]¶
-
- formulate(level=0, args=())[source]¶
Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by
Popen
or shells.- Parameters:
level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed
args – The arguments passed to this command (a tuple)
- Returns:
A list of strings
- popen(args=(), **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- class plumbum.commands.base.BoundCommand(cmd, args)[source]¶
-
- formulate(level=0, args=())[source]¶
Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by
Popen
or shells.- Parameters:
level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed
args – The arguments passed to this command (a tuple)
- Returns:
A list of strings
- popen(args=(), **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- class plumbum.commands.base.BoundEnvCommand(cmd, env=None, cwd=None)[source]¶
-
- formulate(level=0, args=())[source]¶
Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by
Popen
or shells.- Parameters:
level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed
args – The arguments passed to this command (a tuple)
- Returns:
A list of strings
- popen(args=(), cwd=None, env=None, **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- class plumbum.commands.base.ConcreteCommand(executable, encoding)[source]¶
-
- formulate(level=0, args=())[source]¶
Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by
Popen
or shells.- Parameters:
level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed
args – The arguments passed to this command (a tuple)
- Returns:
A list of strings
- popen(args=(), **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- class plumbum.commands.base.StdinDataRedirection(cmd, data)[source]¶
-
- formulate(level=0, args=())[source]¶
Formulates the command into a command-line, i.e., a list of shell-quoted strings that can be executed by
Popen
or shells.- Parameters:
level – The nesting level of the formulation; it dictates how much shell-quoting (if any) should be performed
args – The arguments passed to this command (a tuple)
- Returns:
A list of strings
- popen(args=(), **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- class plumbum.commands.modifiers.Future(proc, expected_retcode, timeout=None)[source]¶
Represents a “future result” of a running process. It basically wraps a
Popen
object and the expected exit code, and provides poll(), wait(), returncode, stdout, and stderr.- poll()[source]¶
Polls the underlying process for termination; returns
False
if still running, orTrue
if terminated
- ready()¶
Polls the underlying process for termination; returns
False
if still running, orTrue
if terminated
- wait()[source]¶
Waits for the process to terminate; will raise a
plumbum.commands.ProcessExecutionError
in case of failure
- property stdout¶
The process’ stdout; accessing this property will wait for the process to finish
- property stderr¶
The process’ stderr; accessing this property will wait for the process to finish
- property returncode¶
The process’ returncode; accessing this property will wait for the process to finish
- __weakref__¶
list of weak references to the object (if defined)
- class plumbum.commands.modifiers.PipeToLoggerMixin[source]¶
This mixin allows piping plumbum commands’ output into a logger. The logger must implement a
log(level, msg)
method, as inlogging.Logger
Example:
class MyLogger(logging.Logger, PipeToLoggerMixin): pass logger = MyLogger("example.app")
Here we send the output of an install.sh script into our log:
local['./install.sh'] & logger
We can choose the log-level for each stream:
local['./install.sh'] & logger.pipe(out_level=logging.DEBUG, err_level=logging.DEBUG)
Or use a convenience method for it:
local['./install.sh'] & logger.pipe_debug()
A prefix can be added to each line:
local['./install.sh'] & logger.pipe(prefix="install.sh: ")
If the command fails, an exception is raised as usual. This can be modified:
local['install.sh'] & logger.pipe_debug(retcode=None)
An exception is also raised if too much time (
DEFAULT_LINE_TIMEOUT
) passed between lines in the stream, This can also be modified:local['install.sh'] & logger.pipe(line_timeout=10)
If we happen to use logbook:
class MyLogger(logbook.Logger, PipeToLoggerMixin): from logbook import DEBUG, INFO # hook up with logbook's levels
- pipe(out_level=None, err_level=None, prefix=None, line_timeout=None, **kw)[source]¶
Pipe a command’s stdout and stderr lines into this logger.
- Parameters:
out_level – the log level for lines coming from stdout
err_level – the log level for lines coming from stderr
Optionally use prefix for each line.
- pipe_info(prefix=None, **kw)[source]¶
Pipe a command’s stdout and stderr lines into this logger (both at level INFO)
- pipe_debug(prefix=None, **kw)[source]¶
Pipe a command’s stdout and stderr lines into this logger (both at level DEBUG)
- __rand__(cmd)[source]¶
Pipe a command’s stdout and stderr lines into this logger. Log levels for each stream are determined by
DEFAULT_STDOUT
andDEFAULT_STDERR
.
- __weakref__¶
list of weak references to the object (if defined)
- exception plumbum.commands.processes.ProcessExecutionError(argv, retcode, stdout, stderr, message=None, *, host=None)[source]¶
Represents the failure of a process. When the exit code of a terminated process does not match the expected result, this exception is raised by
run_proc
. It contains the process’ return code, stdout, and stderr, as well as the command line used to create the process (argv
)- __weakref__¶
list of weak references to the object (if defined)
- exception plumbum.commands.processes.ProcessTimedOut(msg, argv)[source]¶
Raises by
run_proc
when atimeout
has been specified and it has elapsed before the process terminated- __weakref__¶
list of weak references to the object (if defined)
- exception plumbum.commands.processes.ProcessLineTimedOut(msg, argv, machine)[source]¶
Raises by
iter_lines
when aline_timeout
has been specified and it has elapsed before the process yielded another line- __weakref__¶
list of weak references to the object (if defined)
- exception plumbum.commands.processes.CommandNotFound(program, path)[source]¶
Raised by
local.which
andRemoteMachine.which
when a command was not found in the system’sPATH
- __weakref__¶
list of weak references to the object (if defined)
- plumbum.commands.processes.run_proc(proc, retcode, timeout=None)[source]¶
Waits for the given process to terminate, with the expected exit code
- Parameters:
proc – a running Popen-like object, with all the expected methods.
retcode – the expected return (exit) code of the process. It defaults to 0 (the convention for success). If
None
, the return code is ignored. It may also be a tuple (or any object that supports__contains__
) of expected return codes.timeout – the number of seconds (a
float
) to allow the process to run, before forcefully terminating it. IfNone
, not timeout is imposed; otherwise the process is expected to terminate within that timeout value, or it will be killed andProcessTimedOut
will be raised
- Returns:
A tuple of (return code, stdout, stderr)
- plumbum.commands.processes.iter_lines(proc, retcode=0, timeout=None, linesize=-1, line_timeout=None, buffer_size=None, mode=None, _iter_lines=<function _iter_lines_posix>)[source]¶
Runs the given process (equivalent to run_proc()) and yields a tuples of (out, err) line pairs. If the exit code of the process does not match the expected one,
ProcessExecutionError
is raised.- Parameters:
retcode – The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass
None
. It may also be a tuple (or any iterable) of expected exit codes.timeout – The maximal amount of time (in seconds) to allow the process to run.
None
means no timeout is imposed; otherwise, if the process hasn’t terminated after that many seconds, the process will be forcefully terminated an exception will be raisedlinesize – Maximum number of characters to read from stdout/stderr at each iteration.
-1
(default) reads until a b’n’ is encountered.line_timeout – The maximal amount of time (in seconds) to allow between consecutive lines in either stream. Raise an
ProcessLineTimedOut
if the timeout has been reached.None
means no timeout is imposed.buffer_size – Maximum number of lines to keep in the stdout/stderr buffers, in case of a ProcessExecutionError. Default is
None
, which defaults to DEFAULT_BUFFER_SIZE (which is infinite by default).0
will disable bufferring completely.mode – Controls what the generator yields. Defaults to DEFAULT_ITER_LINES_MODE (which is BY_POSITION by default) - BY_POSITION (default): yields
(out, err)
line tuples, where either item may beNone
- BY_TYPE: yields(fd, line)
tuples, wherefd
is 1 (stdout) or 2 (stderr)
- Returns:
An iterator of (out, err) line tuples.