Package plumbum.machines¶
- class plumbum.machines.env.BaseEnv(path_factory, pathsep, *, _curr)[source]¶
The base class of LocalEnv and RemoteEnv
- __call__(*args, **kwargs)[source]¶
A context manager that can be used for temporal modifications of the environment. Any time you enter the context, a copy of the old environment is stored, and then restored, when the context exits.
- Parameters:
args – Any positional arguments for
update()
kwargs – Any keyword arguments for
update()
- __iter__()[source]¶
Returns an iterator over the items
(key, value)
of current environment (like dict.items)
- __getitem__(name)[source]¶
Returns the value of the given environment variable from current environment, raising a
KeyError
if it does not exist
- __setitem__(name, value)[source]¶
Sets/replaces an environment variable’s value in the current environment
- property path¶
The system’s
PATH
(as an easy-to-manipulate list)
- property home¶
Get or set the home path
- property user¶
Return the user name, or
None
if it is not set
- class plumbum.machines.local.PlumbumLocalPopen(*args, **kwargs)[source]¶
- iter_lines(retcode=0, timeout=None, linesize=-1, line_timeout=None, buffer_size=None, mode=None, _iter_lines=<function _iter_lines_posix>)¶
Runs the given process (equivalent to run_proc()) and yields a tuples of (out, err) line pairs. If the exit code of the process does not match the expected one,
ProcessExecutionError
is raised.- Parameters:
retcode – The expected return code of this process (defaults to 0). In order to disable exit-code validation, pass
None
. It may also be a tuple (or any iterable) of expected exit codes.timeout – The maximal amount of time (in seconds) to allow the process to run.
None
means no timeout is imposed; otherwise, if the process hasn’t terminated after that many seconds, the process will be forcefully terminated an exception will be raisedlinesize – Maximum number of characters to read from stdout/stderr at each iteration.
-1
(default) reads until a b’n’ is encountered.line_timeout – The maximal amount of time (in seconds) to allow between consecutive lines in either stream. Raise an
ProcessLineTimedOut
if the timeout has been reached.None
means no timeout is imposed.buffer_size – Maximum number of lines to keep in the stdout/stderr buffers, in case of a ProcessExecutionError. Default is
None
, which defaults to DEFAULT_BUFFER_SIZE (which is infinite by default).0
will disable buffering completely.mode – Controls what the generator yields. Defaults to DEFAULT_ITER_LINES_MODE (which is BY_POSITION by default) - BY_POSITION (default): yields
(out, err)
line tuples, where either item may beNone
- BY_TYPE: yields(fd, line)
tuples, wherefd
is 1 (stdout) or 2 (stderr)
- Returns:
An iterator of (out, err) line tuples.
- class plumbum.machines.local.LocalEnv[source]¶
The local machine’s environment; exposes a dict-like interface
- class plumbum.machines.local.LocalCommand(executable, encoding='auto')[source]¶
-
- popen(args=(), cwd=None, env=None, **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- class plumbum.machines.local.LocalMachine[source]¶
The local machine (a singleton object). It serves as an entry point to everything related to the local machine, such as working directory and environment manipulation, command creation, etc.
Attributes:
cwd
- the local working directoryenv
- the local environmentcustom_encoding
- the local machine’s default encoding (sys.getfilesystemencoding()
)
- classmethod which(progname)[source]¶
Looks up a program in the
PATH
. If the program is not found, raisesCommandNotFound
- Parameters:
progname – The program’s name. Note that if underscores (
_
) are present in the name, and the exact name is not found, they will be replaced in turn by hyphens (-
) then periods (.
), and the name will be looked up again for each alternative- Returns:
A
LocalPath
- path(*parts)[source]¶
A factory for
LocalPaths
. Usage:p = local.path("/usr", "lib", "python2.7")
- __contains__(cmd)[source]¶
Tests for the existence of the command, e.g.,
"ls" in plumbum.local
.cmd
can be anything acceptable by__getitem__
.
- __getitem__(cmd)[source]¶
Returns a Command object representing the given program.
cmd
can be a string or aLocalPath
; if it is a path, a command representing this path will be returned; otherwise, the program name will be looked up in the system’sPATH
(usingwhich
). Usage:ls = local["ls"]
- daemonic_popen(command, cwd='/', stdout=None, stderr=None, append=True)[source]¶
On POSIX systems:
Run
command
as a UNIX daemon: fork a child process to setpid, redirect std handles to /dev/null, umask, close all fds, chdir tocwd
, then fork and execcommand
. Returns aPopen
process that can be used to poll/wait for the executed command (but keep in mind that you cannot access std handles)On Windows:
Run
command
as a “Windows daemon”: detach from controlling console and create a new process group. This means that the command will not receive console events and would survive its parent’s termination. Returns aPopen
object.Note
this does not run
command
as a system service, only detaches it from its parent.New in version 1.3.
- list_processes()[source]¶
Returns information about all running processes (on POSIX systems: using
ps
)New in version 1.3.
- pgrep(pattern)[source]¶
Process grep: return information about all processes whose command-line args match the given regex pattern
- session(new_session=False)[source]¶
Creates a new
ShellSession
object; this invokes/bin/sh
and executes commands on it over stdin/stdout/stderr
- tempdir()[source]¶
A context manager that creates a temporary directory, which is removed when the context exits
- as_user(username=None)[source]¶
Run nested commands as the given user. For example:
head = local["head"] head("-n1", "/dev/sda1") # this will fail... with local.as_user(): head("-n1", "/dev/sda1")
- Parameters:
username – The user to run commands as. If not given, root (or Administrator) is assumed
- as_root()[source]¶
A shorthand for
as_user("root")
- python = LocalCommand(/home/docs/checkouts/readthedocs.org/user_builds/plumbum/envs/latest/bin/python)¶
A command that represents the current python interpreter (
sys.executable
)
- plumbum.machines.local.local = <plumbum.machines.local.LocalMachine object>¶
The local machine (a singleton object). It serves as an entry point to everything related to the local machine, such as working directory and environment manipulation, command creation, etc.
Attributes:
cwd
- the local working directoryenv
- the local environmentcustom_encoding
- the local machine’s default encoding (sys.getfilesystemencoding()
)
- exception plumbum.machines.session.ShellSessionError[source]¶
Raises when something goes wrong when calling
ShellSession.popen
- __weakref__¶
list of weak references to the object (if defined)
- exception plumbum.machines.session.SSHCommsError(argv, retcode, stdout, stderr, message=None, *, host=None)[source]¶
Raises when the communication channel can’t be created on the remote host or it times out.
- exception plumbum.machines.session.SSHCommsChannel2Error(argv, retcode, stdout, stderr, message=None, *, host=None)[source]¶
Raises when channel 2 (stderr) is not available
- exception plumbum.machines.session.IncorrectLogin(argv, retcode, stdout, stderr, message=None, *, host=None)[source]¶
Raises when incorrect login credentials are provided
- exception plumbum.machines.session.HostPublicKeyUnknown(argv, retcode, stdout, stderr, message=None, *, host=None)[source]¶
Raises when the host public key isn’t known
- class plumbum.machines.session.MarkedPipe(pipe, marker)[source]¶
A pipe-like object from which you can read lines; the pipe will return report EOF (the empty string) when a special marker is detected
- class plumbum.machines.session.SessionPopen(proc, argv, isatty, stdin, stdout, stderr, encoding, *, host)[source]¶
A shell-session-based
Popen
-like object (has the following attributes:stdin
,stdout
,stderr
,returncode
)
- class plumbum.machines.session.ShellSession(proc, encoding='auto', isatty=False, connect_timeout=5, *, host=None)[source]¶
An abstraction layer over shell sessions. A shell session is the execution of an interactive shell (
/bin/sh
or something compatible), over which you may run commands (sent over stdin). The output of is then read from stdout and stderr. Shell sessions are less “robust” than executing a process on its own, and they are susseptible to all sorts of malformatted-strings attacks, and there is little benefit from using them locally. However, they can greatly speed up remote connections, and are required for the implementation ofSshMachine
, as they allow us to send multiple commands over a single SSH connection (setting up separate SSH connections incurs a high overhead). Try to avoid using shell sessions, unless you know what you’re doing.Instances of this class may be used as context-managers.
- Parameters:
proc – The underlying shell process (with open stdin, stdout and stderr)
encoding – The encoding to use for the shell session. If
"auto"
, the underlying process’ encoding is used.isatty – If true, assume the shell has a TTY and that stdout and stderr are unified
connect_timeout – The timeout to connect to the shell, after which, if no prompt is seen, the shell process is killed
- __weakref__¶
list of weak references to the object (if defined)
Remote Machines¶
- class plumbum.machines.remote.RemoteEnv(remote)[source]¶
The remote machine’s environment; exposes a dict-like interface
- __setitem__(name, value)[source]¶
Sets/replaces an environment variable’s value in the current environment
- expand(expr)[source]¶
Expands any environment variables and home shortcuts found in
expr
(likeos.path.expanduser
combined withos.path.expandvars
)- Parameters:
expr – An expression containing environment variables (as
$FOO
) or home shortcuts (as~/.bashrc
)- Returns:
The expanded string
- class plumbum.machines.remote.RemoteCommand(remote, executable, encoding='auto')[source]¶
-
- popen(args=(), **kwargs)[source]¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- exception plumbum.machines.remote.ClosedRemoteMachine[source]¶
- __weakref__¶
list of weak references to the object (if defined)
- class plumbum.machines.remote.BaseRemoteMachine(encoding='utf8', connect_timeout=10, new_session=False)[source]¶
Represents a remote machine; serves as an entry point to everything related to that remote machine, such as working directory and environment manipulation, command creation, etc.
Attributes:
cwd
- the remote working directoryenv
- the remote environmentcustom_encoding
- the remote machine’s default encoding (assumed to be UTF8)connect_timeout
- the connection timeout
There also is a _cwd attribute that exists if the cwd is not current (del if cwd is changed).
- class RemoteCommand(remote, executable, encoding='auto')¶
- __init__(remote, executable, encoding='auto')¶
- __repr__()¶
Return repr(self).
- nohup(cwd='.', stdout='nohup.out', stderr=None, append=True)¶
Runs a command detached.
- popen(args=(), **kwargs)¶
Spawns the given command, returning a
Popen
-like object.Note
When processes run in the background (either via
popen
or& BG
), their stdout/stderr pipes might fill up, causing them to hang. If you know a process produces output, be sure to consume it every once in a while, using a monitoring thread/reactor in the background. For more info, see #48- Parameters:
args – Any arguments to be passed to the process (a tuple)
kwargs – Any keyword-arguments to be passed to the
Popen
constructor
- Returns:
A
Popen
-like object
- close()[source]¶
closes the connection to the remote machine; all paths and programs will become defunct
- path(*parts)[source]¶
A factory for
RemotePaths
. Usage:p = rem.path("/usr", "lib", "python2.7")
- which(progname)[source]¶
Looks up a program in the
PATH
. If the program is not found, raisesCommandNotFound
- Parameters:
progname – The program’s name. Note that if underscores (
_
) are present in the name, and the exact name is not found, they will be replaced in turn by hyphens (-
) then periods (.
), and the name will be looked up again for each alternative- Returns:
A
RemotePath
- __getitem__(cmd)[source]¶
Returns a Command object representing the given program.
cmd
can be a string or aRemotePath
; if it is a path, a command representing this path will be returned; otherwise, the program name will be looked up in the system’sPATH
(usingwhich
). Usage:r_ls = rem["ls"]
- property python¶
A command that represents the default remote python interpreter
- session(isatty=False, *, new_session=False)[source]¶
Creates a new
ShellSession
object; this invokes the user’s shell on the remote machine and executes commands on it over stdin/stdout/stderr
- download(src, dst)[source]¶
Downloads a remote file/directory (
src
) to a local destination (dst
).src
must be a string or aRemotePath
pointing to this remote machine, anddst
must be a string or aLocalPath
- upload(src, dst)[source]¶
Uploads a local file/directory (
src
) to a remote destination (dst
).src
must be a string or aLocalPath
, anddst
must be a string or aRemotePath
pointing to this remote machine
- popen(args, **kwargs)[source]¶
Spawns the given command on the remote machine, returning a
Popen
-like object; do not use this method directly, unless you need “low-level” control on the remote process
- list_processes()[source]¶
Returns information about all running processes (on POSIX systems: using
ps
)New in version 1.3.
- class plumbum.machines.ssh_machine.SshTunnel(session, lport, dport, reverse)[source]¶
An object representing an SSH tunnel (created by
SshMachine.tunnel
)- property lport¶
Tunneled port or socket on the local machine.
- property dport¶
Tunneled port or socket on the remote machine.
- property reverse¶
Represents if the tunnel is a reverse tunnel.
- class plumbum.machines.ssh_machine.SshMachine(host, user=None, port=None, keyfile=None, ssh_command=None, scp_command=None, ssh_opts=(), scp_opts=(), password=None, encoding='utf8', connect_timeout=10, new_session=False)[source]¶
An implementation of
remote machine
over SSH. Invoking a remote command translates to invoking it over SSHwith SshMachine("yourhostname") as rem: r_ls = rem["ls"] # r_ls is the remote `ls` # executing r_ls() translates to `ssh yourhostname ls`
- Parameters:
host – the host name to connect to (SSH server)
user – the user to connect as (if
None
, the default will be used)port – the server’s port (if
None
, the default will be used)keyfile – the path to the identity file (if
None
, the default will be used)ssh_command – the
ssh
command to use; this has to be aCommand
object; ifNone
, the default ssh client will be used.scp_command – the
scp
command to use; this has to be aCommand
object; ifNone
, the default scp program will be used.ssh_opts – any additional options for
ssh
(a list of strings)scp_opts – any additional options for
scp
(a list of strings)password – the password to use; requires
sshpass
be installed. Cannot be used in conjunction withssh_command
orscp_command
(will be ignored). NOTE: THIS IS A SECURITY RISK!encoding – the remote machine’s encoding (defaults to UTF8)
connect_timeout – specify a connection timeout (the time until shell prompt is seen). The default is 10 seconds. Set to
None
to disablenew_session – whether or not to start the background session as a new session leader (setsid). This will prevent it from being killed on Ctrl+C (SIGINT)
- __init__(host, user=None, port=None, keyfile=None, ssh_command=None, scp_command=None, ssh_opts=(), scp_opts=(), password=None, encoding='utf8', connect_timeout=10, new_session=False)[source]¶
- popen(args, ssh_opts=(), env=None, cwd=None, **kwargs)[source]¶
Spawns the given command on the remote machine, returning a
Popen
-like object; do not use this method directly, unless you need “low-level” control on the remote process
- nohup(command)[source]¶
Runs the given command using
nohup
and redirects std handles, allowing the command to run “detached” from its controlling TTY or parent. Does not return anything. Depreciated (use command.nohup or daemonic_popen).
- daemonic_popen(command, cwd='.', stdout=None, stderr=None, append=True)[source]¶
Runs the given command using
nohup
and redirects std handles, allowing the command to run “detached” from its controlling TTY or parent. Does not return anything.New in version 1.6.0.
- session(isatty=False, new_session=False)[source]¶
Creates a new
ShellSession
object; this invokes the user’s shell on the remote machine and executes commands on it over stdin/stdout/stderr
- tunnel(lport, dport, lhost='localhost', dhost='localhost', connect_timeout=5, reverse=False)[source]¶
Creates an SSH tunnel from the TCP port (
lport
) of the local machine (lhost
, defaults to"localhost"
, but it can be any IP you canbind()
) to the remote TCP port (dport
) of the destination machine (dhost
, defaults to"localhost"
, which means this remote machine). This function also supports Unix sockets, in which case the local socket should be passed in aslport
and the local bind address should beNone
. The same can be done for a remote socket, by following the same pattern withdport
anddhost
. The returnedSshTunnel
object can be used as a context-manager.The more conventional use case is the following:
+---------+ +---------+ | Your | | Remote | | Machine | | Machine | +----o----+ +---- ----+ | ^ | | lport dport | | \______SSH TUNNEL____/ (secure)
Here, you wish to communicate safely between port
lport
of your machine and portdport
of the remote machine. Communication is tunneled over SSH, so the connection is authenticated and encrypted.The more general case is shown below (where
dport != "localhost"
):+---------+ +-------------+ +-------------+ | Your | | Remote | | Destination | | Machine | | Machine | | Machine | +----o----+ +---- ----o---+ +---- --------+ | ^ | ^ | | | | lhost:lport | | dhost:dport | | | | \_____SSH TUNNEL_____/ \_____SOCKET____/ (secure) (not secure)
Usage:
rem = SshMachine("megazord") with rem.tunnel(1234, "/var/lib/mysql/mysql.sock", dhost=None): sock = socket.socket() sock.connect(("localhost", 1234)) # sock is now tunneled to the MySQL socket on megazord
- download(src, dst)[source]¶
Downloads a remote file/directory (
src
) to a local destination (dst
).src
must be a string or aRemotePath
pointing to this remote machine, anddst
must be a string or aLocalPath
- upload(src, dst)[source]¶
Uploads a local file/directory (
src
) to a remote destination (dst
).src
must be a string or aLocalPath
, anddst
must be a string or aRemotePath
pointing to this remote machine
- class plumbum.machines.ssh_machine.PuttyMachine(host, user=None, port=None, keyfile=None, ssh_command=None, scp_command=None, ssh_opts=(), scp_opts=(), encoding='utf8', connect_timeout=10, new_session=False)[source]¶
PuTTY-flavored SSH connection. The programs
plink
andpscp
are expected to be in the path (or you may provide your ownssh_command
andscp_command
)Arguments are the same as for
plumbum.machines.remote.SshMachine
- class plumbum.machines.paramiko_machine.ParamikoPopen(argv, stdin, stdout, stderr, encoding, stdin_file=None, stdout_file=None, stderr_file=None)[source]¶
- class plumbum.machines.paramiko_machine.ParamikoMachine(host, user=None, port=None, password=None, keyfile=None, load_system_host_keys=True, missing_host_policy=None, encoding='utf8', look_for_keys=None, connect_timeout=None, keep_alive=0, gss_auth=False, gss_kex=None, gss_deleg_creds=None, gss_host=None, get_pty=False, load_system_ssh_config=False)[source]¶
An implementation of
remote machine
over Paramiko (a Python implementation of openSSH2 client/server). Invoking a remote command translates to invoking it over SSHwith ParamikoMachine("yourhostname") as rem: r_ls = rem["ls"] # r_ls is the remote `ls` # executing r_ls() is equivalent to `ssh yourhostname ls`, only without # spawning a new ssh client
- Parameters:
host – the host name to connect to (SSH server)
user – the user to connect as (if
None
, the default will be used)port – the server’s port (if
None
, the default will be used)password – the user’s password (if a password-based authentication is to be performed) (if
None
, key-based authentication will be used)keyfile – the path to the identity file (if
None
, the default will be used)load_system_host_keys – whether or not to load the system’s host keys (from
/etc/ssh
and~/.ssh
). The default isTrue
, which means Paramiko behaves much like thessh
command-line clientmissing_host_policy – the value passed to the underlying
set_missing_host_key_policy
of the client. The default isNone
, which meansset_missing_host_key_policy
is not invoked and paramiko’s default behavior (reject) is employedencoding – the remote machine’s encoding (defaults to UTF8)
look_for_keys – set to False to disable searching for discoverable private key files in
~/.ssh
connect_timeout – timeout for TCP connection
Note
If Paramiko 1.15 or above is installed, can use GSS_API authentication
- Parameters:
gss_auth (bool) –
True
if you want to use GSS-API authenticationgss_kex (bool) – Perform GSS-API Key Exchange and user authentication
gss_deleg_creds (bool) – Delegate GSS-API client credentials or not
gss_host (str) – The targets name in the kerberos database. default: hostname
get_pty (bool) – Execute remote commands with allocated pseudo-tty. default: False
load_system_ssh_config (bool) – read system SSH config for ProxyCommand configuration. default: False
- __init__(host, user=None, port=None, password=None, keyfile=None, load_system_host_keys=True, missing_host_policy=None, encoding='utf8', look_for_keys=None, connect_timeout=None, keep_alive=0, gss_auth=False, gss_kex=None, gss_deleg_creds=None, gss_host=None, get_pty=False, load_system_ssh_config=False)[source]¶
- close()[source]¶
closes the connection to the remote machine; all paths and programs will become defunct
- property sftp¶
Returns an SFTP client on top of the current SSH connection; it can be used to manipulate files directly, much like an interactive FTP/SFTP session
- session(isatty=False, term='vt100', width=80, height=24, *, new_session=False)[source]¶
Creates a new
ShellSession
object; this invokes the user’s shell on the remote machine and executes commands on it over stdin/stdout/stderr
- popen(args, stdin=None, stdout=None, stderr=None, new_session=False, env=None, cwd=None)[source]¶
Spawns the given command on the remote machine, returning a
Popen
-like object; do not use this method directly, unless you need “low-level” control on the remote process
- download(src, dst)[source]¶
Downloads a remote file/directory (
src
) to a local destination (dst
).src
must be a string or aRemotePath
pointing to this remote machine, anddst
must be a string or aLocalPath
- upload(src, dst)[source]¶
Uploads a local file/directory (
src
) to a remote destination (dst
).src
must be a string or aLocalPath
, anddst
must be a string or aRemotePath
pointing to this remote machine
- connect_sock(dport, dhost='localhost', ipv6=False)[source]¶
Returns a Paramiko
Channel
, connected to dhost:dport on the remote machine. TheChannel
behaves like a regular socket; you cansend
andrecv
on it and the data will pass encrypted over SSH. Usage:mach = ParamikoMachine("myhost") sock = mach.connect_sock(12345) data = sock.recv(100) sock.send("foobar") sock.close()