Remote¶
Just like running local commands, Plumbum supports running commands on remote systems, by executing them over SSH.
Remote Machines¶
Forming a connection to a remote machine is very straight forward:
>>> from plumbum import SshMachine
>>> rem = SshMachine("hostname", user = "john", keyfile = "/path/to/idrsa")
>>> # ...
>>> rem.close()
Or as a context-manager:
>>> with SshMachine("hostname", user = "john", keyfile = "/path/to/idrsa") as rem:
... pass
Note
SshMachine
requires ssh
(openSSH
or compatible) installed on your system in order
to connect to remote machines. The remote machine must have bash as the default shell (or any shell
that supports the 2>&1
syntax for stderr redirection).
Alternatively, you can use the pure-Python implementation of
ParamikoMachine.
Only the hostname
parameter is required, all other parameters are optional. If the host has
your id-rsa.pub
key in its authorized_keys
file, or if you’ve set up your ~/.ssh/config
to login with some user and keyfile
, you can simply use rem = SshMachine("hostname")
.
Much like the local object, remote machines expose which()
,
path()
, python
, cwd
and env
. You can also run remote commands, create SSH tunnels,
upload/download files, etc. You may also refer to the full API
, as this guide will only survey the features.
Note
PuTTY users on Windows should use
the dedicated PuttyMachine
instead of
SshMachine
. See also ParamikoMachine.
New in version 1.0.1.
Working Directory and Environment¶
The cwd
and env
attributes represent the remote machine’s working directory and environment
variables, respectively, and can be used to inspect or manipulate them. Much like their local
counterparts, they can be used as context managers, so their effects can be contained.
>>> rem.cwd
<Workdir /home/john>
>>> with rem.cwd(rem.cwd / "Desktop"):
... print(rem.cwd)
/home/john/Desktop
>>> rem.env["PATH"]
/bin:/sbin:/usr/bin:/usr/local/bin
>>> rem.which("ls")
<RemotePath /bin/ls>
Tunneling¶
SSH tunneling is a very useful feature of the SSH protocol. It allows you to connect from your
machine to a remote server process, while having your connection authenticated and encrypted
out-of-the-box. Say you run on machine-A
, and you wish to connect to a server program
running on machine-B
. That server program binds to localhost:8888
(where localhost
refers naturally to to machine-B
). Using Plumbum, you can easily set up a tunnel from
port 6666 on machine-A
to port 8888 on machine-B
:
>>> tun = rem.tunnel(6666, 8888)
>>> # ...
>>> tun.close()
Or as a context manager:
>>> with rem.tunnel(6666, 8888):
... pass
You can now connect a socket to machine-A:6666
, and it will be securely forwarded over SSH
to machine-B:8888
. When the tunnel object is closed, all active connections will be
dropped.
Remote Commands¶
Like local commands, remote commands are created using indexing ([]
) on a remote machine
object. You can either pass the command’s name, in which case it will be resolved by through
which
, or the path to the program.
>>> rem["ls"]
<RemoteCommand(<RemoteMachine ssh://hostname>, '/bin/ls')>
>>> rem["/usr/local/bin/python3.2"]
<RemoteCommand(<RemoteMachine ssh://hostname>, '/usr/local/bin/python3.2')>
>>> r_ls = rem["ls"]
>>> r_grep = rem["grep"]
>>> r_ls()
'foo\nbar\spam\n'
Nesting Commands¶
Remote commands can be nested just like local ones. In fact, that’s how the SshMachine
operates
behind the scenes - it nests each command inside ssh
. Here are some examples:
>>> r_sudo = rem["sudo"]
>>> r_ifconfig = rem["ifconfig"]
>>> print(r_sudo[r_ifconfig["-a"]]())
eth0 Link encap:Ethernet HWaddr ...
[...]
You can nest multiple commands, one within another. For instance, you can connect to some machine over SSH and use that machine’s SSH client to connect to yet another machine. Here’s a sketch:
>>> from plumbum.cmd import ssh
>>> print(ssh["localhost", ssh["localhost", "ls"]])
/usr/bin/ssh localhost /usr/bin/ssh localhost ls
>>>
>>> ssh["localhost", ssh["localhost", "ls"]]()
'bin\nDesktop\nDocuments\n...'
Piping¶
Piping works for remote commands as well, but there’s a caveat to note here: the plumbing takes place on the local machine! Consider this code for instance
>>> r_grep = rem["grep"]
>>> r_ls = rem["ls"]
>>> (r_ls | r_grep["b"])()
'bin\nPublic\n'
Although r_ls
and r_grep
are remote commands, the data is sent from r_ls
to the local
machine, which then sends it to the remote one for running grep
. This will be fixed in a future
version of Plumbum.
It should be noted, however, that piping remote commands into local ones is perfectly fine. For example, the previous code can be written as
>>> from plumbum.cmd import grep
>>> (r_ls | grep["b"])()
'bin\nPublic\n'
Which is even more efficient (no need to send data back and forth over SSH).
Redirection¶
Redirection to and from remote paths is not currently supported, but you can redirect to and from local paths, with the familiar syntax explained in the corresponding section for local commands. Note that if the redirection target/source is given as a string, it is automatically interpreted as a path on the local machine.
Paramiko Machine¶
New in version 1.1.
SshMachine
relies on the system’s ssh
client to run commands; this means that for each
remote command you run, a local process is spawned and an SSH connection is established.
While relying on a well-known and trusted SSH client is the most stable option, the incurred
overhead of creating a separate SSH connection for each command may be too high. In order to
overcome this, Plumbum provides integration for paramiko,
an open-source, pure-Python implementation of the SSH2 protocol. This is the ParamikoMachine
,
and it works along the lines of the SshMachine
:
>>> from plumbum.machines.paramiko_machine import ParamikoMachine
>>> rem = ParamikoMachine("192.168.1.143")
>>> rem["ls"]
RemoteCommand(<ParamikoMachine paramiko://192.168.1.143>, <RemotePath /bin/ls>)
>>> r_ls = rem["ls"]
>>> r_ls()
'bin\nDesktop\nDocuments\nDownloads\nexamples.desktop\nMusic\nPictures\n...'
>>> r_ls("-a")
'.\n..\n.adobe\n.bash_history\n.bash_logout\n.bashrc\nbin...'
Note
Using ParamikoMachine
requires paramiko to be installed on your system. Also, you have
to explicitly import it (from plumbum.machines.paramiko_machine import ParamikoMachine
) as paramiko
is quite heavy.
Refer to the API docs
for more details.
The main advantage of using ParamikoMachine
is that only a single, persistent SSH connection
is created, over which commands execute. Moreover, paramiko has a built-in SFTP client, which is
used instead of scp
to copy files (employed by the .download()
/.upload()
methods),
and tunneling is much more light weight: In the SshMachine
, a tunnel is created by an external
process that lives for as long as the tunnel is to remain active. The ParamikoMachine
, however,
can simply create an extra channel on top of the same underlying connection with ease; this is
exposed by connect_sock()
, which creates a tunneled TCP connection and returns a socket-like
object
Warning
Piping and input/output redirection don’t really work with ParamikoMachine
commands.
You’ll get all kinds of errors, like 'ChannelFile' object has no attribute 'fileno'
or
I/O operation on closed file
– this is due to the fact that Paramiko’s channels are not
real, OS-level files, so they can’t interact with subprocess.Popen
.
This will be solved in a future release; in the meanwhile, you can use the machine’s
.session()
method, like so
>>> s = mach.session()
>>> s.run("ls | grep b")
(0, 'bin\nPublic\n', '')
Tunneling Example¶
On 192.168.1.143
, I ran the following sophisticated server (notice it’s bound to localhost
):
>>> import socket
>>> s=socket.socket()
>>> s.bind(("localhost", 12345))
>>> s.listen(1)
>>> s2,_=s.accept()
>>> while True:
... data = s2.recv(1000)
... if not data:
... break
... s2.send("I eat " + data)
...
On my other machine, I connect (over SSH) to this host and then create a tunneled connection to port 12345, getting back a socket-like object:
>>> rem = ParamikoMachine("192.168.1.143")
>>> s = rem.connect_sock(12345)
>>> s.send("carrot")
6
>>> s.recv(1000)
'I eat carrot'
>>> s.send("babies")
6
>>> s.recv(1000)
'I eat babies'
>>> s.close()
Remote Paths¶
Analogous to local paths, remote paths represent a file-system path of a remote system, and expose a set of utility functions for iterating over subpaths, creating subpaths, moving/copying/ renaming paths, etc.
>>> p = rem.path("/bin")
>>> p / "ls"
<RemotePath /bin/ls>
>>> (p / "ls").is_file()
True
>>> rem.path("/dev") // "sd*"
[<RemotePath /dev/sda>, < RemotePath /dev/sdb>, <RemotePath /dev/sdb1>, <RemotePath /dev/sdb2>]
Note
See the Utilities guide for copying, moving and deleting remote paths
For further information, see the api docs.