Recently I had to do a task where I had to automate the downloading of files from a sftp server. Automating plain ftp download is much easier than automating sftp because there are prompts from the remote server that have to be handled. For automating such interactive applications there is a wonderful tool called Expect. The automation scripts for Expect are written in the Tcl language. There was an existing Tcl Expect script to automate downloading of files from sftp. I found that I did not have Tcl Expect installed and I had heard that Perl Expect is very similar to TCL Expect. So I decided to give it a shot. I ported the original Expect script to Perl version and added few features.
Following is the script and I have provided some explanation below the code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

use strict;
use Expect;
 
# Uncomment the following line if you want to see what expect is doing 
#$Expect::Exp_Internal = 1;
 
# Uncomment the following line if you don't want to see any output from the script
#$Expect::Log_Stdout = 0;
 
# set your username/password here
my $sftpUsername = "username" ;
my $sftpPassword = "password"; 
my $sftpServer = "yourserver";
my $fileToFetch = "myfile";
my $timeout = 10; 
 
# If sftp is not in your path replace with absolute path of sftp program
my $command = 'sftp';
my $params = ("$sftpUsername\@$sftpServer");
 
# Create the Expect object
my $exp = Expect->spawn($command, $params) or die "Cannot spawn sftp command \n";
 
# If this is the first time you are running this , expect will send "yes" to add the key
# for the sftp server to the ~/.ssh/known_hosts file else
# wait for "Password Authentication" string to show up
$exp->expect($timeout,
	["Password Authentication"],
	["Are you sure you want to continue connecting", sub {my $self = shift; $self->send("yes\n");}]
	);
 
# Wait for Password prompt to show up 
$exp->expect($timeout, ["Password:"]);
 
# Sent the sftp password 
$exp->send("$sftpPassword\n");
 
# Wait for sftp prompt
$exp->expect($timeout, ["sftp>"]);
 
# Get yesterday's report file/s and put it in reportsPath directory on the local machine
$exp->send("get $fileToFetch\n");
 
# Wait for sftp prompt
$exp->expect($timeout, ["sftp>"]);
 
# Close ftp session
$exp->send("bye\n");
 
# Destroy the expect object
$exp->soft_close();

You need to install Expect from cpan if you don’t have it installed. To check if the Expect is installed you could check it in your terminal by executing this “perl -MExpect -e1″.
On line 22 I am creating an Expect object using using the spawn function. spawn takes the command to execute as the first argument and the parameters to the command as an array of arguments, hence $params is an array containing single element which is “username@sftpserver”.
Now when the script is run for the first time, there is a prompt to save the key of the sftp server in the ~/.ssh/know_hosts file and if the entry for the remote sftp server is already present in the known_hosts file a “Password Authentication” prompt will be sent by the sftp server. Both these situations have to be handled in the Expect script. Fortunately its very easy to to write handlers for these two possibilities in a single expect function call. The expect function can take any number of possible strings to expect from the remote server at a given point. You can see this on line 28 & 29, each possible string is provided as an array reference. And if there is an action that you want to perform corresponding to a particular string you can do this by providing a subroutine to handle this. You could either pass a reference to a subroutine or provide an anonymous subroutine. On line 29 I have provided an anonymous subroutine to send “yes” when prompted for accepting the key from the sftp server. Its also possible to provide the regular expression instead of literal string matches to the expect function.
The rest of the script is self explanatory.

Read the Expect module documentation on cpan for more information :
http://search.cpan.org/~rgiersig/Expect-1.21/Expect.pod