2008-09-24:
Perl Non-blocking Read On Pipes Or Files

Let's say you want to run a command in the "background" and check it occasionally (asynchronously) for full lines of output.

You can do a simple open with a pipe, but if you try to read the pipe it's blocking. And once you actually figure out how to do a non-blocking read, you get back chunks of data, not necessarily in happy complete lines of output, which is unfortunate if you want to, say, use regexps on the output.

This routine solves that problem by doing non-blocking reads on a filehandle and assembling the complete lines of output for you.

# An non-blocking filehandle read that returns an array of lines read
# Returns:  ($eof,@lines)
my %nonblockGetLines_last;
sub nonblockGetLines {
	my ($fh,$timeout) = @_;

	$timeout = 0 unless defined $timeout;
	my $rfd = '';
	$nonblockGetLines_last{$fh} = ''
		unless defined $nonblockGetLines_last{$fh};

	vec($rfd,fileno($fh),1) = 1;
	return unless select($rfd, undef, undef, $timeout)>=0;
	return unless vec($rfd,fileno($fh),1);
	my $buf = '';
	my $n = sysread($fh,$buf,1024*1024);
	# If we're done, make sure to send the last unfinished line
	return (1,$nonblockGetLines_last{$fh}) unless $n;
	# Prepend the last unfinished line
	$buf = $nonblockGetLines_last{$fh}.$buf;
	# And save any newly unfinished lines
	$nonblockGetLines_last{$fh} =
		(substr($buf,-1) !~ /[\r\n]/ && $buf =~ s/([^\r\n]*)$//) ? $1 : '';
	$buf ? (0,split(/\n/,$buf)) : (0);
}

I've tested it with a pipe I opened using an IO::File, YMMV. Example usage:

$fh = new IO::File; open($fh,"$cmd 2>&1 |");

do {
	($eof,@lines) = nonblockGetLines($fh);
	foreach my $line ( @lines ) {
		print "Pipe saw: [$line]\n";
	}
} until $eof;

close $fh;

I stole the select/vec code and wrote the rest. Otherwise you are free to use this code however you like, any of my work in the above nonblockGetLines code is open source.

Enjoy!

Back to Solutions.