\cat programming cat: unix

UNIX Tips

Cleaning up your /tmp...The safe way by Guy Geens

Writing portable scripts

Using /usr/bin/env
Perl
Tcl

One-liner urldecode functions

awk
GNU sed
GNU Bourne-Again Shell (bash) and sed

I/O redirection

Cleaning up your /tmp...The safe way by Guy Geens

Put the following script (cleantmp) in root's crontab:

#! /usr/bin/perl -w
# cleantmp: Remove old files from /tmp partition
# Copyright (C) 1997 by Guy Geens
# Snail Mail:
# Zwijnaardsesteenweg 183
# 9000 Gent
# Belgium

use File::Find;

# Security measure: chroot to /tmp
$tmpdir = '/tmp/';
chdir ($tmpdir) || die "$tmpdir not accessible: $!";
if (chroot($tmpdir)) {		# chroot() fails when not run by root
    ($prefix = $tmpdir) =~ s,/+$,,;
    $root = '/';
    $test = 0;
} else {
# Not run by root - test only
    $prefix = '';
    $root = $tmpdir;
    $test = 1;
}

@list = ();

&find(\&do_files, $root);
&find(\&do_dirs, $root);

if (@list) {
    print "Cleaned $tmpdir\n";
    print "Deleted files are:\n";
    for (sort @list) {
	print "$prefix$_\n";
    }
}

exit;

sub do_files {
    (($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) &&
	(-f _ || -l _ ) &&
	    (int(-A _) > 3) &&
		! /^\.X.*lock$/ &&
		    &removefile ($_) && push @list, $File::Find::name;
}

sub do_dirs {
    /^\..*-unix$/ && ($File::Find::prune = 1) ||
	(($dev,$ino,$mode,$nlink,$uid,$gid) = lstat($_)) &&
	    -d _ && ($nlink == 2) &&
		! /^lost\+found$/ &&
		    &removedir ($_) && push @list, "$File::Find::name/";
}

sub removedir {
    if ( $test ) {
	1;
    } else {
# Can't use @_: rmdir doesn't take a list argument
	rmdir $_[0];
    }
}

sub removefile {
    if ( $test ) {
	1;
    } else {
	unlink @_;
    }
}

Writing portable scripts

Using /usr/bin/env

On some machines, the user is not allowed to execute /usr/bin/env or it's not available. Even worse is that options cannot be passed to an interpreter.

#!/usr/bin/env perl
...

Perl

http://lists.gnu.org/archive/html/bug-coreutils/2004-05/msg00173.html

By setting the PERL environment variable, one can override the default perl version.

#!/bin/sh
exec "${PERL:-perl}" -x -S "$0" ${1+"$@"}
#!perl -w
...

By the way, what does ${1+"$@"} mean?

man perlrun: If the program will be interpreted by csh, you will need to replace "${1+"$@"}" with $*, even though that doesn't understand embedded spaces (and such) in the argument list.
Chapter 6. Social Engineering: A better construct than $* would be ${1+"$@"}, which handles embedded spaces and such in the filenames, but doesn't work if the script is being interpreted by csh.

For CGIs which are usually out of PATH, the following code (without -S) would be more portable:

#!/bin/sh
exec perl -x "$0" "$@"
#!perl
...

Tcl

#!/bin/sh
# the next line restarts using tclsh \
exec "${TCLSH:-tclsh} "$0" "$@"
...

One-liner urldecode functions

awk

#!/bin/sh
# (C) 2006 GPL by Huidae Cho
awk '
BEGIN{
	for(i = 0; i < 10; i++)
		hex[i] = i
	hex["A"] = hex["a"] = 10
	hex["B"] = hex["b"] = 11
	hex["C"] = hex["c"] = 12
	hex["D"] = hex["d"] = 13
	hex["E"] = hex["e"] = 14
	hex["F"] = hex["f"] = 15
}
{
	gsub(/\+/, " ")
	i = $0
	while(match(i, /%../)){
		if(RSTART > 1)
			printf "%s", substr(i, 1, RSTART-1)
		printf "%c", hex[substr(i, RSTART+1, 1)] * 16 + hex[substr(i, RSTART+2, 1)]
		i = substr(i, RSTART+RLENGTH)
	}
	print i
}
'

OK, OK, I said "One-liner" urldecode functions. Try this! :-)

$ alias urldecode="awk '"'BEGIN{for(i=0;i<10;i++)hex[i]=i;hex["A"]=hex["a"]=10;hex["B"]=hex["b"]=11;hex["C"]=hex["c"]=12;hex["D"]=hex["d"]=13;hex["E"]=hex["e"]=14;hex["F"]=hex["f"]=15;}{gsub(/\+/," ");i=$0;while(match(i,/%../)){;if(RSTART>1);printf"%s",substr(i,1,RSTART-1);printf"%c",hex[substr(i,RSTART+1,1)]*16+hex[substr(i,RSTART+2,1)];i=substr(i,RSTART+RLENGTH);}print i;}'"'"

$ echo "%31+%32%0A%33+%34" | urldecode
1 2
3 4

GNU sed

This script uses an extension of GNU sed (\xNN).

# (C) 2006 GPL by Huidae Cho
urldecode(){
  echo "$@" | sed 's/^.*$/'"`echo "$@" | sed 'y/+/ /; s/%/\\\\x/g; s/\//\\\\\//g'`"'/'
}

$ urldecode "%31+%32%0A%33+%34"
1 2
3 4

GNU Bourne-Again Shell (bash) and sed

It's extremely fast!

# (C) 2004 GPL by Huidae Cho
urldecode(){
  echo -e "$(sed 'y/+/ /; s/%/\\x/g')"
}

$ echo "%31+%32%0A%33+%34" | urldecode

$ echo -e "$(echo "%31+%32%0A%33+%34" | sed 'y/+/ /; s/%/\\x/g')"

outputs

1 2
3 4

Pure bash version: http://thijs.dalhuijsen.com/code/index.shtml?req=5

I/O redirection

Two good examples from the bash man page:

ls > dirlist 2>&1

open(dirlist, ...) = 3: open file
dup2(3, 1): duplicate fd 3 as stdout
close(3): close fd 3
dup2(1, 2): duplicate stdout as stderr

->> duplicate, -> output direction
1 <<- 3
2 <<- 1 (1 <<- 3 at the first step) == 2 <<- 1 <<- 3 == 2 -> 1 -> 3

ls 2>&1 > dirlist

dup2(1, 2): duplicate stdout as stderr
open(dirlist, ...) = 3: open file
dup2(3, 1): duplicate fd 3 as stdout
close(3): close fd 3

2 <<- 1
1 <<- 3 (3 doesn't have any link to 2) == 1 <<- 3 && 2 <<- 1 == 1 -> 3 && 2 -> 1

Programmatically, it's left to right, but actual stream is right to left.

ls > dirlist 2>&1: 2 -> 1, 1 -> dirlist == (2, 1) -> dirlist
ls 2>&1 > dirlist: 1 -> dirlist, 2 -> 1