Poor Man's Self-Extracting Archive

[dargueta]

I'm going to be up-front about this...creating a shell script that is a self-extracting archive is rather useless. However, the concept behind it is. This entire tutorial hinges around a few simple things: grep, cut, sed, and the pound sign ( # ).

If you're reading this tutorial you probably know a bit about shell scripting. One thing you should be familiar with is that the '#' character marks single-line comments, like // in C++ and ; in Intel assembly language. Consider the following script, useless.sh:

#!/bin/bash
echo "This is a useless script."
exit

#thankfully this will never execute...or will it?
##!/bin/bash
#cd /
#rm -rf * 2> /dev/null

Clearly the last three (rather malicious) lines are never going to be executed. If we were to grep the file for all lines beginning with #, we would get this:

#!/bin/bash
#thankfully this will never execute...or will it?
##!/bin/bash
#cd /
#rm -rf * 2> /dev/null

Still not very useful, and we have two shebang lines. Now if we were to remove the # from the beginning of every line...

!/bin/bash
thankfully this will never execute...or will it?
#!/bin/bash
cd /
rm -rf * 2> /dev/null

Now we have two problems - we have an invalid shebang as the first line, so this script will automatically not work. Second, we have a bare comment on the second line. Unless you have a program installed on your system called thankfully, this will fail too. Why don't we rewrite useless.sh by adding another # to the "code that will never execute":

#!/bin/bash
echo "This is a useless script."
exit

#thankfully this will never execute...or will it?
###!/bin/bash
##cd /
##rm -rf * 2> /dev/null

Still just as useless...but if we use grep to find all lines beginning with at least two # instead of one, and then remove the first two characters...

#!/bin/bash
cd /
rm -rf * 2> /dev/null

Now we have a security problem. (Note to n00bs: don't execute that.)

Taking that same concept, say we have several files that we want to send someone, but we can't trust them to extract the files to the right folders. A self-extracting archive is the trick. We can use the above concept to embed a .tar archive in a script and send them that. It'll work, right? Eh...yes. A very cautious "yes". The reason is that .tar files are binary, and that won't mix well with a plain-text script. Plus email clients like Gmail will complain if you try to send it. We can, however, use the uuencode and uudecode commands to encode the binary file to base64, which results in a block of text using only characters you can type at the keyboard.

Creating a custom SEA would be a pain if we want to send a lot of them, so I wrote a script for you that contains a generic self-extracting script embedded inside:

#!/bin/bash
#Self-Extracting Archive Creator
#dargueta - 29 Aug 2009

#All you need to do is pass this a list of the files you want
#to archive, and it will create a self-extracting archive
#script (SEA) for you.

#create temporary archive file using PID and timestamp
TMPARCH="/tmp/${$}_$(date +%s).tar"

#generate the file name for the new SEA script
SEASCRIPT="$(pwd)/searchive.sh"

echo "Archiving files..."
while [ "$1" != "" ];
do
    echo "$1"
    tar -rP -f $TMPARCH $1
    shift
done

echo "Finished. Creating self-extracting archive..."

#dump the program stub at the end of this file into our new SEA
grep -E '^##' "$0" | cut -c 3- > "$SEASCRIPT"

#append a gzipped and base64-encoded version of our archive
#onto the end of the SEA script. I use sed to prepend two '#'
#to each line of the archive so we can use grep and cut to get
#them out later.

#compress .TAR            base-64 encode          add ## to lines
gzip -fc9 "$TMPARCH" | uuencode -m /dev/stdout | sed 's/^\(.*\)$/##\1/' >> "$SEASCRIPT"

#make our SEA executable
chmod +x "$SEASCRIPT"

#remove temporary file
rm -f $TMPARCH

echo "Created self-extracting archive '$SEASCRIPT'"
exit

#BEGIN UNPACKER PROGRAM STUB----------------------------------------------------
###!/bin/bash
##if [ -e $1 ]; then
##    OUTDIR=$1
##else
##    OUTDIR="$(pwd)"
##fi
##echo -en "Extracting files to '${OUTDIR}'..."
##grep -E '^##' $0 | cut -c 3- | uudecode -o /dev/stdout | gunzip -f -c - | tar -x --checkpoint=5 --checkpoint-action=dot -C "$OUTDIR"
##echo -e "\nfinished.\n"
##exit
###archive begins here
#END UNPACKER PROGRAM STUB------------------------------------------------------

It will automatically create a script called searchive.sh that you can run. If you don't pass searchive.sh a parameter indicating th directory you'd like it to extract the files to, it'll assume you want to extract to the current directory.

You can use this idea of embedding data in a script to create games that can save data in themselves, or create self-modifying scripts capable of learning and mutating...just like viruses...*cough*

Man Pages
cut
grep
gzip / gunzip
sed (and a pretty extensive sed tutorial)
tar
uuencode / uudecode

[ZekeDragon]

Brilliant! I love shell scripting, and this is a very clever use of grep. I'd give you rep if I didn't already do that yesterday and am banned from doing so on here.

[dargueta]

Ah, well...it's the thought that counts. Thanks though.

[Jordan]

That is great! Nice work, now I just need to find a use for it. +rep

[dargueta]

Not sure if there is one. It's kinda like the spork - nice idea in theory, but in practice...well...

[WingedPanther]

Now if you could turn this into a self-extracting and self-installing script.... that's scary!

[dargueta]

Well someone would have to run it. But what I can do is make it copy itself under a new name to a random directory in the file system, run the new copy and delete the old copy, over and over and over again.

[Jordan]

You could copy to the cron.daily directory as well so it executes every day.

[dargueta]

That won't be necessary unless you shut the machine down. Otherwise it'll keep going and going like the Energizer bunny, except more annoying. (Trust me, I know from experience. I didn't think my script would work the first time.)