Howto : Resume a failed copy command where it left off

Currently, file copying utilities such as cp, scp, rsync etc. are not able to resume where they left off after a failed copy operation. In this article, a solution is provided using the utility dd to pick up where cp left off. Read on for more.


Installation
To install recp, type the following commands in a terminal

wget http://www.hartvig.de/files/recp
mv recp ~/bin/
chmod +x ~/bin/recp

Usage
Lets assume, that you have tried to copy file1 to file2 with the command

cp file1 file2

and that the copying process has somehow been interrupted. Now, you can simply resume the copying with the command

recp file1 file2

Issues
Please be aware, that since the script uses dd to continue copying it is both processor intensive and slow.

Code

#!/bin/bash

#set -x

if [ $# -ne 2 ]
then
	echo $0: Usage: $0 original copy
	exit 1
fi

# We need some more meaningful names.
ORIGINAL="$1"
COPY="$2"

if [ ! -f "$ORIGINAL" ]
then
	# ORIGINAL wasn't found.
	echo $0: "$ORIGINAL": No such file
	exit 1
fi

# Calculate the number of bytes to skip before we start copying.
if [ -f "$COPY" ]
then
	# This is a continuation of earlier, interrupted copy.

	WC_OUTPUT=$(wc --bytes "$COPY")

	# Since wc pads its output with spaces making it hard to reliably
	# parse, we have to do a little hack job:

	# Make sure there's at least one space at the beginning.
	WC_OUTPUT=" $WC_OUTPUT"

	# Sqeeze out excess spaces, making the size the second field.
	SKIP_BYTES=$(echo "$WC_OUTPUT" | tr -s ' ' | cut -d' ' -f2)
else
	# This is the first attempt at copying--there's no COPY yet.
	SKIP_BYTES=0
fi

# Do the actual copying.
dd if="$ORIGINAL" of="$COPY" conv=notrunc bs=1 skip="$SKIP_BYTES" seek="$SKIP_BYTES"

Credits:
Forum post on inuxquestions.org

Be Sociable, Share!

7 thoughts on “Howto : Resume a failed copy command where it left off

  1. Pingback: Omnigia: Scheme, web applications, tech » Resuming a file copy operation

  2. Trevor Joynson

    Slight variation:

    #!/bin/bash

    death() { echo “$0:” “$@”; exit 1; }

    [ $# -eq 2 ] || death “Usage: $0 original copy”
    SRC=”$1″; DST=”$2″

    # Got SRC?
    [ -f "$SRC" ] || death “$SRC: No such file”

    # Calculate the number of bytes to skip before we start copying. Default to zero.
    SKIP=0; [ ! -f "$DST" ] || SKIP=$(wc –bytes “$DST” | awk ‘{print $1}’ )

    # Do the actual copying.
    dd if=”$SRC” of=”$DST” conv=notrunc bs=1 skip=”$SKIP” seek=”$SKIP”

    Reply
  3. Trevor Joynson

    Using a blocksize other than 1 helped me out with CPU usage quite a bit, and acted as a nice buffer across my network transfers, ala mbuf esque. Sorry to spam your blog, but here’s my final variation if anyone’s interested:

    #!/bin/bash -e

    # Blocksize to truncate to and buffer by
    BS=${BS:-4096}

    death() { echo “$0:” “$@” >&2; exit 1; }

    [ $# -eq 2 ] || death “Usage: $0 original copy”
    SRC=”$1″; DST=”$2″

    # Got SRC?
    [ -f "$SRC" ] || death “$SRC: No such file”

    # Calculate the number of bytes to skip before we start copying. Default to zero.
    SKIP=0; [ ! -f "$DST" ] || SKIP=$(( $(wc –bytes “$DST” | awk ‘{print $1}’ ) / $BS ))

    # Do the actual copying.
    echo dd if=”$SRC” of=”$DST” conv=notrunc bs=$BS skip=”$SKIP” seek=”$SKIP”

    Reply
  4. Kevin Chadwick

    I didn’t want to wait for the read and found atleast installed on OpenBSD by default:

    /usr/bin/ftp -C -o “$DST” file:///”$SRC”

    I guess ftp –reget should work on linux

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>