Well, with no guarantees or promises whatsoever...here's my first
attempt. I'm certain someone else can come up with a much more robust
solution but it's a starting point.
Since I typically see a batch of the error messages during an FTS
update, my current usage is:
1. doveadm fts rescan -u user-to-fix
2. Perform a fts search to do a full mailbox scan
3. Check the mail log to see the date/time of the errors (I usually
have a window open with tail -f)
4. Something like "grep 'Mar 3 20:17' mail.log >
mail.err" gives me a
starting point
execute dovesisfix and see if it helps.
--
Daniel
On 3/3/2015 1:08 PM, Daniel Miller wrote:> This seems simple enough...I'm just not script wizard. If someone can
> throw together a starting point I can test and tweak it from there.
> It seems to me:
>
> 1. Read /var/mail/mail.err or specified logfile
> 2. For each "failed:
> read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line,
> a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists
> i. If attachmentHash is missing display such for possible
> backup searching.
> b. create link attachmentHash-userHash to hashes/attachmentHash
> 3. Continue to end of file
>
> Can this be done via "pure" BASH? Need sed/awk as well?
>
-------------- next part --------------
#!/bin/bash
# These variables need to be customized for your particular installation
LOGFILE='/var/log/mail.err'
ATTACHMENT_STORAGE_BASE='/var/mail/attachments'
# These variables are based on current Dovecot behaviour and should not require
changing
HASH_FOLDER='hashes'
# Initialization
PREVIOUS_ERR=''
ERR=''
function usage
{
echo "Dovecot Single-Instance-Storage Attachment Repair"
echo "usage: dovesisfix [-d] [-t] [-v] [-h]"
echo " -t | --test-only perform logfile analysis and show steps to be
taken without any on-disk modification"
echo " -v | --verbose provide verbose messages at each step"
echo " -d | --debug provide additional debug messages"
echo " -h | --help this screen"
}
while [ "$1" != "" ]; do
case $1 in
-d | --debug ) DEBUG=1
VERBOSE=1
;;
-t | --test-only ) TESTMODE=1
;;
-v | --verbose ) VERBOSE=1
;;
-h | --help ) usage
exit
;;
* ) usage
exit 1
esac
shift
done
while read -r LINE
do
ERR=$LINE
# Format of log line has date, host, process, user, mail storage file, and
then the
# attachment path failure, followed by a duplicate of the path as an
argument to open,
# and then final details.
# Verify this line is indeed a dovecot attachment error. Don't look for
"dovecot" specifically
# in case that name was changed - but the individual worker names are
probably safe searches.
# So we test against "attachments-connector" - hopefully
that's good enough.
TEST=$(echo "$ERR" | sed -n
"s|.*attachments-connector.*|1|p")
if [ "$TEST" != "1" ]; then
# Not found - not relevant
if [ "$DEBUG" = 1 ]; then
echo "Skipping non-relevant log line. $ERR"
fi
continue
fi
# Remove prefacing details from log line - find start of attachment path
within log line
# This is a greedy match - so the second attachment path is returned along
with the trailing info
ATTACH_LINE_FILTER="s|.*$ATTACHMENT_STORAGE_BASE||"
ATTACH_LINE=$(echo "$ERR" | sed "$ATTACH_LINE_FILTER")
# Now extract the aa/bb/ prefix, the base attachment file name, and user
hash
CATEGORY_PATH="${ATTACH_LINE:1:5}"
BASE_HASH="${ATTACH_LINE:7:40}"
USER_HASH="${ATTACH_LINE:48:32}"
ATTACH_SOURCE="$ATTACHMENT_STORAGE_BASE/$CATEGORY_PATH/$HASH_FOLDER/$BASE_HASH"
ATTACH_TARGET="$ATTACHMENT_STORAGE_BASE/$CATEGORY_PATH/$BASE_HASH-$USER_HASH"
# There appear to be duplicate lines - so to try to filter some out.
if [ "$PREVIOUS_TARGET" = "$ATTACH_TARGET" ]; then
if [ "$DEBUG" = 1 ]; then
echo "Skipping duplicate log line for
$ATTACH_SOURCE-$ATTACH_TARGET"
fi
continue
fi
PREVIOUS_TARGET=$ATTACH_TARGET
# If in debug/verbose mode show operation about to occur
if [ "$VERBOSE" = 1 ]; then
echo "The file $ATTACH_SOURCE must be linked to
$ATTACH_TARGET"
fi
# Verify that source exists
if [ ! -f "$ATTACH_SOURCE" ]; then
echo "ERROR: File $ATTACH_SOURCE does not exist. You must restore this
from a backup and run this utility again."
fi
# This is a Good Thing.
if [ "$DEBUG" = 1 ]; then
echo "The file $ATTACH_SOURCE appears to be a valid file."
fi
# Check if user link mysteriously reappeared
if [ -f "$ATTACH_TARGET" ]; then
echo "INFO: File $ATTACH_TARGET exists. This may mean the fault has been
previously corrected. Clearing/rotating the logfile $LOGFILE is appropriate
now."
continue
fi
# Prepare to create user link
LINK_LINE="$ATTACH_SOURCE $ATTACH_TARGET"
if [ "$DEBUG" = 1 ]; then
echo "About to execute command: ln $LINK_LINE"
fi
# If test mode, do nothing
if [ "$TESTMODE" = 1 ]; then
continue
fi
# There's probably more tests I could/should do - but I don't know
how
# So...if we're not in test mode...time to do it to it.
LINK_CREATED=$(ln $LINK_LINE)
if [ "$VERBOSE" = 1 ]; then
echo "Repair result for $ATTACH_TARGET - $LINK_CREATED"
fi
done < "$LOGFILE"
