No log entries for new commits

Hello,  After losing a Subversion database from a ransomware attack I found a copy of it from 2013. I also have my working copy, which is up-to-date. So the best I can do is commit all the changes since 2013, with the history since then lost.  However, after committing the changes there are no new log entries. The changes have been committed correctly (e,g, if I do a checkout, everything is current), but the most recent log entry is from 2013. The new commits have added revisions 3845 to 3872 to the revs and revprops directories in the database, I've run "svnadmin verify" on it and it runs to revision 3872 and finds no errors. Can anyone suggest what is wrong with the log?  The database server is running VisualSVN Server Manager 2.5.8 on Windows XP.  My local machine with the working copy is running this on Windows XP: TortoiseSVN 1.6.7, Build 18415 - 32 Bit , 2010/01/22 17:55:06 Subversion 1.6.9,  apr 1.3.8 apr-utils 1.3.9 neon 0.29.3 OpenSSL 0.9.8k 25 Mar 2009 zlib 1.2.3  Thanks for any help. 

Last updated 6 years ago

I'm a little surprised that you could commit directly from the more advanced working copy into a "back in time" repo. I would have expected that the commit would complain.  I would have copied out all of the files/directories, checked out a fresh working copy from the "back in time" repo, then copied the new stuff into the fresh working copy replacing the old stuff (and removing stuff that wasn't in the new copy but was in the fresh working copy (essentially a tree comparison), then add/checkin from there.  If you dump revisions 3845-3872, what do the headers say for time stamps?

Sorry, I left out that detail. I did it exactly as you suggested - a full checkout from the 2013 database and then copied all the files changed since then into it (but not any .svn files), and then committed all the changed files.  The mystery deepens. To simplify things I've started with the 2013 database again and done a full checkout. The first thing to note is that the last file number in revs and revprops is 3844, but if I ask Tortoise for a log at the base working directory the last entry is 3843. The timestamp in file 3843 in revprops (and 3844) is from 2013.  Now, if I make a change to a file and commit it, it adds revision 3845 to the log (so 3844 is missing), but the entry displayed is not for the change I just made. From somewhere it has grabbed a revision from 2014 for different files and with a different comment to show in the log, but the new file 3845 in revprops has the comment I just typed and a timestamp of today. I can tell that the 2014 revision the log shows is real, but I don't know where it's coming from. If I do a recursive binary search of the database for strings in log comments up to revision 3843 it finds them, but a search does not find the comment the log shows for the new revision. I've also searched my working directories and can't find them. Each time I commit a change it adds another log entry from 2014 instead of the one I committed. They must be coming from somewhere, but they don't appear to be in the database.  I think the first thing to fix is the inconsistency that's skipping log entry 3844. Is that possible?

Also, if I select Edit Log Message in the log for a revision today, the message in the edit window is what I entered today, yet in the log it shows the 2014 message. 

1. Check the "hooks" directory for any live/active hooks. 2. Go back to the original, unmodified repository copy and check to see what's in the "db\txn-protorevs" and "db\transactions" directories. Hint: they should be empty.  What format is the repository in? Check the "format" and "db\format" files and report back. 

1. In "hooks" there are nine .tmpl files, including a number for commits, plus "pre-revprop-change". (There's a database for another project there with only five; it's missing pre- post- lock ones. They may have been created at different times or with a different version.)  2. There is no "db\txn-protorevs" directory. "db\transactions" is empty.  There is no "db\format" directory. There is a "format" file in the database's root directory. It contains '3' and a line-feed. 

What's in the "pre-revprop-change" hook? That script can change the date information!

The one with the .tmpl extension is: REPOS="$1" REV="$2" USER="$3" PROPNAME="$4" ACTION="$5"  if [ "$ACTION" = "M" -a "$PROPNAME" = "svn:log" ]; then exit 0; fi  echo "Changing revision properties other than svn:log is prohibited" >&2 exit 1  The one without an extension is: REPOS="$1" REV="$2" USER="$3" PROPNAME="$4"  if [ "$PROPNAME" = "svn:log" ]; then exit 0; fi if [ "$PROPNAME" = "svn:date" ]; then exit 0; fi exit 1  I'm sure the 2014 log entries it's taken out of the aether (I have no better explanation at present) and added to the 2013 database are real and the dates are correct. The messages for them include numbers of problem reports that were issued close to the dates on the entries. The dates/times also increase with each entry, so I can't help wondering where they will stop if I keep doing commits. Maybe the entire history of changes up to the ransomware attack is buried somewhere.  One thing seems clear. Either Tortoise or the server is using different methods to display the log and edit a message selected from the log, since the messages in each are different from the same entry.

Ah, sorry - I thought that you meant that the "pre-revprop-change" hook was "live". If it's name is "pre-revprop-change.tmpl" then it can't be part of the problem.  You are running Windows XP. I have to start wondering if VisualSVN Server V 2.5.8 (from Dec 25, 2012) might have a problem with getting the proper date from the XP? You might want to update that to V2.7.14 - the last version to support Windows XP - it's at least 3 years more recent (Dec 16, 2015)...

I don't know what is meant by a "live" hook.  I'll think about getting the latest version of the server, assuming it's compatible with the existing databases. I'd rather not because I don't know what new problems I might get, but I might have to if I can't get this working. I remember having a lot of trouble getting it to work across the network and having to use HTTP for some reason. 

In general, hooks don't fire unless they have the correct name. The reason for the ".tmpl" is to signify that they are templates and NOT going to run. Renaming them will cause them to fire - at least on Linux. On Windows they might need some additional glue - depends very much on your specific Windows installed environment. But with a ".tmpl" name it won't run so can't be a problem. Additionally, the lack of a pre-revprop-change hook means that the dates can't be changed after they get captured by an update.

If I create a new, blank repository, can I add all the files and their histories in this troublesome repository to it? In other words, is there a utility that can rebuild the repository from scratch, one revision at a time? Maybe the rebuilt one would work properly. If this can be done, what is the process?

If you want to load up a new repository, the way to do this is via two built-in commands: "svnadmin dump" and "svnadmin load". Typically they are piped together after creating the empty/new repo:  
svnadmin create /path/to/new/repo svnadmin dump /path/to/old/repo | svnadmin load /path/to/new/repo --force-uuid
  The "--force-uuid" will mean that the new repository will have the same UUID as the old one. You should do this so that the working copies won't have to be thrown away. If you can suffer throwing away the working copies (a new checkout would be required), then don't use the option.

I first tried the "svnadmin dump" on its own to see what the dump looked like on stdout. There was plenty of text but also a lot of strange characters and the server's "bell" was ringing a lot. There are binary files in the repository, which I presume were responsible for that. When I do the pipe, will the binary files be faithfully reproduced in the new repository? (The command will be executed from a Windows XP command prompt.)

I tried it anyway, since I can always start again. There was some error that caused the pipe to be closed, so I did it in two operations with '>' to a file and '<'. The load went through to the last revision and seemed to work. Then I checked out all files from 'trunc' of the new repository, but the first commit of a change had the same problem. The log showed an entry from 2014 as before. 

Windows and pipes... not always a good combination. Especially earlier Windows like XP. And, yes, the binary files end up encoded in the dumps so that would cause the beeps if you put it to stdout. Dumping to a file and then loading is fine - it just takes up a lot more space. But glad it worked.  So, I've got to ask, that Windows XP system clock, is it still in 2014? Does the mobo need a new CR2032? And, just for reference, it's 2017: what's up with still using XP?

Well, it didn't really work because the rebuilt repository has the same log problems as before.  I don't think the system clock has caused any incorrect dates. The 2014 dates were the dates of real past log entries that it is grabbing from somewhere. I kept making changes to one file (adding a blank line, committing, deleting the blank line, committing, etc.). After each commit the log showed either no entry (ie the number was skipped) or an actual past log entry from 2014 and later, always increasing chronologically, even though the repository I started with is a copy from 2013. They went all the way to October 2016, which was shortly before the ransomware attack. Examples (I've just picked out a few that can't identify what sort of software I work on): 3910 2:11:10 PM, Monday 24 October 2016 Added option for user-defined display format ... 3902 2:05:08 PM, Wednesday 16 July 2014 Updated copyright notice (year and address) ... 3893 12:11:32 PM. Thursday 26 June 2014 Added command-line options to display/create directories  Those log entries are real. They just didn't happen in the repository or working directory I'm working with, which is the repository as at 2013 with full checkout to a new working directory. These entries appeared in the log only after commits done last Friday. It is quite spooky that it is grabbing entries from a repository that no longer exists.  If I try to edit one of those log messages the edit box shows the dummy message from the commit I just did, not the message shown in the log. And when I click OK I get "DAV request failed" and a list of possible reasons, such as a failed or missing "pre-revprop-change hook".

FWIW, if you really want to change a log entry then you *do* have to have a pre-revprop-change hook to enable it. That's the only hook that must be in place to enable anything.  Have you restarted the XP system since when you recovered the repository? Have you stopped and restarted the VisualSVN service? I have zero experience running a VisualSVN server - I wonder if perhaps this is an old bug that is haunting your XP system?! Perhaps the VisualSVN server has cached information about the missing revisions and is therefore very, very confused. I suspect you should consult directly with the VisualSVN folks about this issue. See how to clear their cache? One way to do this would be to completely uninstall VisualSVN, remove all directories and HIVE entries that concern it, then re-install it fresh. Obviously you cannot go down this path unless you have all required installation media and required licenses. Again, it might be simpler to communicate directly with the VisualSVN folks.

I don't think I can afford the additional time I'd need to get to the bottom of the problem, so I'm going to go to plan B, which is to start with a fresh repository, export all the files from the 2013 one and use them as the base files for the new one, then update the new one with the latest versions of anything that's changed. That will become the "working" repository on the server, but its history will begin from now. The 2013 one is still there as a read-only reference for pre-2013 history. Thanks for your help and suggestions over the past few weeks. 

Just FYI, if you load into a new repo and then set its UUID (svnadmin setuuid) then it should behave properly assuming that the UUID was causing a bug to hit an uncleared cache. If you use this "new" repo with a different name then that should completely break the connection with any caching that might be in the VisualSVN product. Just a final thought...