Using the pre-commit hook to restrict the size of the submitted file failed

Dear Sir: How do you do?  1.Environment description: Visual SVN Server 2.7.13, installed on Windows Server 2007 SP2 operating system. Now I create a version library named test. It works fine.  2.My needs: I want to limit the size of the submitted files.  3.My operation: 1) I copy the file named pre-commit.tmpl and save it as file named pre-commit.sh in hooks directory. 2) Modify the file named pre-commit.sh as follows:  #!/bin/bash  REPOS="$1" TXN="$2"  SVNLOOK="C:/Program Files/VisualSVN Server/bin/svnlook" #SVNLOOK=/usr/bin/svnlook MAX_SIZE=512000  files=$($SVNLOOK changed -t $TXN $REPOS | awk '{print $2}')  # check check #if [[ $files =~ "project_nuli" ]];then for f in $files do  # check file size  filesize=$($SVNLOOK cat -t $TXN $REPOS $f | wc -c)  if [ $filesize -gt $MAX_SIZE ] ; then  echo "File $f is too large (must <= $MAX_SIZE)" >> /dev/stderr  exit 1  fi done #fi exit 0 3) Restart the SVN service  4.actual result: Then I submit a 46k file, but it succeeded. The new pre-commit hook seems not to work?  What’s wrong? Fervently hope expert’s advice. Your early reply will be highly appreciated.  Thank you so much!  Yours yipianlvye  June 30, 2018

Last updated 3 years ago

On Linux the name of the file must be exactly "pre-commit" (not "pre-commit.sh"). I'm not an expert on Windows, but the extension that you use must be registered with Windows as an executable. If you've got ".sh" so registered then you get to move on to the next part of debugging: checking to see what happened. I normally do this by turning on the "shell echo" capability (e.g. "set -x") and then making sure that the output is redirected to some temporary file that I can evaluate later. In general, it might look like:  set -x for x in 1; do  put your code here done > /path/to/some/temp/file 2>&1  Hopefully you'll then figure out what's going wrong.

Thank you for your reply. I’ve solved this problem with the help of RD engineers. They found that the script could not run, mainly because the two commands (awk and wc) under Linux did not have corresponding commands under Windows. They rewritten the script as a Windows script and the problem was solved.

Now I meet another question. Another repository named SW. URL: https://10.7.0.84:8443/svn/SW/软件事业部/01项目区/01工程项目/2017/甘肃掌上操作票专家系统项目/03.Code/3.2.Trunk/P70 Checkout failed, Error: missing update-report close tag  I execute ‘svnadmin verify E:/Repos/SW’ command on the server side, and the result is as follows:  Verified revision 63848.  svnadmin:E140001:zlib :corrupt data:Decompression of svndiff data failed It seems that the 63849 version data has been corrupted. Version 63849 is as follows:  If the version's data is corrupted, what can we do to fix it or remove it completely? The svnadmin dump operation also failed.

Use the dump and specify a range from 0-(N-1) and (N+1)-last. Then load up a new repo. This may not work if N+1 refers to N so you may need to iterate forward.  Any working copy referring to any is the renumberred revisions will need to be discarded and re checked out (save the work outside the WC.  Lot of pain. Repo corruption is very rare. You should look into why. Check windows error logs, etc. Prevent the issue for happening again. 

According to your method, the following commands are successful, and I get 3 dump files:  svnadmin dump E:\Repos\SW -r 0:63848 > G:\SW1.dump  svnadmin dump E:\Repos\SW -r 63850:66598 --incremental > G:\SW2.dump  svnadmin dump E:\Repos\SW -r 66600:88013 --incremental > G:\SW3.dump But the execution of the following command meets an error:  svnadmin load E:\Repos\SW1 < G:\SW1.dump  Error information is below: ---Committed revision 2327 >>>  <<< Started new transaction, based on original revision 2328  * editing path:软件事业部/01项目区/05测试库/Project1 ...svnadmin:E200020 :Invalid svn:mergeinfo value svnadmin:E200020:Unable to parse reversed revision range '96798-2321' svnadmin:E200020:Unable to parse reversed revision range '96798-2321'  What’s wrong? Hope for your suggest. Thank you so much!

That’s messed up meta-data. I’ve seen an old SVN bug that could have caused the issue, but I have no idea about Visual SVN. You should contact the Visual SVN folks for help.

The script does not get the filenames which contain blank spaces. Anyone know how to read filenames with spaces? I tryed including " ", ' ', and \ before each space. For example: It get the size of "file.pdf", but nor for "file 1.pdf". I tryed including " " to the name, like "file .pdf", 'file .pdf', "file\ .pdf"  Maybe SVNLOOK does not get filenames with spaces and there´s nothing to do...

Handling white-space within Linux scripting takes a lot of care in proper quoting. You have not shared the script so I cannot say where things went wrong.

That´s right, sorry! I forgot it :D  This is my script: #!bin/bash REPOS="$1" TXN="$2"  SVNLOOK=/usr/bin/svnlook MAX_SIZE=262144000 #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/") #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed 's/ /\\\$/g' | tr $ " ") files=$($SVNLOOK changed -t $TXN $REPOS | awk '{print $2}') #echo "ARCHIVOS: $REPOS $TXN $files" >> /home/user/prueba.log for f in $files do filesize=$($SVNLOOK cat -t $TXN $REPOS $f | wc -c) #echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log if [ $filesize -gt $MAX_SIZE ]; then echo "File $f is too large, must be lower than 250MB">&2 exit 1 fi done exit 0   I tryed different ways of get results about SVNLOOK changed (commented options). As you see, I insert " " (and also can insert ' ' just changing it on my script). I also tryed with inserting \ before each space. Im checking the results of the script into log files, and I´m introducing " " and ' ' properly, but SVNLOOK cat does not get it...  Anyway, using the loop will be a problem if it check spaces, because it won´t get the proper file, it will get each word after each space as a different word.

First, you could always use the "--xml" option to "svnlook" and then use an xml parser. You'd likely need to change languages away from bash.  Second, using "cut" is, well, tricky: I try to avoid it.  Third, because you're using "awk '{print $2}'" the data you're looking for is already broken since awk, by default, uses white-space to separate fields.  #!/bin/bash --noprofile REPOS="$1" TXN="$2"  SVNLOOK=/usr/bin/svnlook MAX_SIZE=262144000 $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | \ while read afile; do  case "$afile" in  */) continue;; # skip directories  esac  filesize=$($SVNLOOK cat -t "$TXN" "$REPOS" "$afile" | wc -c)  #echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log  if [[ $filesize -gt $MAX_SIZE ]]; then  echo "File $afile is too large, must be lower than 250MB">&2  exit 1  fi done exit 0  Give that a try (might be some syntax mistakes - I don't have time right now to test it.

I tryed it:  #!bin/bash REPOS="$1" TXN="$2"  SVNLOOK=/usr/bin/svnlook MAX_SIZE=104857600 #250MB  files=$($SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//') #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/") #files=$($SVNLOOK changed -t $TXN $REPOS | awk {'print $2'}) #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/\'/;s/$/\'/" | sed 's/ /\\\$/g' | tr $ " ") echo "ARCHIVOS:$files" >> /home/user/prueba.log  #for f in $files #do while read files do case $files in */) echo "CASE:$files" >> /home/user/prueba.log continue ;; esac filesize=$($SVNLOOK cat -t $TXN $REPOS $files | wc -c) #filesize=$($SVNLOOK filesize -t $TXN $REPOS $files) #filesize=$(du -bs $files | cut -f 1) echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log if [ $filesize -gt $MAX_SIZE ]; then echo "SE CUMPLE" >> /home/user/prueba.log echo "File $f is too large, must be lower than 250MB">&2 exit 1 fi done #done exit 0  It does not work. The script does not reach the filesize function. In that case, is not getting the size of the file. I don´t know why actually, I tryed it with filename without spaces and with spaces. No way to reach filesize... I think problem is in "while read", because if I use "for" instead, it works, but it does not go trhough case statment  any idea?  I think even if script works, the problem will be that SVNLOOK does not get file size if name contain spaces.

You can't use an inline function like $() when you need to parse file names that have white-space. So the "files=$()" is just plain not going to work. In the code you posed up there's no $ in front of the "files" in "while read files". And you're not feeding anything into it so no, it's not going to work.  Did you try using the script as I posted it above? What happened?

Yes, I tryed your script but it does not work...  because "afile" variable is empty. In your code, you did not create the variable or save information inside, so code is not working.  Im doing it like you now:  #!bin/bash REPOS="$1" TXN="$2"  SVNLOOK=/usr/bin/svnlook MAX_SIZE=104857600 #250MB #files=$($SVNLOOK changed -t "$TXN" "$REPOS" | cut -d A -f2 | cut -d U -f2 | cut -c4-) $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/"/;s/$/"/") #files=$($SVNLOOK changed -t $TXN $REPOS | awk {'print $2'}) #files=$($SVNLOOK changed -t $TXN $REPOS | cut -d A -f2 | cut -d U -f2 | cut -c4- | sed "s/^/\'/;s/$/\'/" | sed 's/ /\\\$/g' | tr $ " ") echo "ARCHIVOS:$files" >> /home/user/prueba.log  #for f in $files #do while read afile; do echo "WHILE:$afile" >> /home/user/prueba.log case $afile in */) echo "CASE:$afile" >> /home/user/prueba.log continue ;; esac  filesize=$($SVNLOOK cat -t $TXN $REPOS $afile | wc -c)  #filesize=$($SVNLOOK filesize -t $TXN $REPOS $files)  #filesize=$(du -bs $files | cut -f 1)  echo "PRUEBA CON FILESIZE: $filesize" >> /home/user/prueba.log  if [ $filesize -gt $MAX_SIZE ]; then  echo "SE CUMPLE" >> /home/user/prueba.log  echo "File $f is too large, must be lower than 250MB">&2  exit 1  fi #done < "$files" done exit 0 

In my script the line just before "while read afile; do" is:  $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | \  That line ends in a pipe followed by a backslash which should then be followed by a newline (EOL).  Or, you can combine those 2 lines:  $SVNLOOK changed -t "$TXN" "$REPOS" | sed -e '/^D/d' -e 's/^....//' | while read afile; do  NOTE: the quoting for the sed commands should be apostrophe (single quotes) - not something silly that a copy/paste might turn them into. Same with the double-quotes.

I tryed it. Now it react in different way.  The problem is filesize function returns 0 value. I guess It happens because the name contain spaces, and the function "SVNLOOK cat -t $TXN $REPOS $afile | wc -c" can´t get the file size if name contains spaces.  If name does not contain spaces, it works. Thats the thing now...

Here is the result of the script (I also tryed once to introduce " " to the name, as you can see. Also with single quotes).  Also tryed with the different ways of get file size (commented in the script).

filesize=$($SVNLOOK cat -t $TXN $REPOS $afile | wc -c)  should be:  filesize=$($SVNLOOK cat -t $TXN $REPOS "$afile" | wc -c)  Those are double-quotes around the $afile without any white-space.

Thank you very much! I should be care of that :D