SVN pre-commit hook

Hello all,  I'm trying to build a pre-commit script which I can control the commits of the repository. I want to block all commits bigger than 100 MB. I was trying it with that code, but it doesn't work.  #!/bin/ksh REPOS="$1" TXN="$2"  size=$(du -b -s "$REPOS"/db/transactions/"$TXN".txn | cut -f 1) if [[ $size -gt 104857600 ]]; then size=$(du -h -s "$REPOS"/db/transactions/"$TXN".txn | cut -f 1) echo "Your changeset requires a disk space ($size) exceeding our 100MB limit">&2 exit 1 fi   I also tryed another ways to get size of commit, but no one worked(if that´s the problem): #size=$(svnlook cat $REPOS -t $TXN | wc -c) #size=$(svnlook filesize -t $TXN $REPOS $f) Any idea?  Thank you very much!

Last updated 1 year ago

Strange to use Ksh in this day and age of Bash but the commands you've shown shouldn't be any different.  One thing that should be done is to verify that the PATH is set properly. When writing hooks scripts it's always best to set a PATH that is simple and covers all of the necessary tools since you can't depend on a reasonable PATH being set when the hook is run. For instance, did the inherited PATH enable access to the "svnlook" command? How are you debugging this?

Thanks for reply.  Im using Ksh, but I can also use bash, sintax is similar and I also tryed with Bash, but it didn´t work. PATH i think is working. The $REPOS has it. As you see, Im not using svn look, I just tryed with different ways to calculate size but noone worked for me. Also I dont know how to debug it...  What should I do?  Using bash: #!/bin/bash  REPOS="$1" TXN="$2"  size=$(du -b "$REPOS"/db/transactions/"$TXN".txn | cut -f 1)  if [ $size -gt "204800" ]; then #250MB en bytes  size=$(du -b "$REPOS"/db/transactions/"$TXN".txn | cut -f 1)  echo "Your changeset requires a disk space ($size) Kb exceeding our 250MB limit">&2  exit 1 else  exit 0 fi 

In general, the default PATH environment variable should be able to find "du", "svnlook", etc. since they're all normally in "/usr/bin" - but I don't assume that's where you've put it.  Yes, $REPOS is a path so that should be ok.  Have you debugged this at all? One debug technique would be to:

 
1. Always exit non-zero so that nothing works until you've squared things away. 
2. Output STDOUT/STDERR for all into a log file in "/tmp". This can be done by: 

#!/bin/bash --noprofile REPOS="$1" TXN="$2"  for x in 1; do set -x size=$(...) echo files in the TXN are: find "${REPOS}/db/transactions/${TXN}.txn" -type f -a -print0 | xargs -0 ls -ld done > /tmp/mylog.$$ 2>&1

    That output might help you analyze what's going on... For instance, are files compressed?

I will check your code.. thank you.  Files could be or not be compressed. mostly of them are not. They are .doc,xls,csv,pictures and some .zip But the thing is to avoid people to upload .zip higher than 100MB  I think the problem should be getting the size of the committed files, because script should work. Commands work separately. Just maybe therés another way to calculate the size of a commit.

I tryed your code, here´s the result:  [root@localhost tmp]# cat mylog.85476 ++ du -b /repositorio/tecdesoft/db/transactions/84507-1t9x.txn ++ cut -f 1 + size=159876 + echo files in the TXN are: files in the TXN are: + find /repositorio/tecdesoft/db/transactions/84507-1t9x.txn -type -f -a -print0 + xargs -0 ls -ld find: Arguments to -type should contain only one letter dr-xr-xr-x. 19 root root 257 Jul 11 15:50 .  Im trying to commit files about 1,25MB (exactly 1310720 bytes). When I try du -b file.pdf the result is exactly 1310720, totally different to 159876 which I see in log. So maybe that´s not the way to obtain the commit size. Any idea?

That ways does not work: #size=$(svnlook cat $REPOS -t $TXN | wc -c) #size=$(svnlook filesize -t $TXN $REPOS $f) fail...

So, I had a typo above in the "find" command-line. I've just edited my post and fixed it up. The error from the "find" command was quite explicit so easy to diagnose. Just in case it isn't obvious here's the code again:  for x in 1; do set -x echo "files in the TXN are:" find "${REPOS}/db/transactions/${TXN}.txn" -type f -a -print0 | xargs -0 ls -ld done > /tmp/mylog.$$ 2>&  Update your hook and try again please.

I think the problem is here:  size=$(du -b "$REPOS"/db/transactions/"$TXN".txn | cut -f 1)  I mean, I don´t get the proper value. It seems that there´s another way to get the size of a commit, and this way does not work. I see the values using your script, but the values are not the same than the "file size" of the files I want to upload to subversion. It doesnt get the REAL value.  Any idea to get the commit size?

Perhaps change the "du -b ..." to be "du -sb ..." - you need the size of the entire tree?!

I tryed both but no one worked. I would like to get the size of files bigger than 100MB, and block them to avoid multiple bigger files in the repository. Thats the goal of my script, but it does not work... it should be any way to get size of files

The "-s" option to the "du" command will calculate the size of the entire tree (rather than a list of 1 line per directory underneath - which will confuse your script).

Im using du -sb to get size, still not work. It does not get the proper size of commits. Always get 188K or similar values, but it doesnt get the real size of the commit. I don´t care if it works individual or entire tree, just would like to make it work...

Here´s the result of your script... I´m not getting the size of commit at end.  root@localhost hooks]# cat /tmp/mylog.19428 + echo 'files in the TXN are:' files in the TXN are: + find /repositorio/prueba/db/transactions/3-9.txn -type f -a -print0 + xargs -0 ls -ld -rw-rw-r--. 1 user user 101 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/7053669113ea0207878c1dab0902b46984528a83 -rw-rw-r--. 1 user user 231 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/changes -rw-rw-r--. 1 user user 4 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/next-ids -rw-rw-r--. 1 user user 92 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/node.0-1.0 -rw-rw-r--. 1 user user 184 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/node.0-1.0.children -rw-rw-r--. 1 user user 82 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/node.0.0 -rw-rw-r--. 1 user user 68 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/node.0.0.children -rw-rw-r--. 1 user user 218 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/node._1.0 -rw-rw-r--. 1 user user 53 Oct 7 09:31 /repositorio/prueba/db/transactions/3-9.txn/node._1.0.props -rw-rw-r--. 1 user user 266 Oct 7 09:30 /repositorio/prueba/db/transactions/3-9.txn/props 

I just would like to create a script which I can block people who is trying to upload files greather than 200MB... I can´t reach the way to get it

What "file" change are you making when testing this data? For instance, the most reliable way to do this check is to create a new file of 1MB via "dd if=/dev/urandom of=newfile bs=1m count=1" and then add that file. Since it will have never been in the repository before there can't be any "delta" magic going on to prevent you seeing the actual file size.  Give that a try and report back.

we are speaking about different things... good luck

Up to you. I think we are talking about the same thing. What might be missing is the fact that Subversion is computing delta's against files that already exist in the repository. So when your test case includes a file that already exists the "transaction" is much smaller than you might expect. So let's work around that by asking the system for the actual byte counts.  You can use "svnlook" to investigate the transaction (-t option). The following commands should be combined to get the information you need: - "svnlook changed" to get a list of the files changed (consider using the "--copy-info" option so you see everything. - "svnlook filesize" (iterate over the output of "svnlook changed") to get the size of the files.