SmartSVN 8.5: Adding multiple UTF-16 text files causes "Inconsistent EOL" error

[ATTACH]394[/ATTACH]  An error occurred while processing an SVN command  File ... has inconsistent newlines Additional errors  This makes SmartSVN 8.5 unusable to me, since I have lots of UTF-16 encoded text files.  PS: The file does have consistent newlines and SmartSVN 8.0 does not complain about them.

Last updated 8 years ago

Hi,  I'm trying to replicate this at present so I can raise a task for our dev team to look into this, if you could pass on a bit more information about what's happening it'd really help - specifically:  1) Which type of UTF-16 encoding are your files encoded with, or is it a mix? 2) How many files are there? 3) Are you committing to a local or a remote repository? 4) If remote, which protocol are you using? 5) Which OS are you using (for both client and repository server machines, if you're using a remote repo)? It looks like the client machine is running Windows 7, but better to be sure :)  Thanks very much and sorry for the list of questions, better to get it right now so we can sort it as quickly as possible.

Hello orbrey,  no problem:  1) All files are UTF-16 LE (with BOM) 2) The problem is already present when I add a single file (working copy is clean except one new text file) 3) It is a remote repository (Running on Windows Server 2012), but the problem occurs already when I just "add" the file (so before commit) 4) Plain "svn:" protocol 5) Windows 7 x64 Professional

Hi, sorry just a couple of questions I now realise I forgot to ask.  What command are you running when it errors? Is it a commit, or a checkout, or something else - or does it happen with multiple commands?  Also, what version of svn is on the server (if you know)?

So far i have encoutered the error when I tried to commit and when I tried to add files (which commit does implicitly, so I guess it is a problem of the "add" command")  I don't know the exact svn version on the server, but I know it is running svn 1.7.x.

One more thing I noticed is that if I add the file twice, I do not get the error message the second time.  Still if I want to add says 20 files i have to hit the "Add" button 40 times ...

Is there any way you could get the server upgraded to the latest version (1.8.8)? It might mean upgrading the repositories and working copies but the fact that you're already using the latest version of SmartSVN should make things easier from the client side.  If not no worries, I'll keep testing and let you know what I find.

I just ran into this issue with SmartSVN 8.5.4 (build #2211).   The "inconsistent newlines" popup appeared for several files, but the add operation seemed to work OK.   Once the problematic files were individually added, I was able to commit all files (problematic and non-problematic) in one operation.

Matt, is there any news on this issue?  I sometimes have to add 20+ (or even 100+) files to my repositories and doing it file by file while always having to click away the error message is quite annoying :\  PS: I can't update the subversion server to a newer version.

It's an odd one, I've not been able to replicate it. Is there any chance it could be related to this issue: http://support.wandisco.com/index.php?/Knowledgebase/Article/View/484/0/files-incorrectly-showing-as-modified-after-updating-to-smartsvn-8-or-newer-and-upgrading-working-copy  It's the new lines error that's making me think it could be related, hope the fix detailed there works for you?

The thing is, you can't view the EOL style before the file is added (using either the Add or the Commit button).  In other words the error occurs before the file has any SVN properties at all.  If you do a "svn status" it will show as state "?".  I could of course use the command line svn client to add the files, but that kind of defeats the purpose of having a UI tool.  I just tested and the svn client has no problem adding the files.  [ATTACH]399[/ATTACH]

I've just tested again with several hundred text files all encoded in UTF-16 LE (BOM) and all files were added, though the display needed refreshing before they'd all show as such. I'll keep testing and if it keeps happening I'll get it reported, though I've not seen the issue you describe I'm afraid.

Additionaly moving files fails with this newline error.  I think I know why you can't reproduce the problem, we have added this to our %appdata%\Subversion\config file:  [miscellany] global-ignores = __history DCU *.dcu *.bak *.dproj.local *.groupproj.local *.identcache  [auto-props] *.xsl = svn:mime-type=text/xml *.xslt = svn:mime-type=text/xml *.xsd = svn:mime-type=text/xml *.xml = svn:mime-type=text/xml *.txt = svn:mime-type=text/plain *.sql = svn:mime-type=text/plain *.pdf = svn:mime-type=application/pdf  The files have a .sql or .xml extension and without this modified config file SmartSVN adds the files as binary instead of text, because the default detection algorithm of text vs. binary files of Subversion is severly broken for UTF-16 and UTF-32 (they do an analysis of the percentage of zero bytes in the file which always makes them look binary).  The curious thing is that the auto-props aren't applied when I use the svn command line binary (version 1.8.9).

Ok, for the svn command line binary you have to set enable-auto-props yes as well:  [miscellany] global-ignores = __history DCU *.dcu *.bak *.dproj.local *.groupproj.local *.identcache enable-auto-props = yes  [auto-props] *.xsl = svn:mime-type=text/xml *.xslt = svn:mime-type=text/xml *.xsd = svn:mime-type=text/xml *.xml = svn:mime-type=text/xml *.txt = svn:mime-type=text/plain *.sql = svn:mime-type=text/plain *.pdf = svn:mime-type=application/pdf  Now svn works, but SmartSVN is still showing the same error about inconsistent newlines.

Hi Jens,  If you check for the latest build in the SmartSVN help menu you should be able to update - the latest build (I believe it's 2236) should fix this for you.  Hope that helps.

Hello Matt,  I'm afraid that the latest build doesn't fix the issue, I'm still seeing exactly the same problem.

Hi, I can see you've raised a ticket now - our support team are looking into this for you and I believe the issue's been replicated and a workaround offered?  Hope it gets things sorted for you.

Hello Matt,  yes there is an acceptable workaround. For future reference:  You can set the default EOL style in the Project settings (and Default Project Settings). In SmartSVN 8 the default is "native", but in our situation (UTF-16 text files) you have to set it to "as is" which was apparently the default in earlier SmartSVN versions.  So far this seems to work, thanks.