Change end-of-line format for dos-mac-unix
From Vim Tips Wiki
Tip 26 Previous Next created March 5, 2001 · complexity basic · author scrott · version 6.0
See File format for other suggestions. May merge these tips later.
If you work in a mixed environment (Unix and DOS or Windows), you will often see ^M when you open a file. Under Unix, a line is supposed to be terminated with an LF (linefeed) character, but DOS/Windows use CR LF (carriage return followed by linefeed). Sometimes, LF is written as NL (newline).
If Vim believes that a file has unix format, any CR will display as ^M (because CR is Ctrl-M).
Vim supports three file formats: unix (lines end with LF), dos (lines end with CR LF), and mac (lines end with CR).
Use :set ffs? to see the value of your ffs (fileformats) option. :help 'ffs' explains how Vim interprets a file when it is read.
Use :set ff? to see the value of ff (fileformat) for the current buffer. This setting show how Vim interpreted the file when it was read. If you change ff, the file will be written with different line endings when you next write the file. See :help 'ff'.
Some programs create files with inconsistent line endings. For example, the first few lines may end with LF, and Vim will decide the file has the unix file format. But later lines may use CR LF and Vim may show the CR as ^M.
To replace every CR with LF (when searching, \r matches CR, but when replacing, \r inserts LF):
:%s/\r/\r/g
To delete every ^M from the buffer:
:%s/\r//g
The \r is two characters that are interpreted as the single character CR (:help /\r). The above command will delete all CR characters, regardless of where they occur in a line.
To delete ^M only when it occurs at the end of a line:
:%s/\r$//
To delete ^M at line endings, and replace it with a space everywhere else:
:%s/\r$// :%s/\r/ /g
It is possible to use an actual CR instead of the two characters \r in the above commands. To enter a CR, you type Ctrl-V Ctrl-M (or Ctrl-Q Ctrl-M under Windows). The <Enter> key can be used in place of Ctrl-M. You can also use Ctrl-K (enter digraph) to insert the CR, by typing Ctrl-k and then <Enter> twice (once for the key value, once to accept the entry). However, using \r is more easily read, copied, pasted, etc., and thus is the recommended procedure.
[edit] Unix users
Under Unix, you may find that a Vim script does not work because you have downloaded a script that contains Carriage Return (CR) characters. Each CR displays as ^M, and will cause some scripts to fail.
If you put script.vim in your plugin directory, you may not see any useful error messages about CR characters. You can source the script after starting Vim (for example, :source ~/.vim/plugin/script.vim) to see if errors including "^M" are shown.
If there are many files that you need to fix, you can use the Unix dos2unix utility, for example:
cd .vim dos2unix */*.vim # Or, if run as root, you can preserve timestamps: dos2unix -p */*.vim
[edit] Comments
TO DO
Explain use of:
:set ff=unix :set ff=dos
The following reads file foo.txt using dos file format (CR LF is the expected line ending, but LF will also be accepted as a line ending). The second command writes the file using the unix file format, so every line will end with LF only. So this converts from dos format (or a mixture of dos/unix), to unix format.
:e ++ff=dos foo.txt :w ++ff=unix
The following converts from mac format to unix format.
:e ++ff=mac foo.txt :w ++ff=unix
To process, say, all *.txt files in the current directory:
vim *.txt :set hidden :bufdo %s/\r$//e :bufdo %s/\r/ /eg :xa
Suppose you do not want to change the file, but you want to hide the ^M characters. You can use:
:hi SpecialKey guifg=bg
Unfortunately, this hides all special characters.
- As discussed above, just doing :e ++ff=dos will re-read the file in DOS mode. This will also not modify the file, until you save anyway.
If you want to remove ^M at the end of each line, yet have Vim regard the file as unchanged, enter:
:%s/\r$//e | set nomod
The e flag means no error occurs if no matches are found. Also see :help 'mod'.
The following is a little dangerous as it attempts to delete all CR characters from every file that you open. You could put this in your vimrc file.
" Delete one or more CR at the end of each line. autocmd BufRead * silent! %s/\r\+$// " Alternative: Also delete trailing whitespace. autocmd BufRead * silent! %s/[\r \t]\+$//
If you want to write a file with no linebreak at the end:
:set noeol bin
Now, when you write the file, the last line will have no line ending. You do not normally want this. Every line, including the last, should have a line ending.
I had a bunch of HTML files that needed to be converted. As they were all open I just did the following:
:bufdo! set ff=unix | update
You can display the file format for the current buffer in the status line:
set statusline=%<%f%h%m%r%=%{&ff}\ %l,%c%V\ %P
All I care about is if the file format is not unix. If it's not, I want a big red warning. That way I'm not the jerk who checks in a file that causes every line to get modified by the diff patch.
So, I added this to my existing statusline:
%9*%{&ff=='unix'?'':&ff.'\ format'}%*
Here's what is does:
%9*
\- Change highlighting to user setting #9 (see :he hl-User1..9)
%{
\- Begin evaluating as expression until } is encountered
&ff=='unix'?'':&ff.'\ format'
\- This is a ternary that returns either an empty string, or 'XX format'
}
\- This marks the end of the expression
%*
\- Restores normal highlight
So, how do you use it? First I call:
:set statusline?
Which returns:
statusline=%<%f :: %{TagName()} %(%h%m%r %)%=%-15.15(%l,%c%V%)%P
But, remember that if you want to set a status line you must escape all white space. So that line would have to be entered as:
statusline=%<%f\ ::\ %{TagName()}\ %(%h%m%r\ %)%=%-15.15(%l,%c%V%)%P
So when I added my modification I have:
:set statusline=%<%f\ :%9*%{&ff=='unix'?'':&ff.'\ format'}%*:\ %{TagName()}\ %(%h%m%r\ %)%=%-15.15(%l,%c%V%)%P
But then, to make the user highlighting #9 big and red I started by viewing all the existing highlighting configurations (I'm too lazy to write my own) by calling:
:hi
The entry titled ErrorMsg looked good to me so I copied its settings which were:
term=standout cterm=bold ctermfg=7 ctermbg=1
I then called:
:hi User9 term=standout cterm=bold ctermfg=7 ctermbg=1
Now my status line is unchanged and uncluttered, unless I have opened a dos file. That's pretty cool.
I wrote this in REAL basic terms, because I really wish someone had explained it to me like this. I hope it's well received.
If you always want a particular file to have, say, dos file format, you can put a modeline in that file. For example, add this line (a C-style comment) near the beginning or end of the file:
/* vim: set ff=dos: */
[edit] Hide unwanted text
Here is a technique to hide ^M line endings by highlighting them with the Ignore highlight group. This is from a tip that was proposed by Frienddaniel (the tip has been merged to here).
:match Ignore /\r$/
Any ^M characters that are embedded within lines (not at the line endings) will be displayed as normal because they are problems that need to be fixed.
Note If ^M characters are at the end of every line, Vim should open the file in DOS file format. If you see the ^M, there are probably a few lines that do not have one, and the file has been opened in Unix file format. If this is the case, delete all the ^M characters with a simple :s command, and save the file (after :set fileformat=dos if desired). Just hiding the problem with a highlight group is probably a bad idea. If you hide the problem, you will never know if a file has pesky line endings that might screw up your regular expressions (^M doesn't match at the end of line '$' atom), plugins (TagList, for example, has no idea where to jump in a file with inconsistent line endings), or even your use of the file (some compilers, interpreters, parsers, etc will get thrown for a loop with unexpected line-endings, makefiles refuse to work, and all sorts of other trouble can happen). You should always prefer fixing a problem over hiding it!
