Vim as a system interpreter for vimscript

Proposed tip Please edit this page to improve it, or add your comments below (do not use the discussion page).

Please use new tips to discuss whether this page should be a permanent tip, or whether it should be merged to an existing tip.

created November 4, 2012 · complexity basic · version 7.0

---

One of the basic features that make Vim a powerful product is the scripting facility it offers, often called vimscript. It allows many authors to write countless plugins and scripts that can integrate Vim with any software or system you can think of, and that allow Vim to be customized to work in almost any way you need.

The language still uses the original ed / ex / vi commands. These are standard tools today, in the current POSIX (Portable Operating System Interfaces) specification, and scripts of ex commands can be used as a diff and patch format, or to perform other line-oriented manipulation of text.

You can see that ex scripts have a different purpose than the most Vim scripts today. They are used for test-manipulation tasks outside ex/vi/vim, unlike the kind of usage in current Vim scripts is to modify/improve/affect Vim itself. Lets consider the usual Vim scripts (plugin/syntax/compiler/indent and others) as "internal" scripts, and the ex scripts would be "command" or "external" scripts.

Now the nice thing is, Vim can of course run ex scripts too, and is also a command language that can run external command scripts, other than plugins and syntax files. And Vim can use more than plain ex commands, it uses the extended vimscript language, so Vim turns into a real interpreter, like the ones for other languages. Add to that that it can even invoke python and other languages, too.

Needless to say, for any text processing task Vim can be a powerful command tool.

Vim running in ex mode

The -e and -E command options on the Vim command line put Vim in "ex" mode (also reachable with gQ from a running Vim instance), where all input to the program consists of ex and vimscript commands, like in a language interpreter. The difference between -e and -E is that -E introduces command-line editing and history (called improved ex mode) to the plain ex mode selected with -e. However command scripts do not use the command-line editing features and are often run with vim -e.

Even with Ex mode you still get an edit window open, including when running Vim in a terminal, which is not quite as expected from a system interpreter. For this Vim will also accepts the -s command line options when in ex mode, to run in silent (batch) mode (outside ex mode there is another, unrelated, -s command line option). This has several effects:

• you will not see an edit window when running the editor
• all output is suppressed by default, except for some specific commands. This suppresses error messages, too.
• default initialization from ~/.vimrc and ~/.gvimrc is suppressed. Note the side-effect that without a .vimrc file Vim stays in compatible mode, which is not what you want.
• if any error occurs, the exit code from Vim (at the end of the script) will be non-zero.

This is mostly a good approach to running Vim as a system interpreter, except for a few things that need more attention:

• for any non-trivial program, suppressing error output is a show-stopper that will take down any kind of development effort. This can be changed by explicitly setting verbose level on the command line, most likey to the level of 1 with -V1
• Vim is not at all friendly in vi-compatible mode, so we want the -N option on the command line, to explicitly start Vim in non-compatible mode and get all the non-vi features Vim has to offer
• while we are discussing command line, we may also include certain other options:
  • -i NONE will make Vim ignore the viminfo file
  • -n will make Vim use no swap file.
  • -u NORC and -U NONE are on the same category as the others above, but they are implied by -s

One more needed thing to run Vim as an interpreter is to pass a script file to Vim and have Vim exit when the script finishes. This is traditionally achieved with input/output redirection, where the standard input stream is redirected from the script file to be run. But for an interpreter you would expect to also be able to pass in the script file directly, not only with redirection. For this we use the -S srcfile Vim option for Vim to source the given file, or even -S %, for Vim to source the first file argument present on the command line. About quitting Vim, it is of course possible to use :qall! in the script itself. But using -c "qall!" option on the command line is a better way. This way the script only has to deal with the task it performs, and can be re-used and included in other scripts without implicitly ending the Vim session. To summarize, run Vim as a language interpreter like this:

vim -i NONE -u NORC -U NONE -V1 -nNesS script-cmd -c'echo""|qall!' -- args...

The echo "" is here for another reason: in vimscript, echo commands start each line with an end-of-line indicator, instead of eding the lines with an eol, as you would expect in a terminal. So an additional :echo "" is included before the end of the script. Sometimes a space is needed instead of the empty string.

As you can see this article refers to the console (text-mode) version of Vim, vim.exe, but the approach is valid for the GUI version gvim.exe, too.

Interpreter invocation

For the invocation of the needed command line, Linux and Windows systems have different approaches. Lets take Linux first (because Linux is free, and also because the solution is easier here :)

POSIX

On POSIX systems it is possible to compose a carefully crafted shebang line, approximately equivalent to the above command line. The shebang line is a special line that:

• is the first line in the script
• starts with the sharp-bang (shebang) characters
• names the interpreter appropriate for that file, and one possible interpreter argument
• is used by the command shell (usually sh or bash) as the command to be used for running the script

Of course, as the first line in the file, it is also the first line read by Vim when sourcing the file. But Vim knows about shebang lines, and vimscript specifies that a Vim command that starts with #! is ignored (try it). A shebang line can include the interpreter name, in this case /usr/bin/vim, and one possible argument. The command line above has more than one option flags and operands, but hopefully can be rewritten to better serve a shebang line. Remember that by the POSIX standard the following hold true:

• more than one option flags can be concatenated in a single command line argument, as long as only one of them at most, last in group, takes an operand. This is how in the above command the flags -nNesS script-cmd appear in one argument with one operand
• the flags and the operand may also be concatenated together in one command line argument

So we could also write -nNesSscript-cmd. Since the -S script-cmd can also be written as -c 'source script-cmd', which can further be merged with the next -c option, we can now write a single command line argument like

-nNesc'source % | echo "" | qall!'

The script name and any script arguments will automatically follow this argument, when the command line is composed by the shell, so the included source % will source the script. We still need the -i NONE and -V1 arguments, and they can be written as Vim commands to a -c option, like:

-c 'let &viminfo=""|let &verbose=1'

Now on a shebang line there is no need for quoting (it makes sense, there can be at most a singe argument specified), so drop the apostrophes, merge the two -c options, and come up with the following shebang line at the start of the script:

#!/usr/bin/vim -nNesc:let&verbose=1|let&viminfo=""|source%|echo""|qall!

For POSIX and Linux systems, the above line is all you need.

Windows

File type association

For Windows systems a file association is normally used to associate an interpreter with a file. Actually that associates the interpreter with an entire type of files, so we need a file extension to use for our ex scripts, like for example ".exim".

".exim" stands for "ex improved" (mentioned above) and Vim could also be installed with that executable name *:help exim, in addition to the "vim" name.

The extension ".vim" might be a good choice also, but for the moment there are a lot of Vim plugin developers used with the .vim extension to mean Vim plugin or syntax files (internal scripts), and they may or may not like the idea that pressing Enter on a .vim file now means the file will be sourced by Vim.

Extension ".ex" is another intuitive choice, but a web search will show the programing language Euforia is using it.

Next, creating a new file association is introduced in Windows file associations. In our case we also want to add the file extension to the PATHEXT environment variable, and maybe include a Vim icon for the file type. On Windows Vim is not necessarily found on PATH, normally the install directory is found in the registry, and the directory is dependent on installed Vim version.

If you get into all the details, setting up a file type association by scripting can be complicated (and may also require privileges), and is normally achieved by application installers or other application code. However since Vim installer on Windows does not register a file type to be handled with the command line we need, a good choice is to create the association right from your script. The mkvimball plugin includes the code to register a new file type on Windows, and provides the command line options for the task. You can include (or source) it in other scripts mostly as it is, if you only change the last :call statement at the end of the file to append one additional argument to the function, with the value "1". See also the project repository at http://code.google.com/p/dbgp-client/. The script performs actions like:

• request elevation when installing the file type for all users
• notify the Windows shell (when possible), to refresh file type icons
• install a command script that searches Vim dir in the registry or in %ProgramFiles%
• adds the extension to %PATHEXT% variable. The per-user variable value first includes the system %PATHEXT% value.

Wrapper .cmd script

Windows file type associations are only in effect on the computer they have been configured. If you want to distribute a Vim command script, you may distribute a wrapper .cmd script that only invokes Vim with the proper command line to run the ex script. Again, you can instead write your Vim command script to generate this wrapper, with a special option. Some of the install issues mentioned above for file type associations, like locating Vim installation directory, apply to this approach too. You can use the mkvimball.vim script (Vim script 4219), that includes the option to generate wrapper scripts for Windows cmd.exe. This is generally the most flexible solution, although not an elegant one.

Usage

Now Vim is primarily a text editor, and the scripting feature is only a facility. As it is now (at the time of writing this article) Vim was not meant to run as a full language interpreter integrated in the system, so you still need to observe a number of difficulties if you choose to write Vim command scripts:

• Arguments to your script are really arguments to Vim. If you want arguments for the script only, the command line must use -- to separate the script name from the rest of the arguments.
• At the verbose level of 1, mentioned above, Vim will output messages in the console, when loading files or for other operations, so the script itself is responsible only for part of the output text. You can use :silent commands or a verbose level of 0, but that will also suppress errors (not recommended).
• There is no easy way to echo a multi-line string. All you can do is iterate over the lines in the string and echo each one at a time.
• In a try| ... |catch block the echoerr statement raises an exception and breaks your code block. In case of error you should only count on the first echoerr line that you output, to reach the user.
• You can use standard input in a command script if you :call input(), but Vim should never see an end-of-file there, or it will immediately exit your script. This is because in "ex" mode Vim expect to read commands from the standard input.
• for some versions of Vim running on Windows, some commands (:normal "ggdd", :call input()) may still scramble the screen on occasion (by moving the cursor around), even though a Vim window is not visible.
• on Cygwin vim.exe is a symlink to gvim.exe. When the shebang line is used Vim will still try to reach an X server, and will output some error in the process. As a result your command script will always return with an error, even if it actually succeeds. This is a problem in Makefiles, and using the explicit command line may be needed.

References

• :help starting