Getting the Vim source with Mercurial

Proposed tip Please edit this page to improve it, or add your comments below (do not use the discussion page).

Please use new tips to discuss whether this page should be a permanent tip, or whether it should be merged to an existing tip.

created May 14, 2010 · complexity basic · author Tonymec · version 7.0

---

This tip describes how to get the Vim source from the new Mercurial repository. It assumes that you know how to compile Vim once you have the source. Examples are for Linux but it shouldn't be hard to adapt them to whatever OS you're running on (provided of course that there exists a version of Mercurial that runs on your OS). Also, the examples are written for GNU make: in case of doubt, try replacing make by gmake everywhere.

First time only: Creating your repository

Make sure that Mercurial and Python are installed on your system

e.g. by checking that there is a program named hg and another named python in your PATH. If you haven't got Mercurial, install it, either (if available) by installing the Mercurial package from your distro, or else from the Mercurial site. As of this writing (15 May 2010) the Python version required by the Mercurial package is python 2.4. (I use it with no trouble with python 2.6.2; I got both packages from my openSUSE Linux distro — Tonymec 23:41, May 14, 2010 (UTC).)

Install a copy of Bram's latest source, runtime and ancillary files

1. Set up a directory head "for building" and cd to it, e.g. (on Linux)

mkdir -p ~/.build/hg
cd ~/.build/hg

2. Clone Bram's repository (this may take some time, depending on the speed of your CPU and Internet connection):

(date && hg clone https://vim.googlecode.com/hg/ vim) 2>&1 |tee hg-vim.log

The above is slightly more complicated than absolutely necessary because I recommend to keep a running log. The important part starts with hg and ends immediately before the right parenthesis. Similarly for other hg commands below.

Simple case: Getting new patches

This applies if all of the following are true:

• You have no local changes to the Vim source, ancillary and runtime files (including the Makefile: see my Compiling HowTo pages for a way to set compilation options with no changes to the Makefile)
• You compile only one Vim and therefore don't use a shadow directory
• You don't build a cscope database inside the Vim source tree

In this case, all you need to check for (and, if necessary, get) any changes to the Vim source is:

cd ~/.build/hg/vim
(date && hg pull -u) 2>&1 |tee -a ../hg-vim.log

Complex case: You have local changes and/or compile several Vim versions

Enable the fetch extension

by adding the following to the .hg/hgrc file in your repository (it was created by the clone process, with Bram's repository as the default remote source):

[extensions]
hgext.fetch = 

No need to add anything after the equal sign, this extension comes packed with Mercurial (but disabled by default)

Patch your .hgignore file

It is found at the top level of your repository. This step is unnecessary if you have neither additional help files nor additional Vim versions and you don't add a cscope database inside the Vim source tree. Copy the following patch and apply it with patch -p1 < hgignore.diff from the top directory where the file resides.

diff -r 2bd29808d1f6 -r a5e628a08c4e .hgignore
--- a/.hgignore Fri May 14 18:56:38 2010 +0200
+++ b/.hgignore Fri May 14 20:05:50 2010 +0200
@@ -30,12 +30,25 @@ src/auto/pathdef.c
 *.res
 *.RES
 src/pathdef.c
 src/Obj*/pathdef.c
 gvimext.dll
 gvimext.lib
 
 # All platforms
+runtime/doc/tags
 *.rej
 *.orig
 *.mo
 *~
+
+# shadow directories
+# the directory names could be anything but we restrict them
+# to shadow (the default) or shadow-*
+src/shadow
+src/shadow-*
+# src/runtime and src/pixmaps are softlinks needed for proper 'make install'
+# when in a shadow directory
+src/runtime
+src/pixmaps
+# avoid tracking cscope.out even if built here
+src/cscope.out

Then commit this change:

(date && hg commit -m 'Ignore shadow directories and help tag changes') 2>&1 |tee -a ../hg-vim.log

Create shadow directories

If you want to compile more than one version of Vim, create shadow directories to avoid conflicts between them. We create them with names starting with shadow- so .hgignore (after applying the above patch) will see them. Here is an example:

cd ~/.build/hg/vim/src
(date && SHADOWDIR='shadow-huge' make -e shadow) 2>&1 |tee -a ../../hg-vim.log
(date && SHADOWDIR='shadow-tiny' make -e shadow) 2>&1 |tee -a ../../hg-vim.log

The -e command-line switch is necessary so that make won't override our SHADOWDIR names with the default which is just shadow

Apply your local changes and commit them

You do this by editing the sources in the src directory. The case where you want to compile different versions of Vim with not only different configuration options but even different changes to the sources is not covered by the present tip, and neither is the case when your changes could conflict with Bram's (the changes I have consist of added sections in out-of-the-way parts of Makefile and feature.h plus Bill McCarthy's extra float functions).

Also, check that your "nonstandard" patches are mentioned in src/version.c so that they will appear in the :version output of your home-compiled Vim: here is the relevant part of mine, normally you would add one line (with a string followed by a comma) per extra patch with a /**/ comment between each of them:

static char *(extra_patches[]) =
{   /* Add your patch description below this line */
/**/
#ifdef FEAT_FLOAT
    "Extra float functions (Bill McCarthy)",
#endif
/**/
    NULL
};

Then, commit your local changes with

(date && hg commit -m 'Local source changes') 2>&1 |tee -a ../hg-vim.log

Get any new official patches and merge them with your local ones

cd ~/.build/hg/vim
(date && hg fetch --switch-parent) 2>&1 |tee -a ../hg-vim.log

The --switch-parent switch places the local directory (where changes are few and far between) as the "first" parent of any resulting merge, so that the local changes won't be removed and added back every time: this reduces the number of source files which make will see as "modified".

Before compiling, you may want to check any files listed as "merged" (possibly with the help of the hg diff function) to make sure that the changes are what you would expect.

See also

Manpages

• man hg
• man hgrc
• man hgignore

In this wiki

• Category:Building Vim

External links

• The Mercurial site
• The Mercurial Guide
• Compiling Vim on Windows
• Compiling Vim on Unix/Linux

Comments

Please sign your comments with ~~~~ and separate unrelated topics with ---- on its own line. — Tonymec 23:41, May 14, 2010 (UTC)

---

Known bug: Logging and interactive merge

A problem which I haven't solved about the above method of keeping a running log, is that is doesn't mesh well with Mercurial's "interactive merge tool": if the interactive merge asks a question, you don't see it, and you will have to blind-type a guessed response.

In my experience, questions are about runtime/doc/tags being deleted locally, reintroduced on the remote repository, and the question is "Use (c)changed version or leave (d)eleted?" -- in that case I reply d and the fetch goes on. I get this on the first non-null fetch after a "make install".

If you hit Ctrl-C to break the run, you won't see what Mercurial has output so far: it is not in the log; and your local repository is left in an uncertain state: once I did this, but then "hg status" replied nothing and "hg fetch" gave an error saying local files needed update -C or merge. Finally "hg merge" (with no logging) gave me the c/d question again so I could bring back my local repo to a "sane" state by replying d then manually committing the merge (with a -m message inspired by the "description" on the remote head, as obtained with "hg heads").

— Tonymec 04:39, May 18, 2010 (UTC)

---

Useful commands to "get information"

• hg help
• hg help |less

what are the basic hg commands, and what do they do?

• hg help command

(where command is a Mercurial command) what are the possible arguments to that command (and what do they mean)?

• hg status

are there local files waiting for a disposition (commit possibly preceded by add and/or forget)? The reply is one filename per line, preceded by A (added to the list of tracked files), M (merged), D (deleted i.e. specifically not tracked), ? (new file, neither "tracked" nor "not tracked" yet).

• hg heads

what are all the "tree leaves" ? (commits which are not a parent to a later commit)

• hg log -l N

(where N is a number): show the N latest commits, with relative number and short hash, branch name if any, tags if any, parent(s) if not, or not only, the immediately older commit, and description (commit message)

• hg log -l N -f

same as above, but only for the current tip and its parents and ancestors.

• hg diff -r xxxxxxxxxxxx [-r yyyyyyyyyyyy] [file ...]

where xxxxxxxxxxxx (and yyyyyyyyyyyy if present) are revision IDs: Produce a diff of the file(s) named, as they changed between the named revisions. With only one revision: compare it to the current tip. With no filenames: give all differences between these revisions.

Notes:

1. Contrary to what is said in the manpage, with no revision there is no output.
2. I find it useful to use |view - as a pager for the output of this command. Then a simple :saveas patchname.diff will, if you want to, create the patch file after you review it.
3. To always use view as pager for the more "bulky" output-producing Mercurial commands, add the following to your ~/.hgrc file (or, on Windows, your %USERPROFILE%\Mercurial.ini):

[pager]
pager = view -
attend = annotate, cat, diff, export, glog, log, qdiff

— Tonymec 09:43, May 24, 2010 (UTC)

---

Changing branches

Changing branches is one of the functions of the hg update command (which has up as an alias) — in general, its function is to "update" the files in the current repository (the "working copy" as Mercurial names it) to any revision you specify. So:

To go to the Vim 7.3 source

hg up vim73

To go back to Vim 7.2

hg up default

To clone a new repository and immediately set it up for Vim 7.3

hg clone -U https://vim.googlecode.com/hg/ vim73a
cd vim73a
hg update vim73

The -U switch avoids an unnecessary pull of the Vim 7.2 source.
— Tonymec 23:08, May 23, 2010 (UTC)

---

Tony: I haven't had an opportunity to seriously read this, but as part of my normal "standard format" I notice that there are unexpected trailing spaces. They don't matter, but I like to get the stuff I can understand sorted... I'm pretty sure the space after hgext.fetch = is redundant, and nearly all the spaces in the "Patch your .hgignore file" section can be deleted. But then I noticed that the leading spaces look a bit odd. There's no need to explain it to me, but I don't understand the "diff -r 2bd29808d1f6 -r a5e628a08c4e .hgignore" line (but it's ignored by the patch), and I expected two spaces in front of most the diff lines. Anyway, if there is anything which needs fixing (like redundant trailing spaces, or the leading two characters in the patch), please do so. Use &nbsp; for the last trailing space on any lines which require leading spaces so I don't accidentally remove them later. Thanks. JohnBeckett 01:56, May 24, 2010 (UTC)

The trailing spaces in the patch come from copy-pasting from konsole, I've removed them. The diff -r (etc.) line is part of the output of hg diff (as I got that patch by doing an hg diff between .hgignore before and after I changed it). The patch program will consider that line as what its manpage names "leading garbage" as it does the "Problem" "Solution" and "Files" headers in Bram's patches -- IOW, it is not necessary but doesn't hurt. The problem with &nbsp; on the otherwise empty lines of a patch is that the patch program needs a true space (0x20), not a no-break space (0xA0). — Tonymec 09:13, May 24, 2010 (UTC)

P.S. One of the differences between "unified diff" (as produced by hg diff and by diff -u, with added and removed lines in a single section) and "traditional" context diffs (as produced by diff -c, with separate sections relating to the "old" and the "new" file) is that traditional context diffs have one space between the initial ! (changed), + (added), - (removed), space (unchanged) and the line text, while in unified diffs the line text immediately follows the initial + - or space. — Tonymec 09:25, May 24, 2010 (UTC)