Changes: Power of g

Revision as of 08:41, 18 September 2007

Please review this tip:

This tip was imported from vim.org and needs general review.
You might clean up comments or merge similar tips.
Add suitable categories so people can find the tip.
Please avoid the discussion page (use the Comments section below for notes).
If the tip contains good advice for current Vim, remove the {{review}} line.

Tip: #227 - Power of g

Created: March 24, 2002 3:15 Complexity: intermediate Author: Arun Easi Version: 5.7 Karma: 900/308 Imported from: Tip#227

g is something very old and which is very powerful. I just wanted to illustrate the use of it

with some examples. Hope, it will be useful for someone.

Brief explanation for ":g"

Syntax is:

:[range]:g/<pattern>/[cmd]

You can think the working as, for the range (default whole file), execute

the colon command(ex) "cmd" for the lines matching <pattern>. Also, for all

lines that matched the pattern, "." is set to that particular line (for

certain commands if line is not specified "." (current line) is assumed).

Some examples

Display context (5 lines) for all occurences of a pattern

:g/<pattern>/z#.5

:g/<pattern>/z#.5

Comments

Thanx Karma for extending my g// knowledge

Here's a very useful g// for day to day use

'a,'bg/pattern/s/pattern2/string3/gi

zzapper

david--AT--tvis.co.uk , March 28, 2002 7:10

g WITH CONFIRM c

Here's something very curious:

%s/foo/bar/gc

is of course a substitution effective for the entire file with confirm on each occurrence of "foo" with the option of quitting at any point.

However, using something similar,

g/foo/s//bar/gc

using the global g to effect the entire file -- does NOT allow quitting at any point (even with the use of <ESC>).

If there are hundreds of "foo" -- it's an important fine point...

Invite further comments...

Hal Atherton , April 23, 2002 10:12

g/foo/s//bar/gc => run the command s//bar/gc for each of the line matching foo. It is like running multiple "s//" commands (Hence you have to press q for each of the invocation). The g in "s///gc" does not imply entire file, it just implies all occurence on a line (or else, it would have substituted only the first)

Arun Easi , April 23, 2002 23:33

Here is one that deletes every other line (adjusting double spaced files):

g/.*/norm jdd

dyang--AT--entropia.com , May 30, 2002 9:04

This can be done much simpler:

%norm jdd

pagaltzis()gmx_de , August 6, 2002 15:12

Another cool g feature is to count the number of lines matching /regexp/

let x=0 | g/regexp/let x=x+1 echo x

Great, if you are editing data files.

Regards Mike

mra--AT--frogne.dk , February 10, 2003 4:49

Reverse all the lines in a file:

g/^/m0

I have found that useful . . . honest!

PK

setanta5--AT--excite.com , March 27, 2003 2:14

Can I do something like this using ":g" (or anything else)

I have a file which contains following kind of lines

abc123=1,2,3 bcd123=100,200,300 abcb123=1,3,4

I want to convert this to following

abc123=1,abc,2,abc,3,abc bcd123=100,bcd,200,bcd,300,bcd abcb123=1,abcb,3,abcb,4,abcb

Basically I want to replace each comma in a line with first few letters, which are coming before 123, of that line surrounded by 2 commas.

kkgahlot--AT--yahoo.com , September 3, 2003 5:06

To answer kkgahlot's question:

global // execute "s/\$[=,][^,]*\$/\\1, " . matchstr (getline ("."), "^.\\{-}\$123\$\\--AT--=") . "/g"

To make the whole thing a little more transparent some explanations (from the inside out):

We want to execute on each line a command like

s/$[=,][^,]*$/\1, abc/g

for each line, but abc gets changed on each line.

The function

matchstr (getline ("."), "^.\\{-}\$123\$\\--AT--=")

returns the string that matches the pattern /^.\{-}\(123)\--AT--=/ on the current line. In the given examples this is the text preceding 123= at the beginning of the line. Depending on the actual requierements, a simpler expression like /^[a-z]*/ could work too.

The command

execute "s/\$[=,][^,]*\$/\\1, " . matchstr (getline ("."), "^.\\{-}\$123\$\\--AT--=") . "/g"

assembles the desired substitute command for the current line by joining some static text with the return value of the matchstr function and then executes it.

As execute works only on the current line, the command

global // execute ...

applies it to all line. If only certain lines should be proecessed replace // with some other pattern.

Anonymous , September 3, 2003 12:45

Reverting lines in a file can also be done via

%!tac

instead of tac, you can also use sort, xxd, xxd -r, cut, your-own-filter

utzb--AT--gmx.de , November 21, 2003 10:39

too bad :g does not have a /c parameter for confirmed execution of ex commands. Something like the /c used in :<range>s/<PAT>/<REPL>/c for replacing only confirmed pattern locations.

or does it ??

Anonymous , March 29, 2004 7:18

I just want to add that the :g command in vi/ed is the basis for the name for the Unix grep command, i.e. Global Regular Expression Print. The "Print" part described the default behaviour of :g when no command was given. Obviously this behaviour was more useful when using ed.

jdimpson--AT--yahoo.com , April 14, 2004 10:06

Thank you for the tip

bsjawle--AT--hotmail.com , August 16, 2004 23:55

Thanks for reminding about the power of :g! I used to use it back when there was only "vi". But together with vim it becomes so much stronger. Now, if someone could explain:

:g/^/pu _

I guess I should know and I guess it's an RTFM thinngy. But anyway. =) /PEZ

pez--AT--pezius.com , December 16, 2004 4:33

The command

g/^/pu _

is the global command with ^ as search pattern. The ^ matches the beginning of a line and therefore each line of the file is a match. The pu _ command is the :put command putting the contents of the _ blackhole register after the current line followed by a newline. The blackhole register _ never contains anything and therefore it can be used to insert an empty line. This is like doing cp /dev/nul empty.file to create an empty file. What the entire command does is putting an empty line after each line of the file.

jpaulus--AT--fNrOeeSnPeAt.dMe - NOSPAM , February 22, 2005 12:15

The above note contains a typo. cp /dev/nul empty.file is wrong. cp /dev/null empty.file is right. Doing touch empty.file is another way to create an empty file if it does not exist yet.

Anonymous , February 22, 2005 12:32

I work a lot with coordinate lists that have the following format:

XXXXXXX.XXXXXX YYYYYYY.YYYYYY

these lists can be extremely long. Sometimes, the same coordinate appears twice or more times, how can I delete multiple occurences?

Anonymous , December 22, 2005 2:02

how can you globally delete lines 'not matching' a pattern

Anonymous , March 17, 2006 12:41

Mention should also be made of :g's cousin, :v which is the same except it runs the command on every line that *does not* match the pattern.

So to delete all the lines that do not match a particular pattern you do

v/pattern/d

jonathantan86--AT--hotmail.com , March 18, 2006 2:29

Is there way I can use :g to search a pattern and delete from that pattern to the end of line instead of deleting the whole line.

ra_v_na--AT--rediffmail.com , March 24, 2006 9:21

g/<pattern>7d does not work ? Is there any way to accompolish this. I want to delete more than 1 lines based on a pattern

amjain.gzb--AT--gmail.com , April 4, 2006 6:55

Run a macro on a matching lines (example assuming a macro recorded as 'q'):

:g/<pattern>/normal --AT--q

g and normal combined is cool.

noah--AT--noah.org , April 20, 2006 15:20

A co-worker keeps asking me how to globally delete lines that do not match a pattern. Very useful.

To delete all things matching "pattern", you would do

%g/pattern/d

So to delete the lines that do not match, you negate the pattern using "!", like this

%g!/pattern/d

This is very useful when trying to make sense of huge logfiles, especially when used with the "or" operator. If I wanted to open a logfile and remove everything that did not match "steve", "May 2", or "addition" (sometimes you're looking for multiple things in a logfile, you get the idea), I would do something like this

%g!/steve\|May 2\|addition/d

kaan--AT--erdener.org , May 2, 2006 18:41

Regarding the post above by <amjain.gzb--AT--gmail.com>: You should be able to do something like this:

g/<pattern>/ norm d7d

Anonymous , June 2, 2006 16:12

To delete (subsequent) duplicate lines from a file:

g/^$.*$$\r\?\n\1$\+$/d

To just view the duplicates use: /^$.*$$\r\?\n\1$\+$

-Rob (phxrider\at\gmail)

Anonymous , July 13, 2006 12:39

Hi!

Сan somebody help me?

How can I copy a result of the command :g/pattern/# from the command-line window?

ruslan.gordeev--AT--gmail.com , July 18, 2006 2:29

you can use

g/^/exec "s/^/".strpart(line(".")." ",0,4)

(for inserting line numbers) and then

v/<patern>/d

...output is desired i think :) ...but it's not very beautiful way :o) then just twice u (undo) to get original content

uzivatel--AT--szm.sk , July 24, 2006 4:19

Can the search pattern matches also give the number of matches found in the file. Let's say i execute the command similar to

/<search_pattern>

In the results while it is searching can it print, the number of matches found for that pattern. say "30 lines 50 words 100 characters matched" be printed at the command area ?

msrinirao[at]gmail[dot]com , July 26, 2006 4:18

Increment the number items from current line to end-of-document by one

:.,$g/^\d/exe "normal! \<c-a>"

Thanks for the above tip..but i am bit curious to know how it works. can any one explain me how it's able to increment that means what <c-a> will represent???

k.nagakiran--AT--gmail.com , August 20, 2006 23:41

<C-A> means press Ctrl + A keys on keyboard it increments number under cursor in normal mode ...but it doesn't work if u have mapped this key (e.g. by vim under windows where are these keys (<C-V>,<C-S>,... - see output of :map command)

mapped after installing)

then u can use

unmap <C-A> to remove this mapping temporally (it will work again after new start of vim or command :source <your_PATH>_vimrc)

and then try it in normal mode

uzivatel--AT--szm.sk , August 23, 2006 1:31

Is there a way to delete from beginning of line (^) up to <pattern> on a line for all lines in a file?

OR

delete X characters on each line for Y number of lines?

Thanks!

memyselfandi--AT--graffiti.net , August 28, 2006 15:45

to everyone asking "is there a way to match ....... ": The answer is probably yes. Vim uses regular expressions, which are a very powerful way of finding patterns. They take some work to get good at (i've only been using them for 2 weeks), but any time you put into them will be greatly rewarded! And, since you are a Vim user, you are obviously used to at least one power tool that takes some learning but greatly improves your productivity.

The best way imo to learn regular expressions is 1) get a simple cheat sheet for Vim regular expressions 2) look through a list of examples, eg: http://www.rayninfo.co.uk/vimtips.html 3) any time you're doing a repetitive editing task like deleting all the

comment characters from the beginning of a code block, think of 
how a regular expression might make the job easier & figure out 
how to do it.

the Vim manual has lots of stuff on regular expressions, but it's not the easisest to follow. here are some of the help pages:

bare basics on searching: chapter 3, section 9 (usr_03.txt) a little more comprehensive: chapter 27 (usr_27.txt) pretty in-depth: regexp & search commands (pattern.txt)

to ra_v_na:

i'm not sure about :g but :s will work.

%s/$pattern$.*$/\1/

the % says to do the substitution on every line in the document pattern is the pattern we're searching for the $ and $ stores our pattern in a variable (in our case, variable \1) the .* matches any amount of characters the $ matches the end of the line the \1 is a variable storing pattern so the whole thing replaces pattern and everything following it until the end of the line by the pattern alone

to memyselfandi:

your first question:

%s/^.*$pattern$/\1/

the % says to do the substitution on every line in the document the ^ says to match the beginning of a line the .* says to match any amount of characters the $ and $ stores our pattern in a variable (in our case, variable \1) "pattern" is whatever pattern you want to search for \1 is a variable holding the contents of the pattern so basically, the whole thing replaces everything from the beginning of the line to the pattern by the pattern alone

your second question:

1,7s/^.\{5\}//

the 1,7 wold do the replacement on lines 1 through 7. Replace 1 and 7 by whatever you want the ^ says to match the beginning of the line the . says to match any character the \{5\} says to match the preceding character 5 times. Replace 5 by whatever you want so basically, the whole thing deletes the first 5 characters from lines 1 through 7

Anonymous , September 4, 2006 13:55

Compress multiple occurences of blank lines into a single blank line

:v/./,/./-j

cd to the "docs" directory of vim,

grep '\/,\/' *.txt

to find occurences of /,/ that can be investigated further to find an explanation of what vim is doing.

daN.thompOsonS--AT--yaPhAooM , September 16, 2006 18:11

v/./,/./-j

I'll break down this incredible collapse-multiple-blank-lines command for everyone, now that I finally figured out how it works.

First, however, I'll rewrite it this way to illustrate that some of those slashes have totally different meaning than others:

v_._,/./-1join

Note that to delimit expressions like these, just about any symbol can be used in place of the typical slashes... in this case, I used underscores. What we have is an inverse search (:v, same as :g!) for a dot ('.') which means anything except a newline. So this will match empty lines and proceed to execute [command] on each of them.

v_._[command]

The remaining [command] is this, which is a fancy join command, abbreviated earlier as just 'j'.

,/./-1join

The comma tells it to work with a range of lines:

help :,

With nothing before the comma, the range begins at the cursor, which is where that first blank line was. The end of the range is specified by a search, which to my knowledge actually does require slashes. The slash and dot mean to search for anything (again), which matches the nearest non-empty line and offsets by {offset} lines.

/./{offset}

The {offset} here is -1, meaning one line above. In the original command we just saw a minus sign, to which vim assumes a count of 1 by default, so it did the same thing as how I've rewritten it, but simply with one character fewer to type.

/./-1

There is a caveat about join that makes this trick possible. If you specify a range of only one line to "join", it will do nothing. For example, this command tells vim to join into one line all lines from 5 to 5, which does nothing:

5,5join

In this case, any time you have more than one empty line (the case of interest), the join will see a range greater than one and join them together. For all single empty lines, join will leave it alone.

There's no good way use a delete command with :v/./ because you have to delete one line for every empty line you find. Join turned out to be the answer.

This command only merges truly "empty" lines... if any lines contain spaces and/or tabs, they will not be collapsed. To make sure you kill those lines, try this:

v/^[^ \t]\+$/,/^[^ \t]\+$/-j

Or, to just clean such lines up first,

%s/^[ \t]\+$//g

Anonymous , October 17, 2006 18:18

"Is there way I can use :g to search a pattern and delete from that pattern to the end of line instead of deleting the whole line."

Not that I have found!

I use the ${var%string} from bash/ksh instead

ask2--AT--runbox.com , October 30, 2006 23:37

I have a malformed XML file with a structure like this:

<authors> <author> Blogs, Joe </author> <author> Smith, Fred </author> <author> Jones, David </author> </authors>

How do I use :g and the join command to get rid of those new lines, so the structure ends up like this:

<authors> <author>Blogs, Joe</author> <author>Smith, Fred</author> <author>Jones, David</author> </authors>

What I need to do is search for '<author>' and then join the next line (-1j, right?), and then the next one (containing </author>) to the previous one. These XML files are large (>1000 lines) so a global find and replace would be great. Thanks in advance!

tatlar--AT--yahoo.com , November 1, 2006 9:50

Figured out my malformed XML issue in the note above, in a 2 step process:

1) Find and replace '<author>\n' (where \n is a newline) with '<author>':

%s/<author>\n/<author>/g

2) Use :g to find '</author>' and do a negative join (join with previous line):

g/<\/author>/-1j

tatlar--AT--yahoo.com , November 1, 2006 10:17

I am searching for a regexp on a line with thousands of line, while doing :g/<regexp>/p it prints the whole line, I want it to only print the regexp i.e. \1 variable so that I can copy all those lines manually in a file, and is there any way inside vim only to re-direct the regexp results found into a file.

mail_the_cv--AT--yahoo.com , January 28, 2007 23:46