Changes: Faster loading of large files

Revision as of 17:28, 19 October 2015

Tip 343 Printable Monobook Previous Next

created 2002 · complexity intermediate · author Erik Remmelzwaal · version 6.0

A user can sometimes experience long load times on very large files. While the definition of a very large file depends on the user, their machine specifications, and their patience, at one point or another any user can experience a load time delay. A method to load these files faster is to remove some of the options available like undo histories, syntax parsing, and swap file creation.

By adding the following to the .vimrc file, the user can automatically remove certain options for files over a certain size (in this example, 10mb).

:let g:LargeFile=10
:e

" Protect large files from sourcing and other overhead.
" Files become read only
if !exists("my_auto_commands_loaded")
  let my_auto_commands_loaded = 1
  " Large files are > 10M
  " Set options:
  " eventignore+=FileType (no syntax highlighting etc
  " assumes FileType always on)
  " noswapfile (save copy of file)
  " bufhidden=unload (save memory when other file is viewed)
  " buftype=nowrite (file is read-only)
  " undolevels=-1 (no undo possible)
  let g:LargeFile = 1024 * 1024 * 10
  augroup LargeFile
    autocmd BufReadPre * let f=expand("<afile>") | if getfsize(f) > g:LargeFile | set eventignore+=FileType | setlocal noswapfile bufhidden=unload buftype=nowrite undolevels=-1 | else | set eventignore-=FileType | endif
    augroup END
  endif

Comments

Improvement proposal

I changed the code a bit here and the final result is the following. Differences are:

Files with filesize too large are recognized too (getfsize = -2)
Readability
Message that options are changed at startup
Got rid of things like my_autocommands_loaded (I didn't get the use...)
And what about the first <pre.. block? (didn't get the use too...)

" file is large from 10mb
let g:LargeFile = 1024 * 1024 * 10
augroup LargeFile 
 autocmd BufReadPre * let f=getfsize(expand("<afile>")) | if f > g:LargeFile || f == -2 | call LargeFile() | endif
augroup END

function LargeFile()
 " no syntax highlighting etc
 set eventignore+=FileType
 " save memory when other file is viewed
 setlocal bufhidden=unload
 " is read-only (write with :w new_filename)
 setlocal buftype=nowrite
 " no undo possible
 setlocal undolevels=-1
 " display message
 autocmd VimEnter *  echo "The file is larger than " . (g:LargeFile / 1024 / 1024) . " MB, so some options are changed (see .vimrc for details)."
endfunction

--DartThis (talk) 13:34, March 7, 2015 (UTC)

@@ Line 1: / Line 1: @@
-{{Duplicate|611}}
-{{review}}
 {{TipImported
 |id=343
 |previous=341
 |next=344
-|created=October 12, 2002
+|created=2002
 |complexity=intermediate
 |author=Erik Remmelzwaal
 |version=6.0
 |rating=52/28
-|category1=
+|category1=File Handling
-|category2=
+|category2=Plugin
 }}
+A user can sometimes experience long load times on very large files. While the definition of a very large file depends on the user, their machine specifications, and their patience, at one point or another any user can experience a load time delay. A method to load these files faster is to remove some of the options available like undo histories, syntax parsing, and swap file creation.
-In the past I experienced long loading times for large files (size over 10MB). These files are normally generated by SQL tracing, XML message based protocols tracing etc. One of the causes of long loading times was syntax parsing, creating swap file etc. Normally one want to view these files and remove not relevant details by deleting lines, but do not want to have undo capabilities and auto recalculation of syntax highlighting.
-The code below, I put in my vimrc to switch off a number of defaults for large files.
-One can modify the g:LargeFile variable and reload a file to test:
+By adding the following to the .vimrc file, the user can automatically remove certain options for files over a certain size (in this example, 10mb).
 <pre>
 :let g:LargeFile=10
@@ Line 35: / Line 30: @@
   " noswapfile (save copy of file)
   " bufhidden=unload (save memory when other file is viewed)
-  " buftype=nowritefile (is read-only)
+  " buftype=nowrite (file is read-only)
   " undolevels=-1 (no undo possible)
   let g:LargeFile = 1024 * 1024 * 10
@@ Line 44: / Line 39: @@
 </pre>
-==Comments==
+==See also==
+*{{script|id=1506|text=Edit large files quickly}} plugin to handle many of the above changes and more
-Can we just set scratch buffer for such purpose? {{help|special-buffers}}
+==Comments==
-----
-File will load faster if you open it read-only: vim -R HugeFile.txt
+===Improvement proposal===
-----
+I changed the code a bit here and the final result is the following. Differences are:
-The {{script|id=1506|text=Edit large files quickly}} script is better.
+* Files with filesize too large are recognized too (getfsize = -2)
+* Readability
+* Message that options are changed at startup
+* Got rid of things like my_autocommands_loaded (I didn't get the use...)
+* And what about the first <pre.. block? (didn't get the use too...)
+ " file is large from 10mb
-----
+ let g:LargeFile = 1024 * 1024 * 10
+ augroup LargeFile
+  autocmd BufReadPre * let f=getfsize(expand("<afile>")) | if f > g:LargeFile || f == -2 | call LargeFile() | endif
+ augroup END
+ function LargeFile()
+  " no syntax highlighting etc
+  set eventignore+=FileType
+  " save memory when other file is viewed
+  setlocal bufhidden=unload
+  " is read-only (write with :w new_filename)
+  setlocal buftype=nowrite
+  " no undo possible
+  setlocal undolevels=-1
+  " display message
+  autocmd VimEnter *  echo "The file is larger than " . (g:LargeFile / 1024 / 1024) . " MB, so some options are changed (see .vimrc for details)."
+ endfunction
+--[[User:DartThis|DartThis]] ([[User talk:DartThis|talk]]) 13:34, March 7, 2015 (UTC)