Wikia

Vim Tips Wiki

Bastl/Notes on the import

Talk0
1,610pages on
this wiki

< User:Bastl

For the records: Here some notes on the import of http://www.vim.org/tips to this wiki.

I (bastl) joined the discussion on vim@vim.org on the wiki thing late, but since I love the idea (and technical possibilities) of wikis, I joined in and in the end did it on my own. But I was standing on the shoulders of giants. Let me give some credits here:

Credits Edit

  • vimtips.py by Ali Polatel

This was a great python script, I used and adapted heavily. Well documented, well structured. It was very easy to dive in with nothing more than some general object-oriented experience and some more specific experiences with Regexes. (have you read Mastering Regular Expressions ??)

A Mediawiki bot that allows to upload pages in bulk. Not too elegant, but useful.

Even cooler than bulkinsert.php: A filesystem that allows to deal with mediawiki articles as with normal files. Edit them with vim, moven them with mv, copy them with cp. Great piece of work.

  • mwdumper

A great little tool to process mediawiki dumpfiles.

The Wikia community. Very kind ppl that help immediatly with everything that's related to mediawiki or wikia. Thanks to Splarka, Rieke, Datrio, Sannse and everyone I forgot ... The support is great.

Tom originally started this wiki, and pushed the discussion on vim@vim.org forward by setting arbitrary deadlines. Very cool :-)

  • vim@vim.org

The maillist of vim was quite eager in discussion of this task. Thanks for all tips and remarks!

Procedure Edit

  1. wget all the HTML from http://www.vim.org/tips
  2. install a local mediawiki to test, mount it with wikipediafs
  3. use vimtips.py to generate files directly in the wikipediafs (two files at once: VimTip123 as a redirect to the real page)
  4. some tips failed since wikipediafs is a very young tool. catch these tips and process them with bulkinsert.php to my local mediawiki.
  5. do a dump export of the local mediawiki
  6. post process it with mwdumper (filter out only vimtips)
  7. send the dumpfile to wikia.com and let the wikia-admins do a dump import

Sounds complicated? It was :-) In fact I repeated steps 2 to 4 very often for testing purposes. Regexes on all those vim commands (e.g. [[) are really nasty. And the many different tools and programming languages (python, php, bash, mediawiki-markup, regexes) made life additionally complicated, but in the end it was fun ...


Notes on vimtips.py Edit

to be done


bastl 15:42, 12 June 2007 (UTC)

Around Wikia's network

Random Wiki