Bastl/Notes on the import
Talk0this wiki
< User:Bastl
For the records: Here some notes on the import of http://www.vim.org/tips to this wiki.
I (bastl) joined the discussion on vim@vim.org on the wiki thing late, but since I love the idea (and technical possibilities) of wikis, I joined in and in the end did it on my own. But I was standing on the shoulders of giants. Let me give some credits here:
Credits
- vimtips.py by Ali Polatel
This was a great python script, I used and adapted heavily. Well documented, well structured. It was very easy to dive in with nothing more than some general object-oriented experience and some more specific experiences with Regexes. (have you read Mastering Regular Expressions ??)
A Mediawiki bot that allows to upload pages in bulk. Not too elegant, but useful.
- wikipediafs by Mathieu Blondel
Even cooler than bulkinsert.php: A filesystem that allows to deal with mediawiki articles as with normal files. Edit them with vim, moven them with mv, copy them with cp. Great piece of work.
- mwdumper
A great little tool to process mediawiki dumpfiles.
- #wikia on freenode
The Wikia community. Very kind ppl that help immediatly with everything that's related to mediawiki or wikia. Thanks to Splarka, Rieke, Datrio, Sannse and everyone I forgot ... The support is great.
Tom originally started this wiki, and pushed the discussion on vim@vim.org forward by setting arbitrary deadlines. Very cool :-)
- vim@vim.org
The maillist of vim was quite eager in discussion of this task. Thanks for all tips and remarks!
Procedure
- wget all the HTML from http://www.vim.org/tips
- install a local mediawiki to test, mount it with wikipediafs
- use vimtips.py to generate files directly in the wikipediafs (two files at once: VimTip123 as a redirect to the real page)
- some tips failed since wikipediafs is a very young tool. catch these tips and process them with bulkinsert.php to my local mediawiki.
- do a dump export of the local mediawiki
- post process it with mwdumper (filter out only vimtips)
- send the dumpfile to wikia.com and let the wikia-admins do a dump import
Sounds complicated? It was :-) Many different tools and programming languages (python, php, bash, mediawiki-markup, regexes) made life complicated, but in the end it was fun ...
Notes on vimtips.py
to be done
bastl 15:42, 12 June 2007 (UTC)