Hi all, We''ve got a working (at least to some extent) Instiki implementation backed by an ActiveRecord backend and SQLite database. It''s in a Subversion branch http://svn.instiki.org/instiki/branches/instiki-ar/, revision 360. If you find yourself thinking "heck, so what?", one possible consequence is that one day it will drastically reduce rubyonrails.org downtime and increase its performance. Not to mention that all the people running public Instiki instances will get the FastCGI deployment option most of them are crying for. Besides, <hype>Instiki is the most popular end-user application in the Ruby world</hype> :) "Working to some extent" refers to the fact that it passes all available automated tests (unit, functional and Watir), as well as half an hour of random poking around, while running on WEBRick with SQLite DB, under Windows XP. It probably will be heinously slow on any wiki larger than ~100 pages, so we are just half-way through the "make it work, then make it fast" road. Still need to do performance testing, refactor the database for speed, define indexes, implement some caching etc. In other words, all the "wizardry" bits. So, I thought maybe I can get some wizards with Rails production expertise interested to the point of getting involved at this stage. Even if you are not a wizard yet, you are most welcome to try it out and tell me about the problems you encounter. Best regards, Alex
Hi Alex, This is great news - the ability to run Instiki with MySQL and Lighttpd/FastCGI is something that will greatly ease deployment for us. I''ll take a look at it sometime today and see if there''s anything I can help with. Cheers, Ben
Alex Verhovsky wrote:> Even if you are not a wizard yet, you are most welcome to try it out and > tell me about the problems you encounter.Alex, This is a completely off-the-cuff remark, i.e. I''ve not experienced this with Instiki but have some experience with slimserver which uses an SQLite backend (http://slimdevices.com). You may find you run into locking problems with SQLite, ie. SQLite only permitting one process at a time to lock the DB. I suggest you test this extensively. R. -- http://robinbowes.com If a man speaks in a forest, and his wife''s not there, is he still wrong?
Go with caches_page for optimizing, then all you have to really worry about is searching. Thats going to be slow no matter what but at least its going to run against a practically idle database because all the other pages come from static files. This worked wonders for typo. Here is a article[1] by one of the typo devs about our experiences with this caching facility: I can imagine that instiki lends itself much better to static caches then typo did. [1]: http://scottstuff.net/blog/articles/2005/08/08/problems-with-rails-and-page-caching On 8/14/05, Alex Verhovsky <alex-vV7tgcE2N9Nhl2p70BpVqQ@public.gmane.org> wrote:> Hi all, > > We''ve got a working (at least to some extent) Instiki implementation > backed by an ActiveRecord backend and SQLite database. It''s in a > Subversion branch http://svn.instiki.org/instiki/branches/instiki-ar/, > revision 360. > > If you find yourself thinking "heck, so what?", one possible consequence > is that one day it will drastically reduce rubyonrails.org downtime and > increase its performance. Not to mention that all the people running > public Instiki instances will get the FastCGI deployment option most of > them are crying for. Besides, <hype>Instiki is the most popular end-user > application in the Ruby world</hype> :) > > "Working to some extent" refers to the fact that it passes all available > automated tests (unit, functional and Watir), as well as half an hour of > random poking around, while running on WEBRick with SQLite DB, under > Windows XP. It probably will be heinously slow on any wiki larger than > ~100 pages, so we are just half-way through the "make it work, then make > it fast" road. Still need to do performance testing, refactor the > database for speed, define indexes, implement some caching etc. In other > words, all the "wizardry" bits. So, I thought maybe I can get some > wizards with Rails production expertise interested to the point of > getting involved at this stage. > > Even if you are not a wizard yet, you are most welcome to try it out and > tell me about the problems you encounter. > > Best regards, > Alex > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Tobi http://typo.leetsoft.com - Open source weblog engine http://blog.leetsoft.com - Technical weblog
OK so its simple. I had a couple of .9x installations, I am moving them to the latest stable version and so I follow some tweaks and guides here and there and enable file uploads and so crate and empty web, go to the new_web/import page, select a zip exported by the same instiki version, hit import and wait. Then I get the file import success or whatever and now look at the ''all pages'' and low and behold lots of orphaned pages. and lots of missing pages. The reason? any page with a space for example I am a page becomes on export I+am+a+page.textile which when reimported becomes I%26am%26a%26page which is wrong, oh so so so wrong. So my exports are useless, I cant g back to .9 easily because of upgrading rby and a bunch of other stuff. WHY WHY WHY WHY, I really need a fix now I have hundreds of pages which I need to get back online and looking over the code (I am more a php guy) I get the jist of whats going on but I dont see a way to fix it easily. Anyone who can just give me a quick fix, even if its change this line to this I will be eternally grateful.... perhaps a simple url decode of the filename would do? (now using latest stable trunk version, latest ruby) john
you drop in .9 and and extract your most recent backup, then you fix the problem in a development environment before f***ing up something in production.... again! :) rule 1: stop putting spaces in file names, this is just wrong on so many levels. rule 2: production is not a testing environment, stop treating it like one. rule 3: if you were in a serious hurry a mailing list is the last place id be looking for a "quick response". as for your ''migration'' i dont know exactly what your moving from .9 to whatever. and id never expect some script you got from god knows where to rename files for no reason what so ever. On 8/15/05, John Evans <john-M6MfiH3Ctdn10XsdtD+oqA@public.gmane.org> wrote:> OK so its simple. I had a couple of .9x installations, I am moving > them to the latest stable version and so I follow some tweaks and > guides here and there and enable file uploads and so crate and empty > web, go to the new_web/import page, select a zip exported by the same > instiki version, hit import and wait. Then I get the file import > success or whatever and now look at the ''all pages'' and low and > behold lots of orphaned pages. and lots of missing pages. > > The reason? any page with a space for example > > I am a page > > becomes on export > > I+am+a+page.textile > > which when reimported becomes > > I%26am%26a%26page > > which is wrong, oh so so so wrong. So my exports are useless, I cant > g back to .9 easily because of upgrading rby and a bunch of other stuff. > > WHY WHY WHY WHY, I really need a fix now I have hundreds of pages > which I need to get back online and looking over the code (I am more > a php guy) I get the jist of whats going on but I dont see a way to > fix it easily. Anyone who can just give me a quick fix, even if its > change this line to this I will be eternally grateful.... > > perhaps a simple url decode of the filename would do? > > (now using latest stable trunk version, latest ruby) > > john > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Zachery Hostens <zacheryph-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
ack. should have read entire email. some of what i stated will probably be null and void as i have never used instiki. rules 2 & 3 are still in place however :x On 8/15/05, Zachery Hostens <zacheryph-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> you drop in .9 and and extract your most recent backup, then you fix > the problem in a development environment before f***ing up something > in production.... again! :) > > rule 1: stop putting spaces in file names, this is just wrong on so many levels. > > rule 2: production is not a testing environment, stop treating it like one. > > rule 3: if you were in a serious hurry a mailing list is the last > place id be looking for a "quick response". > > as for your ''migration'' i dont know exactly what your moving from .9 > to whatever. and id never expect some script you got from god knows > where to rename files for no reason what so ever. > > > On 8/15/05, John Evans <john-M6MfiH3Ctdn10XsdtD+oqA@public.gmane.org> wrote: > > OK so its simple. I had a couple of .9x installations, I am moving > > them to the latest stable version and so I follow some tweaks and > > guides here and there and enable file uploads and so crate and empty > > web, go to the new_web/import page, select a zip exported by the same > > instiki version, hit import and wait. Then I get the file import > > success or whatever and now look at the ''all pages'' and low and > > behold lots of orphaned pages. and lots of missing pages. > > > > The reason? any page with a space for example > > > > I am a page > > > > becomes on export > > > > I+am+a+page.textile > > > > which when reimported becomes > > > > I%26am%26a%26page > > > > which is wrong, oh so so so wrong. So my exports are useless, I cant > > g back to .9 easily because of upgrading rby and a bunch of other stuff. > > > > WHY WHY WHY WHY, I really need a fix now I have hundreds of pages > > which I need to get back online and looking over the code (I am more > > a php guy) I get the jist of whats going on but I dont see a way to > > fix it easily. Anyone who can just give me a quick fix, even if its > > change this line to this I will be eternally grateful.... > > > > perhaps a simple url decode of the filename would do? > > > > (now using latest stable trunk version, latest ruby) > > > > john > > _______________________________________________ > > Rails mailing list > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > > http://lists.rubyonrails.org/mailman/listinfo/rails > > > > > -- > Zachery Hostens <zacheryph-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> >-- Zachery Hostens <zacheryph-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
Have you considered doing a UNIX kind of replace on the filenames? Replace all + in the .textile files with spaces? Just throwing it out there. On 8/15/05, John Evans <john-M6MfiH3Ctdn10XsdtD+oqA@public.gmane.org> wrote:> OK so its simple. I had a couple of .9x installations, I am moving > them to the latest stable version and so I follow some tweaks and > guides here and there and enable file uploads and so crate and empty > web, go to the new_web/import page, select a zip exported by the same > instiki version, hit import and wait. Then I get the file import > success or whatever and now look at the ''all pages'' and low and > behold lots of orphaned pages. and lots of missing pages. > > The reason? any page with a space for example > > I am a page > > becomes on export > > I+am+a+page.textile > > which when reimported becomes > > I%26am%26a%26page > > which is wrong, oh so so so wrong. So my exports are useless, I cant > g back to .9 easily because of upgrading rby and a bunch of other stuff. > > WHY WHY WHY WHY, I really need a fix now I have hundreds of pages > which I need to get back online and looking over the code (I am more > a php guy) I get the jist of whats going on but I dont see a way to > fix it easily. Anyone who can just give me a quick fix, even if its > change this line to this I will be eternally grateful.... > > perhaps a simple url decode of the filename would do? > > (now using latest stable trunk version, latest ruby) > > john > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- - Aaron ''Jomdom'' Ransley - Web: www.jomdom.net - Mail: jomdom-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
I use Jim Willsher''s bulk rename utility under windows to rename thousands of files and strip out stuff like spaces from filenames. http://www.pcworld.com/downloads/file_description/0,fid,22983,00.asp AS ----- Original Message ----- From: "Aaron Ransley" <jomdom-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> To: <rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org> Sent: Tuesday, August 16, 2005 2:09 PM Subject: Re: [Rails] import you mother f*!*er Have you considered doing a UNIX kind of replace on the filenames? Replace all + in the .textile files with spaces? Just throwing it out there. On 8/15/05, John Evans <john-M6MfiH3Ctdn10XsdtD+oqA@public.gmane.org> wrote:> OK so its simple. I had a couple of .9x installations, I am moving > them to the latest stable version and so I follow some tweaks and > guides here and there and enable file uploads and so crate and empty > web, go to the new_web/import page, select a zip exported by the same > instiki version, hit import and wait. Then I get the file import > success or whatever and now look at the ''all pages'' and low and > behold lots of orphaned pages. and lots of missing pages. > > The reason? any page with a space for example > > I am a page > > becomes on export > > I+am+a+page.textile > > which when reimported becomes > > I%26am%26a%26page > > which is wrong, oh so so so wrong. So my exports are useless, I cant > g back to .9 easily because of upgrading rby and a bunch of other stuff. > > WHY WHY WHY WHY, I really need a fix now I have hundreds of pages > which I need to get back online and looking over the code (I am more > a php guy) I get the jist of whats going on but I dont see a way to > fix it easily. Anyone who can just give me a quick fix, even if its > change this line to this I will be eternally grateful.... > > perhaps a simple url decode of the filename would do? > > (now using latest stable trunk version, latest ruby) > > john > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- - Aaron ''Jomdom'' Ransley - Web: www.jomdom.net - Mail: jomdom-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
Andrew Stuart wrote:> I use Jim Willsher''s bulk rename utility under windows to rename thousands > of files and strip out stuff like spaces from filenames.In a similar vein, back when I was still using Windows, I loved CKRename: http://www.musicsucks.com/CKSoft/CKRename/ ~Dave -- Dave Silvester Rent-A-Monkey Website Development Web: http://www.rentamonkey.com/
On Aug 16, 2005, at 2:15 PM, Andrew Stuart wrote:> I use Jim Willsher''s bulk rename utility under windows to rename > thousands > of files and strip out stuff like spaces from filenames. > > http://www.pcworld.com/downloads/file_description/0,fid,22983,00.asp > > AS > > > ----- Original Message ----- > From: "Aaron Ransley" <jomdom-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> > To: <rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org> > Sent: Tuesday, August 16, 2005 2:09 PM > Subject: Re: [Rails] import you mother f*!*er > > > Have you considered doing a UNIX kind of replace on the filenames? > Replace all + in the .textile files with spaces? > > Just throwing it out there.Of course I have, but then it fails to import anything and just ignores everything after a filename with a character it doesn''t like, spaces being one of them. This kinda renders the above useless too. I found a solution though. In file_controller.rb, look for the lines regarding the importer adding entries to the wiki. and just modify page_name in each entry to read CGI.unescape(page_name) so for example wiki.write_page(@web.address, CGI.unescape(page_name), page_content, Time.now, @author) Then it works fine, or as close to it as possible But the importer is not only broken but very temperamental, for example downloading a zip of the textile, unzipping and re-zipping on a mac causes it to not work, seems it has issues with thing like .DS_STORE or whatever, and a few other issues I cant even debug as the amount of information regarding the import isn''t exactly there. But then its off for a reason right. Also knowing Ruby would help :)
Robin Bowes wrote:> You may find you run into locking problems with SQLite, ie. SQLite > only permitting one process at a time to lock the DB. I suggest you > test this extensively.Thanks for the tip. According to http://www.sqlite.org/lockingv3.html, it''s got a lot better in this respect. Which version were you dealing with? Anyway, we want to also support MySQL and Postgres (with an abstraction layer like AR, this should be quite feasible). Alex
Tobias Luetke wrote:>Go with caches_page for optimizing, then all you have to really worry >about is searching. Thats going to be slow no matter what but at least >its going to run against a practically idle database because all the >other pages come from static files. > >I hear you. Have a question though. For the sake of searches, All Pages and especially RSS feeds. Why not cache the raw markup, too, and do it in memory? If we only cache the last revisions this way, even on something as [relatively] big as rubyonrails.org we are probably talking about 2000 pages * 2 kb / page =~ 4 Mb of RAM per FastCGI process. Even if I''m somehow off by an order of magnitude, and it''s 40 Mb (for a rather huge wiki), it''s still not unreasonable for an active web site. Alex
Caching in memory has the same problems as madeline. If someone updates the page you have to somehow invalidate the cache in every fcgi process. Caching needs to be backed by something centralized. Ignoring all the approaches which work but suck (heh) there is basically the file system and memcached. Memcached is perfect fit for memory caching because its blazingly fast and offers great features like timed expiry. The downside is that you will require your users to start and maintain a memcached server. Ruling this out this leaves you with the file system, you can either write out the memory cache as PStore and load it per request or you just dump the entire html file in the web servers root and save yourself the trouble. The beauty about the caches_page approach is that the requests for the files completely bypass rails. A good web server is able to send the html form disk to network card using a DMA and bypassing even the CPU. Hope this helps. On 8/16/05, Alex Verhovsky <alex-vV7tgcE2N9Nhl2p70BpVqQ@public.gmane.org> wrote:> Tobias Luetke wrote: > > >Go with caches_page for optimizing, then all you have to really worry > >about is searching. Thats going to be slow no matter what but at least > >its going to run against a practically idle database because all the > >other pages come from static files. > > > > > I hear you. Have a question though. > > For the sake of searches, All Pages and especially RSS feeds. Why not > cache the raw markup, too, and do it in memory? If we only cache the > last revisions this way, even on something as [relatively] big as > rubyonrails.org we are probably talking about 2000 pages * 2 kb / page > =~ 4 Mb of RAM per FastCGI process. Even if I''m somehow off by an order > of magnitude, and it''s 40 Mb (for a rather huge wiki), it''s still not > unreasonable for an active web site. > > Alex > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >-- Tobi http://typo.leetsoft.com - Open source weblog engine http://blog.leetsoft.com - Technical weblog