Has anyone already put together the ability to use text files as a storage container? One thought is to have DNS style text files as the backend. So when I write and read records, it is writing and reading the DNS text files. There are other files sets that have a similar style in addition to DNS. The first thing that jumps out at me though is that editing these files directly from an app is a bad idea. Maybe it is just better to read the DNS files, parse them, store them in a SQL database, do what I gotta do, then write them back. One of the problems is that these files have a potential to be modified by some other entity. I don''t have complete control over the contents of the files. So I would have to constantly check the integrity of the DB every time I look at them. That''s why I thought using as a backend might get me to where I wanted to go. Maybe these are just wacky thoughts and I should go sit in the corner for a while facing the wall until I have figured out a better way. Any ideas? Encouragement? Banashiment? :) Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
> One of the problems is that these files have a potential to be modified > by some other entity. I don''t have complete control over the contents > of the files. So I would have to constantly check the integrity of the > DB every time I look at them.Well, I''m FAR from a guru, but if I were doing this, and assuming that the files won''t be changing constantly, I''d go the SQL route. Just check the modified date of the file, and if it''s newer then the cached version, read it into the database. If the databased version is modified by the web app, just update the file when you update the database. This should be alot faster then reading the entire text file every time, especially if the files are potentially large. You''ll need to deal with the possibility that the disk file is modified while the web app is editing it, but that''s true regardless of your storage method.
If you really want to use a DB to manage your DNS-style data, you could do the following: - load your DNS data into a Postgres (note: *not* MySQL) database, in a Rails-friendly format - put a trigger on the appropriate table/s so that, when any INSERT/UPDATE/DELETE changes are made to the table/s, data is automatically extracted from the DNS database table/s and written as flat files in the appropriate format. As far as I''m aware, triggers are still in beta on MySQL, but you could do this using Postgres, Oracle, DB2, ... I know this can be done using Postgres, and pretty sure you could do it with Oracle or DB2 as well - put a Rails front-end on your database - enjoy! Dave Mitchell On 6/21/05, Mike <mbarsalou-gwwlNbBYuTWB+jHODAdFcQ@public.gmane.org> wrote:> Has anyone already put together the ability to use text files as a > storage container? > > One thought is to have DNS style text files as the backend. So when I > write and read records, it is writing and reading the DNS text files. > > There are other files sets that have a similar style in addition to DNS. > > The first thing that jumps out at me though is that editing these files > directly from an app is a bad idea. > > Maybe it is just better to read the DNS files, parse them, store them > in a SQL database, do what I gotta do, then write them back. > > One of the problems is that these files have a potential to be modified > by some other entity. I don''t have complete control over the contents > of the files. So I would have to constantly check the integrity of the > DB every time I look at them. > > That''s why I thought using as a backend might get me to where I wanted to go. > > Maybe these are just wacky thoughts and I should go sit in the corner > for a while facing the wall until I have figured out a better way. > > Any ideas? Encouragement? Banashiment? :) > > Mike B. > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Have you seen a product called PowerDNS (http://www.powerdns.com)? I have used it previously and it is very powerful (and fast) and uses a MySQL backend. -----Original Message----- From: rails-bounces-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org [mailto:rails-bounces-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org] On Behalf Of Mike Sent: Tuesday, 21 June 2005 9:33 AM To: rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org Subject: [Rails] text files as backend Has anyone already put together the ability to use text files as a storage container? One thought is to have DNS style text files as the backend. So when I write and read records, it is writing and reading the DNS text files. There are other files sets that have a similar style in addition to DNS. The first thing that jumps out at me though is that editing these files directly from an app is a bad idea. Maybe it is just better to read the DNS files, parse them, store them in a SQL database, do what I gotta do, then write them back. One of the problems is that these files have a potential to be modified by some other entity. I don''t have complete control over the contents of the files. So I would have to constantly check the integrity of the DB every time I look at them. That''s why I thought using as a backend might get me to where I wanted to go. Maybe these are just wacky thoughts and I should go sit in the corner for a while facing the wall until I have figured out a better way. Any ideas? Encouragement? Banashiment? :) Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
Thanks for the suggestions. One thing I didn''t make clear enough is that the files "look" like DNS files....I used that as an example to the type of file to which I am referring. Another example might be Nagios configuration files. There are a bunch more applications that use this sort of config file. I am looking for a way to leverage Ruby to use these files, without complicating the process for applications that use those types of file formats. In other words, keep the file formats as they are but use Ruby to manipulate them. The end goal is to allow me to modify them with either the Ruby web interface or directly modify the text files. Obviously there is a locking issue here, but that aside, I wanted to abstract the access to the files to use the "standard" Action calls. So have I engaged on an impossible task? I''m not asking anyone to do it for me, just guidance on how I might go about it. Another potential reason for doing this is to allow a migration from the text style config files to a LDAP or Postgres setup with having to change the Ruby code. Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.
Mike <mbarsalou-gwwlNbBYuTWB+jHODAdFcQ@public.gmane.org> writes:> The end goal is to allow me to modify them with either the Ruby web > interface or directly modify the text files. Obviously there is a > locking issue here, but that aside, I wanted to abstract the access to > the files to use the "standard" Action calls.There has been some discussion about doing Rails web development without ActiveRecord. If you don''t really need a database, I think this is a perfectly reasonable thing to do. The only requirement is to have some data model objects that represent your data files. Just have your controllers use those models instead of AR models. -- doug-jGAhs73c5XxeoWH0uzbU5w@public.gmane.org
>> The end goal is to allow me to modify them with either the Ruby web >> interface or directly modify the text files. Obviously there is a >> locking issue here, but that aside, I wanted to abstract the access >> to the files to use the "standard" Action calls.> There has been some discussion about doing Rails web development > without ActiveRecord. If you don''t really need a database, I think > this is a perfectly reasonable thing to do. The only requirement is > to have some data model objects that represent your data files. Just > have your controllers use those models instead of AR models.That''s what I will probably end up doing, however, the real end goal here is the future migration of these text based files to using LDAP, Postgres, etc. Maybe the earlier suggestion of loading the text files into Postgres if the data has changed is the way to go, since I would have to spend so much time making the text files work, only to migrate from them later. It just seemed to me that as new software that comes along, that they may initially choose to use these types of text files. Having that type of backend helps move folks towards ruby, then eventually to using a "proper" database backend. I guess I can always do both! :) Thanks for the converstation. Mike B. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program.