Advanced ActiveRecord Question

Sunny Hirai
Sat, 23 Dec 2006 23:54:22 -0000

First of all, I just want to apologize for the length of the post and to
thank all of you who take the time to get through it.

I am an experienced developer and have written an eCommerce website
builder that hosts tens of thousands of customers and we offer a
reseller program to hundreds of businesses; however, I am inexperienced
in Ruby and in Rails.

Through the books I''ve read on Ruby on Rails (and reading through
ActiveRecord source code), there appears to be no stock answer to this
question. It will probably require extending ActiveRecord, but I''m not
sure how I should do this without breaking the rest of ActiveRecord.

The question will sound familiar, but it''s not the same.

Here''s what I want to do:

1. I want to be able to store multiple fields into a single field. I''d
like to do it by adding a method call something like this during
initialization:

class User < ActiveRecord::Base
  encoded_field :encoded_stuff {
    age => "23",
    hair_color => "black",
    eye_color => "brown"
  }
end

:encoded_stuff is a field in a table. The "hash" passed in as the
second
argument to encoded_field is what I call the "prototype" because it is
a
prototype of what the data inside :encoded_stuff should look like. The
keys in the hash will be converted into attributes in ActiveRecord.

2. When a new activeRecord object is created, it will have age,
hair_color and eye_color (the prototype keys) as parts of its attributes
and they should be accessible like regular attributes.

John = User.new
John.age  --> "23"
John.hair_color --> "black"
John.hair_color = "blonde"
John.hair_color --> "blonde"

Note that while these attributes can be accessed, there is technically
no such fields (e.g. age, hair_color and eye_color) in the database.

3. At save time, all those attributes get encoded into the field
"encoded_stuff" above. The encoding I have is proprietary but similar
to
YAML, only very lightweight and fast to process. I''m not too worried
about the encoding code because it is simple to write or I might end up
using YAML anyways.

John.save

The above call encodes the attributes from the prototype (age,
hair_color and eye_color) into the db field "encoded_stuff" something
like this:

age`23|
hair_color`black|
eye_color`blonde|

This is half the functionality I need.

4. At read time, the field "encoded_stuff" gets decoded back into
attributes.

John = User.find(1)
John.age --> "23"
John.hair_color --> "blonde"

However, if the value for an attribute is missing in the encoded data,
it will use the value provided in the "prototype".

Right now, if we did this

John.has_glasses

Would probably return some sort of has_glasses method not found error.

But, let''s say we redeclared the prototype to add a
"has_glasses"
attribute like this:

class User < ActiveRecord::Base
  encoded_field :encoded_stuff {
    age => "23",
    hair_color => "black",
    eye_color => "brown",
    has_glasses => "yes"
  }
end

Now, let''s try it again

John = User.find(1)
John.has_glasses --> "yes"

There is no "has_glasses" value encoded in the
"encoded_stuff" field;
however, since it was in the prototype, the default value "yes" has
been
added to it.

THE HARD PART?

I think the hard part is knowing where to inject the encoding/decoding.

I''m afraid if they are not added in the correct places, things like the
validation code in ActiveRecord would fail because the attributes would
get encoded into a single field. If this happened before the validation,
there would be problems because the attributes that the validation
checks against are simply missing.

For example, if you were checking the eye_color was not "yellow", but
the encoded already happened, there would be no "eye_color" attribute
to
check against.

WHY DO THIS?

I''m sure a lot of people will chime in with other solutions, some of
which may be valid, so I wanted to explain why I need the ability to add
an encoded_field with a prototype.

When my application was small, adding fields to the database was easy.
You just go into the schema, and add the new field that you need. As the
tables got bigger, it became impossible to make changes to the database
without having long periods of downtime. As I anticipate the record
counts to go into the hundreds of millions, adjusting the table could
take a long time (minutes to hours).

By encoding the fields, I can change the application by simply changing
the prototype of the encoded_field. The application would not break. In
fact, the data wouldn''t even need to be refactored at all.

Basically it means we can keep modifying our application in an agile
manner without having to schedule large 3:00 a.m. downtimes which are
scary and can be dangerous.

I already have this set up under our current development environment and
it works great.

I know I can store serialized Objects using YAML but it doesn''t solve
problems like when we need to add more attributes. Furthermore, I want
the storage to be transparent in that one day, I may decide that "age"
is an important enough field that I want to actually have it be part of
the database schema. Under the proposed method above, I can do this
without having to change any of the code that interacts with
ActiveRecord because the attributes will look identical, only the
storage would be different.

Under my application, I anticipate having dozens of attributes encoded
into a single field and they could change quite regularly.

I know this may cause some unsolvable issues like table joins, but since
I probably wouldn''t do table joins on encoded data anyways, I think
many
of these issues could be ignored.

Thanks in advance for any help.

Sunny Hirai
CEO, MeZine.com Inc.

--
Posted via http://www.ruby-forum.com/.