Re: dynamic tablenames

Isak Hansen
Tue, 23 Jan 2007 16:39:33 -0000

On 1/23/07, Piet Hadermann
<piet.hadermann-/1BeJOKqmzrQT0dZR+AlfA@public.gmane.org>
wrote:
>
>
> Hi List!
>
> May I tap into your combined common sense ?
> I have a legacy logging app that needs to be modernized. At the moment,
Data
> is going to be stored in 5 tables, one table is going to contain more than
> 200 million and 2 others about 60 million rows. Mysql will be used, unless
> someone sees a major advantage in using something else (at the moment data
> is stored in a raw positioned file format, which I''m not keeping).

I''d go with Postgres unless you have some really solid reasons for
staying with mysql.

I suggest you set aside a couple of days to configure both db''s. (pg
ships with very modest defaults -- performance will suffer if you
don''t give it a bit more shared+working memory - and do turn on auto
vacuum), load the data, vacuum analyze, add some indexes, and then try
to run a few queries.

Try some fancy subqueries, for instance. IIRC, mysql performance
suffers badly from not being able to use an index more than once per
query, or something along those lines.

> For performance reasons I''m thinking about storing the data for
each entity
> in separate tables. So instead of a ''logdata'' table with
>200 million rows,
> I''d create ''logdata_001'',
''logdata_002'',... tables where the rowcount of
> even the biggest entity logdata stays below 3 million.
>
> Rails seems to have no problem with me setting the tablename
> (set_table_name) on each incoming request before accessing the database.
> I have this gut feeling though I''m overlooking something nasty.
> Anyone have any comments/suggestions ? Am I about to create my own personal
> maintenance nightmare ?

Keep the data in rows, that''s what RDBMS are made for.

Just make sure you add indexes on the FKs. Periodically clustering on
that index could be beneficial too, if you need to do a lot of seq
scans on single entities.

There''s no benefit to splitting the data across multiple tables and it
makes your app more complex. And don''t forget about what happens when
you''re asked to do something that spans all those entities, for
whatever reason.


Isak


> Thanks in advance for your insights!
>
> Piet.