Re: Database recommendations?

Philip Nelson
Sat, 24 Feb 2007 11:11:19 -0000

Thanks for that.   It should be finished sometime this week.

Phil

On Saturday 24 February 2007 08:45, Mohit Sindhwani
wrote:
> Thanks Philip for a very comprehensive summary of various ideas.
I''ve
> toyed with the idea of using DB2 for Rails though haven''t got
around to
> it.  But your mail does push me to consider revisiting it again.
>
> I''d like to get my hands on a copy of your presentation, and if no
one
> else volunteers to review it, put me on the list (my reluctance is due
> to my lack of experience rather than lack of willingness).
>
> Cheers
> Mohit.
>
> Philip Nelson wrote:
> > Paul,
> >
> > Been watching this thread with interest, being a database person
myself.
> >
> > Asking for a database recommendation is a pretty dangerous thing to do
> >
> > : DBMS
> >
> > debates can get just about as heated as those over preferred editor or
> > IDE.
> >
> > In many ways, the choice of DBMS comes down to two things -
> >
> > 1) Experience (folks choose what they know and are comfortable with)
> >
> > 2) Money (how much you are prepared to spend for the product and
support)
> >
> > I am not a MySQL person myself (my preferred DBMS is IBM''s
DB2, and
> > for that
> >
> > my preferred operating platform is Linux).   I have used it on
> > occasion, just
> >
> > as I have been exposed to Oracle, MS SQL Server, Sybase, Informix and
> > IMS in
> >
> > my years in the field.
> >
> > My recommendation to you is that if you are generally comfortable with
> > MySQL
> >
> > then that would be where you should stay.   The obvious advice would
> > be to
> >
> > raise a problem report with MySQL support.   I would not advise anyone
> > who is
> >
> > running a 24x7 database which is critical to the life of their
> > business to
> >
> > run without some sort of support contract.   I''m sure that
MySQL support
> >
> > would help diagnose your problem.
> >
> > If you do want to consider a change, then you have the opportunity to
use
> >
> > almost all of the databases currently available, albeit unsupported,
> > for free
> >
> > these days.    That''s true for DB2 (Express-C), Oracle
(Express
> > Edition) and
> >
> > SQL Server (Express Edition).   But be aware that there all of these
free
> >
> > editions have limitations on their use, some more restrictive than
> > others.
> >
> > The basic details of these are -
> >
> > DB2 Express-C : server can have maximum of 2 processors (these can be
> >
> > multi-core however) and 4 gig of RAM.   No limit on database size
> >
> > Oracle Express : server can only have 1 processor (again multi-core)
> > and 1 gig
> >
> > of RAM.  Database is limited to 4 gig in size.  Only one database
> > allowed per
> >
> > server.   Also no 64-bit support (e.g. AMD64 or EM64T).
> >
> > SQL Server Express : will only use one CPU on a server.   1 gig of RAM
> > only.
> >
> > 4 gig maximum database size.   And, of course, only available on
Windows
> >
> > I think you will agree the DB2 Express-C is much less restricted than
the
> >
> > others.    I can thoroughly recommend it as an excellent platform for
> > 24x7
> >
> > operations.   It runs beautifully on Linux, and especially on an
x64-64
> >
> > (AMD64 or Intel Xeon with EM64T) platform.
> >
> > I''m giving a presentation in May at IDUG
(http://www.idug.org) on
> > using DB2
> >
> > with Ruby on Rails.   It is nearly finished (the final version is due
> > in less
> >
> > than a fortnight) and if any wants to help me review it ;-) then drop
> > me an
> >
> > email.   It goes through in some detail using the ibm_db2 adaptor from
> > IBM
> >
> > Alphaworks.    It doesn''t show you how to install DB2, since
my
> > audience will
> >
> > know this, but I can help there if you want : given a Linux box, an
SSH
> >
> > connection and a root password my record for going from tarball to
> > working
> >
> > DB2 system, including sample database, is 12 minutes.
> >
> > Anyway, as I said before : stay with what you know if you can resolve
the
> >
> > issue.   If you want to look elsewhere you could do a lot worse than
> > DB2 !!!
> >
> > HTH
> >
> > Phil
> >
> > On Thursday 22 February 2007 21:13, Paul Butcher wrote:
> > > Ilan Berci wrote:
> > > > I think your real problem is trying to diagnose why your
slave is
> > > >
> > > > crashing before you assume that you are running up against
the upper
> > > >
> > > > limits of MySQL replication.
> > >
> > > I''d certainly welcome advice on how to perform this
diagnosis. Part of
> > >
> > > the problem that we have is that MySQL provides us with almost no
help
> > >
> > > whatsoever in diagnosing the problem.
> > >
> > >
> > >
> > > We know what was the last statement which went through before the
> > > crash,
> > >
> > > but it''s "just a query". As far as we can
tell, the problem is that the
> > >
> > > master and the slave got "out of whack" at some point
in the past (and
> > >
> > > we have no means to tell when this point was). The point of
failure is
> > >
> > > some stage after that when an otherwise entirely innocuous query
runs.
> > >
> > >
> > >
> > > Diagnosing the problem is also not helped by the fact that it has
only
> > >
> > > happened twice in 14 months! Most of the time everything runs
just
> > > fine.
> > >
> > > And we''ve not found a way to reproduce it (we do have
automated stress
> > >
> > > tests, but they don''t seem to do the trick).
> > >
> > >
> > >
> > > We''re thinking of moving database at least in part
because, quite apart
> > >
> > > from this problem, we know that we''re going to hit a
wall with MySQL
> > >
> > > replication anyway. The slave exists to provide a failover if the
> > > master
> > >
> > > dies. Let''s imagine that happens 6 months down the line.
We can
> > > failover
> > >
> > > to the slave (assuming that the replication is working at the
time!),
> > >
> > > but at some point we''ll need to recreate a replication
relationship
> > > from
> > >
> > > the new master to a new slave. MySQL replication requires us to
take
> > > the
> > >
> > > master down while we create the slave. With a database which is
an
> > > order
> > >
> > > of magnitude larger than we have right now, which is the size we
expect
> > >
> > > it to be in 6 months, this is going to mean being down for an
hour -
> > >
> > > which is completely unacceptable for our app.
> > >
> > >
> > >
> > > So we''ve been thinking of moving anyway - the current
problems we''ve
> > >
> > > been experiencing have just accelerated our thinking.
> > >
> > > >  What are the details of your master/slave
> > > >
> > > > setup?, (a)synchronous?, distributed query load to slave?,
> > > >
> > > > auto-generated keys? load?.
> > >
> > > Most of our tables are InnoDB, but we do have a couple of MyISAM
tables
> > >
> > > (because we use MySQL fulltext indices on those tables). The
slave runs
> > >
> > > read-only and replicates from the binary log on the master (which
is a
> > >
> > > very standard MySQL replication setup as far as I can tell). Just
about
> > >
> > > all tables are "standard" RoR style tables, so they
have autoincrement
> > >
> > > "id" columns, but are pretty much "plain
vanilla" apart from that. No
> > >
> > > stored procedures, no transactions (other than the implicit
> > > transactions
> > >
> > > that InnoDB creates as part of its normal operation). We do use
foreign
> > >
> > > key constraints though (although I can''t see why that
should cause any
> > >
> > > problems).
> > >
> > > > It should be fairly straight forward to look at the db logs
on the
> >
> > slave
> >
> > > > to see the last statement that went through before the
crash..   I
> > > > also
> > > >
> > > > guess it''s obvious to assume that your slave is a
dedicated db
> > > > server..
> > > >
> > > > right?   Is the crash dependant upon load?
> > >
> > > Yes, the slave is a dedicated server. It''s hard to say
whether the
> > > crash
> > >
> > > is dependent on load - as it''s only happened twice
it''s hard to see a
> > >
> > > pattern!
> > >
> > >
> > >
> > > Thanks for any suggestions you can offer!
> > >
> > >
> > >
> > > paul.butcher->msgCount++