asterisk users - Oct 2006 - Asterisk RT on Disk On Module Performance andDurability

I'm just going to jump in here, and ask a stoopid question.
 
How could you possibly write a multi-user front end in AJAX without using a
database backend like MySQL?
 
Doug.

-----Original Message-----
From: Erick Perez [mailto:eaperezh@gmail.com]
Sent: Monday, October 09, 2006 1:58 PM
To: Asterisk Users Mailing List - Non-Commercial Discussion
Subject: Re: [asterisk-users] Asterisk RT on Disk On Module Performance
andDurability


Jeremy, Cohen, Kris, thanks to all of you.
 
Indeed after reading the Sandisk paper it shed a lot of light on this matter.
The whole idea is to have a large scale system with no moving parts (we call a
large system something with 250 users, at least down here ;-)  )
 
the whole idea is for a customer that needs an IVR in 4 languages with
autoattendant, extensive CDR and plotted usage patterns as well as voicemail.
Voicemail will be used *a lot*, probably about one thousand voicemails per day
and the customer does not want VM-to-Email (God knows why!).
 
Oh, and the whole idea of the database is because the developers are working in
an AJAX based interface that does the asterisk config/plotting/vm/day-to-day
stuff with ARA, so a db is needed.
I started learning asterisk with flat files...it works for me...but hey...times
are changing.
 
Who knows, maybe the whole thing can be fitted in ram (except for the vm
part)...we'll see. I had to ask anyway, but i don't like Dbs
either....it adds and extra breakup layer (maybe Im kind of outdated).
 
Smaller iPBXs will definitely be CF and RAM based and I, at least, will force
VMtoEmail and do all the processing in RAM.
 
Again,
 
Thanks to all of you.

P.D. I will later follow this thread with the full working configs that will
take place at user premises. And for the sake of the test. I will try to kill a
sandisk USB with the full config.

 
On 10/8/06, Kristian Kielhofner < kris@krisk.org> wrote: 

Jeremy McNamara wrote:
> Tzafrir Cohen wrote:
>  > Hmmmm, I'm not sure that this is exactly the data you're
after.
>>
>> You're looking for the ammounts of writes for the disk block that
gets
>> the most writes.
>>
>> E.g: for a standard ext3 filesystem, the journal area would probably
>> have very frequent writes, whereas most of the system would remain
>> mostly unchanged.
>
>
> Again, if the embedded system is setup properly, there is NO writing to
> the flash during normal operations, thus the device won't be killed by 
> its alleged 2 million write limitation.
>
> Kris and I had a quick discussion on this topic, off-list, and his
> original flash-based device is still in constant operation after 2 years
> and I have flash modules that I purposely tried to kill with writes. It 
> took significant effort to start causing error situations, which were
> very easily detected before the system would become unusable.
>
> Erick, you should focus on having a quick action restoration plan and 
> extra DOMs always readily available.  Then when a failure situation is
> detected, you can react very quickly.
>
>
>
>
> Jeremy McNamara
Jeremy, Erick -

       I have always pointed to this SanDisk whitepaper: 

http://www.sandisk.com/Assets/File/OEM/WhitePapersAndBrochures/RS-MMC/WPaperWearLevelv1.0.pdf

       While it specifically discusses their industrial line of CF cards, it 
is pretty obvious that flash can, and often does, last much longer than
other components in a system when properly implemented.  You will notice
that the SHORTEST expected life of a CF card in their test scenarios was 
over 70 years!  How long is your power supply going to last?  Even if
the consumer level cards had 1/10 the life expectancy, that is still
seven years.  I expect to get at least that from my original AstLinux
system.  It's been two so far, I'll let you know how it is doing in
another five years :).

       JFFS (and similar FSs) are not appropriate for CF cards or DOMs.  They
are meant to be used directly on flash memory and do their own wear 
leveling and in some cases, compression.  All kinds of commercial
devices use JFFS2.  If you are using a CF or DOM with Linux, ext2 is the
best FS to use.  CF cards and DOMs use their own wear leveling, so none
is required in the operating system or file system.  CF cards and DOMs
hide wear leveling from you and expose themselves as an ordinary IDE device.

       I echo Jeremy's conclusions.  With a properly designed operating 
system, decent flash memory, and a reasonable usage pattern, I can tell
you (with a great amount of certainty) that in most situations, CF cards
will outlast just about any hard drive (even SCSI) when used 24/7. 
These days, it really is pretty tough to trash flash.

       However, if you are running a MySQL cluster or something with several,
multi-gigabyte databases, no type of flash memory will last very long! :)

       To get back to answering your question, I HIGHLY recommend that you
avoid MySQL and realtime on your box with a DOM.  Nothing against either
(MySQL or Realtime), but they will probably make your device more 
complicated than it needs to be while substantially shortening the life
of your DOM.  If you absolutely have to use MySQL, you might have better
luck using a MySQL storage engine that uses fewer writes than InnoDB, 
but I am no expert on that.

--
Kristian Kielhofner
_______________________________________________
--Bandwidth and Colocation provided by Easynews.com --

asterisk-users mailing list 
To UNSUBSCRIBE or update options visit:
   http://lists.digium.com/mailman/listinfo/asterisk-users





-- 
------------------------------------------------------------
Erick Perez
Panama Sistemas
Integradores de Telefonia IP y Soluciones Para Centros de Datos
Panama, Republica de Panama
Cel Panama. +(507) 6694-4780 
------------------------------------------------------------ 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20061009/74713a24/attachment.htm

Very very carefully ;)  I'm thinking pizza, and maybe some red-bull...
very little time for sleep

Aaron

On Mon, 2006-10-09 at 14:19 -0600, Douglas Garstang
wrote:
> I'm just going to jump in here, and ask a stoopid question.
>  
> How could you possibly write a multi-user front end in AJAX without
> using a database backend like MySQL?
>  
> Doug.
>         -----Original Message-----
>         From: Erick Perez [mailto:eaperezh@gmail.com]
>         Sent: Monday, October 09, 2006 1:58 PM
>         To: Asterisk Users Mailing List - Non-Commercial Discussion
>         Subject: Re: [asterisk-users] Asterisk RT on Disk On Module
>         Performance andDurability
>         
>         
>         Jeremy, Cohen, Kris, thanks to all of you.
>          
>         Indeed after reading the Sandisk paper it shed a lot of light
>         on this matter. The whole idea is to have a large scale system
>         with no moving parts (we call a large system something
>         with 250 users, at least down here ;-)  ) 
>          
>         the whole idea is for a customer that needs an IVR in 4
>         languages with autoattendant, extensive CDR and plotted usage
>         patterns as well as voicemail. Voicemail will be used *a lot*,
>         probably about one thousand voicemails per day and the
>         customer does not want VM-to-Email (God knows why!). 
>          
>         Oh, and the whole idea of the database is because the
>         developers are working in an AJAX based interface that does
>         the asterisk config/plotting/vm/day-to-day stuff with ARA, so
>         a db is needed.
>         I started learning asterisk with flat files...it works for
>         me...but hey...times are changing.
>          
>         Who knows, maybe the whole thing can be fitted in ram (except
>         for the vm part)...we'll see. I had to ask anyway, but i
don't
>         like Dbs either....it adds and extra breakup layer (maybe Im
>         kind of outdated).
>          
>         Smaller iPBXs will definitely be CF and RAM based and I, at
>         least, will force VMtoEmail and do all the processing in RAM.
>          
>         Again,
>          
>         Thanks to all of you.
>         
>         P.D. I will later follow this thread with the full working
>         configs that will take place at user premises. And for the
>         sake of the test. I will try to kill a sandisk USB with the
>         full config.
>         
>          
>         On 10/8/06, Kristian Kielhofner <kris@krisk.org> wrote: 
>                 Jeremy McNamara wrote:
>                 > Tzafrir Cohen wrote:
>                 >  > Hmmmm, I'm not sure that this is exactly the
data
>                 you're after. 
>                 >>
>                 >> You're looking for the ammounts of writes for
the
>                 disk block that gets
>                 >> the most writes.
>                 >>
>                 >> E.g: for a standard ext3 filesystem, the journal
>                 area would probably
>                 >> have very frequent writes, whereas most of the
>                 system would remain
>                 >> mostly unchanged.
>                 >
>                 >
>                 > Again, if the embedded system is setup properly,
>                 there is NO writing to
>                 > the flash during normal operations, thus the device
>                 won't be killed by 
>                 > its alleged 2 million write limitation.
>                 >
>                 > Kris and I had a quick discussion on this topic,
>                 off-list, and his
>                 > original flash-based device is still in constant
>                 operation after 2 years
>                 > and I have flash modules that I purposely tried to
>                 kill with writes. It 
>                 > took significant effort to start causing error
>                 situations, which were
>                 > very easily detected before the system would become
>                 unusable.
>                 >
>                 > Erick, you should focus on having a quick action
>                 restoration plan and 
>                 > extra DOMs always readily available.  Then when a
>                 failure situation is
>                 > detected, you can react very quickly.
>                 >
>                 >
>                 >
>                 >
>                 > Jeremy McNamara
>                 
>                 Jeremy, Erick -
>                 
>                        I have always pointed to this SanDisk
>                 whitepaper: 
>                 
>                
http://www.sandisk.com/Assets/File/OEM/WhitePapersAndBrochures/RS-MMC/WPaperWearLevelv1.0.pdf
>                 
>                        While it specifically discusses their
>                 industrial line of CF cards, it 
>                 is pretty obvious that flash can, and often does, last
>                 much longer than
>                 other components in a system when properly
>                 implemented.  You will notice
>                 that the SHORTEST expected life of a CF card in their
>                 test scenarios was 
>                 over 70 years!  How long is your power supply going to
>                 last?  Even if
>                 the consumer level cards had 1/10 the life expectancy,
>                 that is still
>                 seven years.  I expect to get at least that from my
>                 original AstLinux
>                 system.  It's been two so far, I'll let you know
how
>                 it is doing in
>                 another five years :).
>                 
>                        JFFS (and similar FSs) are not appropriate for
>                 CF cards or DOMs.  They
>                 are meant to be used directly on flash memory and do
>                 their own wear 
>                 leveling and in some cases, compression.  All kinds of
>                 commercial
>                 devices use JFFS2.  If you are using a CF or DOM with
>                 Linux, ext2 is the
>                 best FS to use.  CF cards and DOMs use their own wear
>                 leveling, so none
>                 is required in the operating system or file
>                 system.  CF cards and DOMs
>                 hide wear leveling from you and expose themselves as
>                 an ordinary IDE device.
>                 
>                        I echo Jeremy's conclusions.  With a properly
>                 designed operating 
>                 system, decent flash memory, and a reasonable usage
>                 pattern, I can tell
>                 you (with a great amount of certainty) that in most
>                 situations, CF cards
>                 will outlast just about any hard drive (even SCSI)
>                 when used 24/7. 
>                 These days, it really is pretty tough to trash flash.
>                 
>                        However, if you are running a MySQL cluster or
>                 something with several,
>                 multi-gigabyte databases, no type of flash memory will
>                 last very long! :)
>                 
>                        To get back to answering your question, I
>                 HIGHLY recommend that you
>                 avoid MySQL and realtime on your box with a
>                 DOM.  Nothing against either
>                 (MySQL or Realtime), but they will probably make your
>                 device more 
>                 complicated than it needs to be while substantially
>                 shortening the life
>                 of your DOM.  If you absolutely have to use MySQL, you
>                 might have better
>                 luck using a MySQL storage engine that uses fewer
>                 writes than InnoDB, 
>                 but I am no expert on that.
>                 
>                 --
>                 Kristian Kielhofner
>                 _______________________________________________
>                 --Bandwidth and Colocation provided by Easynews.com --
>                 
>                 asterisk-users mailing list 
>                 To UNSUBSCRIBE or update options visit:
>                   http://lists.digium.com/mailman/listinfo/asterisk-users
>         
>         
>         
>         -- 
>         ------------------------------------------------------------
>         Erick Perez
>         Panama Sistemas
>         Integradores de Telefonia IP y Soluciones Para Centros de
>         Datos
>         Panama, Republica de Panama
>         Cel Panama. +(507) 6694-4780 
>         ------------------------------------------------------------ 
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com --
> 
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>    http://lists.digium.com/mailman/listinfo/asterisk-users
-- 
Aaron Daniel
Computer Systems Technician
Sam Houston State University
amdtech@shsu.edu
(936) 294-4198

Douglas, Im just the asterisk guy. If they decide to write a cross-browser
multi-tier interface in AJAX, assembly language or Pascal, that's up to them
(the programmers). I will let them know what can/can't be done.

Thinking of that...15 years ago...the last time i used pascal.



On 10/9/06, Douglas Garstang <dgarstang@oneeighty.com>
wrote:
>
>  I'm just going to jump in here, and ask a stoopid question.
>
> How could you possibly write a multi-user front end in AJAX without using
> a database backend like MySQL?
>
> Doug.
>
> -----Original Message-----
> *From:* Erick Perez [mailto:eaperezh@gmail.com]
> *Sent:* Monday, October 09, 2006 1:58 PM
> *To:* Asterisk Users Mailing List - Non-Commercial Discussion
> *Subject:* Re: [asterisk-users] Asterisk RT on Disk On Module Performance
> andDurability
>
> Jeremy, Cohen, Kris, thanks to all of you.
>
> Indeed after reading the Sandisk paper it shed a lot of light on this
> matter. The whole idea is to have a large scale system with no moving parts
> (we call a large system something with 250 users, at least down here ;-)  )
>
> the whole idea is for a customer that needs an IVR in 4 languages with
> autoattendant, extensive CDR and plotted usage patterns as well as
> voicemail. Voicemail will be used *a lot*, probably about one thousand
> voicemails per day and the customer does not want VM-to-Email (God knows
> why!).
>
> Oh, and the whole idea of the database is because the developers are
> working in an AJAX based interface that does the asterisk
> config/plotting/vm/day-to-day stuff with ARA, so a db is needed.
> I started learning asterisk with flat files...it works for me...but
> hey...times are changing.
>
> Who knows, maybe the whole thing can be fitted in ram (except for the vm
> part)...we'll see. I had to ask anyway, but i don't like Dbs
either....it
> adds and extra breakup layer (maybe Im kind of outdated).
>
> Smaller iPBXs will definitely be CF and RAM based and I, at least, will
> force VMtoEmail and do all the processing in RAM.
>
> Again,
>
> Thanks to all of you.
>
> P.D. I will later follow this thread with the full working configs that
> will take place at user premises. And for the sake of the test. I will try
> to kill a sandisk USB with the full config.
>
>
> On 10/8/06, Kristian Kielhofner <kris@krisk.org> wrote:
> >
> > Jeremy McNamara wrote:
> > > Tzafrir Cohen wrote:
> > >  > Hmmmm, I'm not sure that this is exactly the data
you're after.
> > >>
> > >> You're looking for the ammounts of writes for the disk
block that
> > gets
> > >> the most writes.
> > >>
> > >> E.g: for a standard ext3 filesystem, the journal area would
probably
> > >> have very frequent writes, whereas most of the system would
remain
> > >> mostly unchanged.
> > >
> > >
> > > Again, if the embedded system is setup properly, there is NO
writing
> > to
> > > the flash during normal operations, thus the device won't be
killed by
> >
> > > its alleged 2 million write limitation.
> > >
> > > Kris and I had a quick discussion on this topic, off-list, and
his
> > > original flash-based device is still in constant operation after
2
> > years
> > > and I have flash modules that I purposely tried to kill with
writes.
> > It
> > > took significant effort to start causing error situations, which
were
> > > very easily detected before the system would become unusable.
> > >
> > > Erick, you should focus on having a quick action restoration plan
and
> > > extra DOMs always readily available.  Then when a failure
situation is
> > > detected, you can react very quickly.
> > >
> > >
> > >
> > >
> > > Jeremy McNamara
> >
> > Jeremy, Erick -
> >
> >        I have always pointed to this SanDisk whitepaper:
> >
> >
> >
http://www.sandisk.com/Assets/File/OEM/WhitePapersAndBrochures/RS-MMC/WPaperWearLevelv1.0.pdf
> >
> >        While it specifically discusses their industrial line of CF
> > cards, it
> > is pretty obvious that flash can, and often does, last much longer
than
> > other components in a system when properly implemented.  You will
notice
> > that the SHORTEST expected life of a CF card in their test scenarios
was
> >
> > over 70 years!  How long is your power supply going to last?  Even if
> > the consumer level cards had 1/10 the life expectancy, that is still
> > seven years.  I expect to get at least that from my original AstLinux
> > system.  It's been two so far, I'll let you know how it is
doing in
> > another five years :).
> >
> >        JFFS (and similar FSs) are not appropriate for CF cards or
> > DOMs.  They
> > are meant to be used directly on flash memory and do their own wear
> > leveling and in some cases, compression.  All kinds of commercial
> > devices use JFFS2.  If you are using a CF or DOM with Linux, ext2 is
the
> > best FS to use.  CF cards and DOMs use their own wear leveling, so
none
> > is required in the operating system or file system.  CF cards and DOMs
> > hide wear leveling from you and expose themselves as an ordinary IDE
> > device.
> >
> >        I echo Jeremy's conclusions.  With a properly designed
operating
> > system, decent flash memory, and a reasonable usage pattern, I can
tell
> > you (with a great amount of certainty) that in most situations, CF
cards
> > will outlast just about any hard drive (even SCSI) when used 24/7.
> > These days, it really is pretty tough to trash flash.
> >
> >        However, if you are running a MySQL cluster or something with
> > several,
> > multi-gigabyte databases, no type of flash memory will last very long!
> > :)
> >
> >        To get back to answering your question, I HIGHLY recommend that
> > you
> > avoid MySQL and realtime on your box with a DOM.  Nothing against
either
> > (MySQL or Realtime), but they will probably make your device more
> > complicated than it needs to be while substantially shortening the
life
> > of your DOM.  If you absolutely have to use MySQL, you might have
better
> > luck using a MySQL storage engine that uses fewer writes than InnoDB,
> > but I am no expert on that.
> >
> > --
> > Kristian Kielhofner
> > _______________________________________________
> > --Bandwidth and Colocation provided by
Easynews.com<http://easynews.com/>--
> >
> > asterisk-users mailing list
> > To UNSUBSCRIBE or update options visit:
> >   http://lists.digium.com/mailman/listinfo/asterisk-users
> >
>
>
>
> --
> ------------------------------------------------------------
> Erick Perez
> Panama Sistemas
> Integradores de Telefonia IP y Soluciones Para Centros de Datos
> Panama, Republica de Panama
> Cel Panama. +(507) 6694-4780
> ------------------------------------------------------------
>
>
> _______________________________________________
> --Bandwidth and Colocation provided by Easynews.com
<http://easynews.com/>--
>
> asterisk-users mailing list
> To UNSUBSCRIBE or update options visit:
>   http://lists.digium.com/mailman/listinfo/asterisk-users
>
>
>

-- 
------------------------------------------------------------
Erick Perez
Panama Sistemas
Integradores de Telefonia IP y Soluciones Para Centros de Datos
Panama, Republica de Panama
Cel Panama. +(507) 6694-4780
------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.digium.com/pipermail/asterisk-users/attachments/20061009/4f0c4999/attachment.htm

On 9 Oct 2006, at 21:19, Douglas Garstang wrote:
> I'm just going to jump in here, and ask a stoopid question.
>
> How could you possibly write a multi-user front end in AJAX without  
> using a database backend like MySQL?
Wandering off topic here, but I'll bite....

There are a few options for backing store when a full database is  
overkill.
	0) in-memory - load it all into a hash table at start-up and query  
that (like asterisk in static mode)
	1) carefully structured filesystem, using directory trees for  
navigation, files for records etc
		(you can even use symlinks for foreign keys) - you'd be amazed how  
good the filesystem is
		as a database :-)
	3) xml files with xsl to query the data. This works really well for  
smallish files that don't change
		much. Each file change requires a re-write of the whole file, so it  
is inefficient for writes
	4) variation on 0 - one of the in-memory relational databases

I used (don't now) to take the view that anyone who didn't need  
oracle didn't need a
real database, so one of the above would be adequate....

The option discussed earlier of mysql on a ram disk is a variation of  
4 I guess.


Tim Panton

www.mexuar.com