Dear Rers, I am trying to run a for-loop in R. During each iteration I read in an mp3 file and do some basic processing. If I do what I need to do for each file one by one - it works fine. But once I start running a loop, it soon runs out of memory and says: can't allocate a vector of size... In each iteration of my loop I always overwrite the previously created object and do gc(). Any hints on how to fight this? Thanks a lot! [[alternative HTML version deleted]]
You don't say what processing you are doing.. the answer to your question is
very likely there. To communicate effectively on this mailing list,
self-contained examples are needed. And in order to not corrupt the example you
will need to post in plain text.
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<jdnewmil at dcn.davis.ca.us> Basics: ##.#. ##.#. Live
Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------
Sent from my phone. Please excuse my brevity.
On October 20, 2014 8:30:21 AM PDT, Dimitri Liakhovitski
<dimitri.liakhovitski at gmail.com> wrote:>Dear Rers,
>
>I am trying to run a for-loop in R.
>During each iteration I read in an mp3 file and do some basic
>processing.
>If I do what I need to do for each file one by one - it works fine.
>But once I start running a loop, it soon runs out of memory and says:
>can't
>allocate a vector of size...
>In each iteration of my loop I always overwrite the previously created
>object and do gc().
>
>Any hints on how to fight this?
>
>Thanks a lot!
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>R-help at r-project.org mailing list
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
On Mon, Oct 20, 2014 at 10:30 AM, Dimitri Liakhovitski < dimitri.liakhovitski at gmail.com> wrote:> Dear Rers, > > I am trying to run a for-loop in R. > During each iteration I read in an mp3 file and do some basic processing. > If I do what I need to do for each file one by one - it works fine. > But once I start running a loop, it soon runs out of memory and says: can't > allocate a vector of size... > In each iteration of my loop I always overwrite the previously created > object and do gc(). > > Any hints on how to fight this? > > Thanks a lot! > > >?Please don't use HTML for messages. What occurs to me, from reading the other replies, is that perhaps within the loop you are causing other objects to be allocated. And that can be done just by doing a simple assignment, so it may not be obvious. What this can do is cause what we called a "sand bar" in the old days. That's where you allocate a big chunk of memory for an object. Say this take up 1/2 of your available space. You now create a small object. This object is _probably_ right next to the large object. You now release the large object. Your apparent free space is now almost what it was at the beginning. But when you try to allocate another large object which is, say, 2/3 of the maximum space, you can't because that small object is sitting right in the middle of our memory space. So you _can_ allocate 2 large objects which are 1/3 your free space size, but not 1 object which is 2/3 of the free space size. Which can lead to your type of situation. This is just a SWAG based on some experience in other systems. Most "garbage collection" do _not_ do memory consolidation. I don't know about R.? -- The temperature of the aqueous content of an unremittingly ogled culinary vessel will not achieve 100 degrees on the Celsius scale. Maranatha! <>< John McKown [[alternative HTML version deleted]]