search for: arou

...The recorder writes the streams in 10s blocks, so when it is for example recording 50 streams it is creating 5 files a second, each about 5M. it uses a write-then-rename process. I simulated that with a small script, that wrote 5M files and renamed them as fast as it could, and could easily create around 100 files/s (which abouts saturates the network). So I think the cluster is up to the task. However if we try the actualy workload we run in to trouble. Running the recorder software we can gradually ramp up the number of streams it records (and thus the number of files it creates), and at arou...

...streams in 10s blocks, so > when it is for example recording 50 streams it is creating 5 files a > second, each about 5M. it uses a write-then-rename process. > > I simulated that with a small script, that wrote 5M files and renamed them > as fast as it could, and could easily create around 100 files/s (which > abouts saturates the network). So I think the cluster is up to the task. > > However if we try the actualy workload we run in to trouble. Running the > recorder software we can gradually ramp up the number of streams it records > (and thus the number of files i...

...s, so >> when it is for example recording 50 streams it is creating 5 files a >> second, each about 5M. it uses a write-then-rename process. >> >> I simulated that with a small script, that wrote 5M files and renamed >> them as fast as it could, and could easily create around 100 files/s (which >> abouts saturates the network). So I think the cluster is up to the task. >> >> However if we try the actualy workload we run in to trouble. Running the >> recorder software we can gradually ramp up the number of streams it records >> (and thus t...