<div>Hi Alejandro,</div><div>Here is mine:</div><div><p><limits><br /> <workers>4</workers><br /> <clients>100000</clients><br /> <sources>2000</sources><br /> <queue-size>102400</queue-size><br /> <client-timeout>30</client-timeout><br /> <header-timeout>15</header-timeout><br /> <source-timeout>10</source-timeout><br /> <burst-on-connect>1</burst-on-connect><br /> <burst-size>65536</burst-size><br /></limits></p></div><div>Your queue-size is larger, i will try to increase that.</div><div> </div><div>thanks!</div><div>--zahar</div><div> </div><div>10.06.2016, 14:31, "Alejandro" <cdgraff@gmail.com>:</div><blockquote type="cite"><div><div>Please share your config, almost the LIMITS part, this is my setup</div><div> </div><div><div style="font-size:12.8px;"> <limits></div><div style="font-size:12.8px;"> <workers>8</workers></div><div style="font-size:12.8px;"> <clients>100000</clients></div><div style="font-size:12.8px;"> <sources>700</sources></div><div style="font-size:12.8px;"> <queue-size>524288</queue-size></div><div style="font-size:12.8px;"> <client-timeout>30</client-timeout></div><div style="font-size:12.8px;"> <header-timeout>15</header-timeout></div><div style="font-size:12.8px;"> <source-timeout>10</source-timeout></div><div style="font-size:12.8px;"> <span style="font-size:12.8px;"><burst-size>65535</burst-size></span></div><div style="font-size:12.8px;"> </limits></div></div></div><div><br /><div><span>2016-06-10 2</span>:28 GMT-03:00 Popov, Zahar <span><<a href="mailto:zahar.popov1978@yandex.com" target="_blank">zahar.popov1978@yandex.com</a>></span>:<br /><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>Hi Alejandro,<div>Many thanks for your message.</div><div> </div><div>I changed it to 4 (i have 4 cores), but it didn’t really help. I see that all 4 cores are now working, but the connections are still being dropped.</div><div> </div><div>Which VM type are you using? Or it’s not running on AWS?</div><div> </div><div>thanks!</div><div>—zahar</div><div><div><div><br /><div><blockquote type="cite"><div>On Jun 10, 2016, at 1:29 PM, Alejandro <<a href="mailto:cdgraff@gmail.com" target="_blank">cdgraff@gmail.com</a>> wrote:</div><br /><div><div>Sorry, 35K concurrent with 8 workers at 60% cpu</div><div><br /><div><span>2016-06-10 1</span>:28 GMT-03:00 Alejandro <span><<a href="mailto:cdgraff@gmail.com" target="_blank">cdgraff@gmail.com</a>></span>:<br /><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex;"><div>Hi Zahar, what value has into <div> </div><div><div style="font-size:12.8px;"> <workers>8</workers></div></div><div> </div><div>This value is recommended to set at 1 by virtual core.</div><div> </div><div>I 'm using for some years, KH branch with 35 concurrent listeners into 8 core vm.</div><div> </div><div>Regards, </div><div>Alejandro</div></div><div><br /><div><div><div><span>2016-06-10 0</span>:50 GMT-03:00 Zahar Popov <span><<a href="mailto:zahar.popov1978@yandex.com" target="_blank">zahar.popov1978@yandex.com</a>></span>:</div></div><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex;"><div><div><div>Hello</div><div>I'm trying to measure the performance of the icecast relay server on 64kbps streams.</div><div> </div><div>The server is running in AWS (i've tried various instance types) and the test clients are running on other machines in AWS. The test client is a very simple libuv application that sends a GET request and basically ignores everything it receives in the response. I'm using the icecast-kh fork.</div><div> </div><div>I'm able to go up to around 9K simultaneous connections to the server (from two machines). The CPU usage is low, about 15% or so (on one core). However connections are starting to be dropped. Checking netstat i see many frames being lost. Increasing the transmit queue length helped, but still i can't go beyond around 9K connections. I have increased the file descriptor limits and configured IRQ balancing (even though the problem doesn't seem to be CPU bound)</div><div> </div><div>It doesn't matter if i run one or more instances of the relay server, the limit seems to be OS global so when one instance is running with 5K connections and the other instance is getting close to 4K connections they both start dropping connections.</div><div> </div><div>I assume that there is some other setting of the stack that i didn't configure so i was wondering if anybody was able to run a few dozens of thousands of connections on one server. </div><div> </div><div>thanks!</div><div>--zahar</div></div></div>_______________________________________________<br /> Icecast-dev mailing list<br /> <a href="mailto:Icecast-dev@xiph.org" target="_blank">Icecast-dev@xiph.org</a><br /> <a href="http://lists.xiph.org/mailman/listinfo/icecast-dev" target="_blank">http://lists.xiph.org/mailman/listinfo/icecast-dev</a><br /> </blockquote></div></div></blockquote></div></div></div></blockquote></div></div></div></div></div></blockquote></div></div></blockquote>
Zahar, how are you testing? with some CURL stress test? BTW, IPTABLES is enabled? I was running most time in VMWARE, but i run 10k users in medium size box in AWS, just move out for the high transfer cost. 2016-06-10 2:36 GMT-03:00 Zahar Popov <zahar.popov1978 at yandex.com>:> Hi Alejandro, > Here is mine: > > <limits> > <workers>4</workers> > <clients>100000</clients> > <sources>2000</sources> > <queue-size>102400</queue-size> > <client-timeout>30</client-timeout> > <header-timeout>15</header-timeout> > <source-timeout>10</source-timeout> > <burst-on-connect>1</burst-on-connect> > <burst-size>65536</burst-size> > </limits> > Your queue-size is larger, i will try to increase that. > > thanks! > --zahar > > 10.06.2016, 14:31, "Alejandro" <cdgraff at gmail.com>: > > Please share your config, almost the LIMITS part, this is my setup > > <limits> > <workers>8</workers> > <clients>100000</clients> > <sources>700</sources> > <queue-size>524288</queue-size> > <client-timeout>30</client-timeout> > <header-timeout>15</header-timeout> > <source-timeout>10</source-timeout> > <burst-size>65535</burst-size> > </limits> > > 2016-06-10 2:28 GMT-03:00 Popov, Zahar <zahar.popov1978 at yandex.com>: > > Hi Alejandro, > Many thanks for your message. > > I changed it to 4 (i have 4 cores), but it didn’t really help. I see that > all 4 cores are now working, but the connections are still being dropped. > > Which VM type are you using? Or it’s not running on AWS? > > thanks! > —zahar > > On Jun 10, 2016, at 1:29 PM, Alejandro <cdgraff at gmail.com> wrote: > > Sorry, 35K concurrent with 8 workers at 60% cpu > > 2016-06-10 1:28 GMT-03:00 Alejandro <cdgraff at gmail.com>: > > Hi Zahar, what value has into > > <workers>8</workers> > > This value is recommended to set at 1 by virtual core. > > I 'm using for some years, KH branch with 35 concurrent listeners into 8 > core vm. > > Regards, > Alejandro > > 2016-06-10 0:50 GMT-03:00 Zahar Popov <zahar.popov1978 at yandex.com>: > > Hello > I'm trying to measure the performance of the icecast relay server on > 64kbps streams. > > The server is running in AWS (i've tried various instance types) and the > test clients are running on other machines in AWS. The test client is a > very simple libuv application that sends a GET request and basically > ignores everything it receives in the response. I'm using the icecast-kh > fork. > > I'm able to go up to around 9K simultaneous connections to the server > (from two machines). The CPU usage is low, about 15% or so (on one core). > However connections are starting to be dropped. Checking netstat i see many > frames being lost. Increasing the transmit queue length helped, but still i > can't go beyond around 9K connections. I have increased the file descriptor > limits and configured IRQ balancing (even though the problem doesn't seem > to be CPU bound) > > It doesn't matter if i run one or more instances of the relay server, the > limit seems to be OS global so when one instance is running with 5K > connections and the other instance is getting close to 4K connections they > both start dropping connections. > > I assume that there is some other setting of the stack that i didn't > configure so i was wondering if anybody was able to run a few dozens of > thousands of connections on one server. > > thanks! > --zahar > _______________________________________________ > Icecast-dev mailing list > Icecast-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/icecast-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/icecast-dev/attachments/20160610/25858dc8/attachment.html>
I wrote a test application which is based on libuv. iptables is disabled. I’m running the test application from two other machines. Do you have any suggestions for testing? thanks! —zahar> On Jun 10, 2016, at 2:38 PM, Alejandro <cdgraff at gmail.com> wrote: > > Zahar, how are you testing? with some CURL stress test? BTW, IPTABLES is enabled? > > I was running most time in VMWARE, but i run 10k users in medium size box in AWS, just move out for the high transfer cost. > > 2016-06-10 2:36 GMT-03:00 Zahar Popov <zahar.popov1978 at yandex.com <mailto:zahar.popov1978 at yandex.com>>: > Hi Alejandro, > Here is mine: > <limits> > <workers>4</workers> > <clients>100000</clients> > <sources>2000</sources> > <queue-size>102400</queue-size> > <client-timeout>30</client-timeout> > <header-timeout>15</header-timeout> > <source-timeout>10</source-timeout> > <burst-on-connect>1</burst-on-connect> > <burst-size>65536</burst-size> > </limits> > > Your queue-size is larger, i will try to increase that. > > thanks! > --zahar > > 10.06.2016, 14:31, "Alejandro" <cdgraff at gmail.com <mailto:cdgraff at gmail.com>>: >> Please share your config, almost the LIMITS part, this is my setup >> >> <limits> >> <workers>8</workers> >> <clients>100000</clients> >> <sources>700</sources> >> <queue-size>524288</queue-size> >> <client-timeout>30</client-timeout> >> <header-timeout>15</header-timeout> >> <source-timeout>10</source-timeout> >> <burst-size>65535</burst-size> >> </limits> >> >> 2016-06-10 2:28 GMT-03:00 Popov, Zahar <zahar.popov1978 at yandex.com <mailto:zahar.popov1978 at yandex.com>>: >> Hi Alejandro, >> Many thanks for your message. >> >> I changed it to 4 (i have 4 cores), but it didn’t really help. I see that all 4 cores are now working, but the connections are still being dropped. >> >> Which VM type are you using? Or it’s not running on AWS? >> >> thanks! >> —zahar >> >>> On Jun 10, 2016, at 1:29 PM, Alejandro <cdgraff at gmail.com <mailto:cdgraff at gmail.com>> wrote: >>> >>> Sorry, 35K concurrent with 8 workers at 60% cpu >>> >>> 2016-06-10 1:28 GMT-03:00 Alejandro <cdgraff at gmail.com <mailto:cdgraff at gmail.com>>: >>> Hi Zahar, what value has into >>> >>> <workers>8</workers> >>> >>> This value is recommended to set at 1 by virtual core. >>> >>> I 'm using for some years, KH branch with 35 concurrent listeners into 8 core vm. >>> >>> Regards, >>> Alejandro >>> >>> 2016-06-10 0:50 GMT-03:00 Zahar Popov <zahar.popov1978 at yandex.com <mailto:zahar.popov1978 at yandex.com>>: >>> Hello >>> I'm trying to measure the performance of the icecast relay server on 64kbps streams. >>> >>> The server is running in AWS (i've tried various instance types) and the test clients are running on other machines in AWS. The test client is a very simple libuv application that sends a GET request and basically ignores everything it receives in the response. I'm using the icecast-kh fork. >>> >>> I'm able to go up to around 9K simultaneous connections to the server (from two machines). The CPU usage is low, about 15% or so (on one core). However connections are starting to be dropped. Checking netstat i see many frames being lost. Increasing the transmit queue length helped, but still i can't go beyond around 9K connections. I have increased the file descriptor limits and configured IRQ balancing (even though the problem doesn't seem to be CPU bound) >>> >>> It doesn't matter if i run one or more instances of the relay server, the limit seems to be OS global so when one instance is running with 5K connections and the other instance is getting close to 4K connections they both start dropping connections. >>> >>> I assume that there is some other setting of the stack that i didn't configure so i was wondering if anybody was able to run a few dozens of thousands of connections on one server. >>> >>> thanks! >>> --zahar >>> _______________________________________________ >>> Icecast-dev mailing list >>> Icecast-dev at xiph.org <mailto:Icecast-dev at xiph.org> >>> http://lists.xiph.org/mailman/listinfo/icecast-dev <http://lists.xiph.org/mailman/listinfo/icecast-dev> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/icecast-dev/attachments/20160610/e518112c/attachment-0001.html>