Hi Andrew, thanks for the reply, On Wed, Oct 25, 2017 at 11:56:22PM +1300, Andrew Bartlett wrote:> On Wed, 2017-10-25 at 09:58 +0200, Ervin Hegedüs via samba wrote: > > > > here is a Samba4 AD with LDB. > > > > Is there any limit for number of possible objects? I mean users, computers, > > and other kind of nodes. > >[...]> > May be an ugly question, but is that means the Samba4 supports "only" about > > 300k objects per db? > > There has been and will continue to be work to make Samba scale much > further, both though incremental efforts (such as the recently > completed GUID index work), bug fixes (the locking fixes in 4.7 also > fixed a major bottleneck) and more radical steps. > > As an example, with the git master current code I recently showed > adding 100,000 user objects to Samba, and adding those to either 1, 2, > 3 or 4 groups in 2 hours. The bulk of that time was in the group > handling, interestingly the user insertion is no longer the dominant > factor. > > Can you please give me some more details about what you propose to use > Samba for, so I can understand your concerns further?We have to build a Samba AD, with several DC's. The designed customer number is about 500 000, or more (user). The main function of the AD is only the authentication, so there is a captive portal and some firewall system, user connects to the network (after auth, with 802.1x), and based on its group memberships it can use the internet connection. The spped of each methods (add new user, modify its data, eg. password, group membership) is no matter. Only the read capability is important. But I don't know after the two articles (descibed in my first e-mail) that should we store in all users in one domain, or we have to plan that split them to more domains. And if it needs (to split them), how many objects is the maximum in an LDB? Thanks again, a.
On Wed, 2017-10-25 at 13:40 +0200, Ervin Hegedüs via samba wrote:> Hi Andrew, > > thanks for the reply, > > On Wed, Oct 25, 2017 at 11:56:22PM +1300, Andrew Bartlett wrote: > > On Wed, 2017-10-25 at 09:58 +0200, Ervin Hegedüs via samba wrote: > > > > > > here is a Samba4 AD with LDB. > > > > > > Is there any limit for number of possible objects? I mean users, computers, > > > and other kind of nodes. > > > > > [...] > > > > May be an ugly question, but is that means the Samba4 supports "only" about > > > 300k objects per db? > > > > There has been and will continue to be work to make Samba scale much > > further, both though incremental efforts (such as the recently > > completed GUID index work), bug fixes (the locking fixes in 4.7 also > > fixed a major bottleneck) and more radical steps. > > > > As an example, with the git master current code I recently showed > > adding 100,000 user objects to Samba, and adding those to either 1, 2, > > 3 or 4 groups in 2 hours. The bulk of that time was in the group > > handling, interestingly the user insertion is no longer the dominant > > factor. > > > > Can you please give me some more details about what you propose to use > > Samba for, so I can understand your concerns further? > > We have to build a Samba AD, with several DC's. > > The designed customer number is about 500 000, or more (user). > > The main function of the AD is only the authentication, so there > is a captive portal and some firewall system, user connects to > the network (after auth, with 802.1x), and based on its group > memberships it can use the internet connection. > > The spped of each methods (add new user, modify its data, eg. > password, group membership) is no matter. Only the read > capability is important. > > But I don't know after the two articles (descibed in my first > e-mail) that should we store in all users in one domain, or we > have to plan that split them to more domains. > > And if it needs (to split them), how many objects is the maximum > in an LDB?So, if I were you and the business application can be easily changed to shard the users into multiple domains I would do that. Samba is improving but 500,000 users is much above what anybody is know to have deployed Samba with. Naturally, you should do your own testing with Samba master and see how far you can push it, as each use case is a little different. My primary concern is that: - loading an index of objectclass=user would get increasingly slow - full scan of the database (still done occasionally, particularly for replication) would be an issue. - it is likely that joining a new DC and replication would be quite painful Finally, depending on what is in the record it simply might not fit, as there is a 4GB limit on tdb right now. For the size 64-bit aspect, I hope to see this situation change soon, perhaps for Samba 4.9. This isn't a promise yet, as it all depends on funding coming though, but I want to mention this so you know there is the potential that we will have a 64bit ldb re-based onto LDMB, removing the 32 bit limitation. Andrew Bartlett -- Andrew Bartlett http://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba
Hi Andrew, many thanks for your reply again, On Sat, Oct 28, 2017 at 10:18:33PM +1300, Andrew Bartlett wrote:> On Wed, 2017-10-25 at 13:40 +0200, Ervin Hegedüs via samba wrote: > > > > We have to build a Samba AD, with several DC's. > > > > The designed customer number is about 500 000, or more (user).[...]> > But I don't know after the two articles (descibed in my first > > e-mail) that should we store in all users in one domain, or we > > have to plan that split them to more domains. > > > > And if it needs (to split them), how many objects is the maximum > > in an LDB? > > So, if I were you and the business application can be easily changed to > shard the users into multiple domains I would do that. Samba is > improving but 500,000 users is much above what anybody is know to have > deployed Samba with. > > Naturally, you should do your own testing with Samba master and see how > far you can push it, as each use case is a little different. > > My primary concern is that: > - loading an index of objectclass=user would get increasingly slow > - full scan of the database (still done occasionally, particularly for > replication) would be an issue. > - it is likely that joining a new DC and replication would be quite > painful > > Finally, depending on what is in the record it simply might not fit, as > there is a 4GB limit on tdb right now. > > For the size 64-bit aspect, I hope to see this situation change soon, > perhaps for Samba 4.9. This isn't a promise yet, as it all depends on > funding coming though, but I want to mention this so you know there is > the potential that we will have a 64bit ldb re-based onto LDMB, > removing the 32 bit limitation.now I think everything is clear. Looks like we have to split the users to multiple domain to store them. Then probably the new DC join will be less painful. Probably that will be generates more administrative tasks - well, perhaps we can live with that together :). Thanks again for all help, a.