thr3ads.net - samba - [Samba] Azure AD Connect and replication issues / WORKAROUND [Oct 2020]

If this information is useful, please help other people find it:
Share via:

Michal Bruncko

2020-Oct-22 15:40 UTC

[Samba] Azure AD Connect and replication issues

just small update:

- idfix tool (Directory Synchronization Error Remediation Tool / 
https://github.com/microsoft/idfix) shows just small issues like 
empty/missing displayName attrib in some of objects which I have 
corrected and no more issues present at all.
- no errors from AAD connect event viewer: final log message is 
"Scheduler::SchedulerThreadMain : Completed configured scheduler 
operations." (no errors before)
- aad connect info:

Azure AD Tenant - Settings

PasswordHashSync: True
ForcePasswordChangeOnLogOn: False
UserWriteback: False
DeviceWriteback: False
UnifiedGroupWriteback: False


Azure AD Connect - Global Settings

Microsoft.AADFilter.ApplicationList:
Microsoft.AADFilter.AttributeExclusionList:
Microsoft.Configuration.ImportDate: 2020-10-19 17:08:19Z
Microsoft.ConnectDirectories.WizardDirectoryMode: AD
Microsoft.DeviceWriteBack.Container:
Microsoft.DeviceWriteBack.Forest:
Microsoft.DirectoryExtension.SourceTargetAttributesMap:
Microsoft.OptionalFeature.AutoUpgradeState: Enabled
Microsoft.OptionalFeature.AutoUpgradeSuspensionReason:
Microsoft.OptionalFeature.DeviceWriteBack: False
Microsoft.OptionalFeature.DeviceWriteUp: True
Microsoft.OptionalFeature.DirectoryExtension: True
Microsoft.OptionalFeature.DirectoryExtensionAttributes:
Microsoft.OptionalFeature.ExchangeMailPublicFolder: False
Microsoft.OptionalFeature.ExportDeletionThreshold: True
Microsoft.OptionalFeature.ExportDeletionThresholdValue: 500
Microsoft.OptionalFeature.FilterAAD: True
Microsoft.OptionalFeature.GroupFiltering: False
Microsoft.OptionalFeature.GroupWriteBack: False
Microsoft.OptionalFeature.HybridExchange: False
Microsoft.OptionalFeature.UserWriteBack: False
Microsoft.SynchronizationOption.AnchorAttribute: mS-DS-ConsistencyGuid
Microsoft.SynchronizationOption.CustomAttribute:
Microsoft.SynchronizationOption.JoinCriteria: AlwaysProvision
Microsoft.SynchronizationOption.UPNAttribute: userPrincipalName
Microsoft.Synchronize.MaintenanceEnabled: True
Microsoft.Synchronize.NextStartTime: Thu, 22 Oct 2020 15:37:03 GMT
Microsoft.Synchronize.RunHistoryPurgeInterval: 7.00:00:00
Microsoft.Synchronize.ServerConfigurationVersion: 1.5.45.0
Microsoft.Synchronize.SchedulerSuspended: False
Microsoft.Synchronize.StagingMode: False
Microsoft.Synchronize.SynchronizationPolicy: Delta
Microsoft.Synchronize.SynchronizationSchedule: True
Microsoft.Synchronize.TimeInterval: 00:30:00
Microsoft.SystemInformation.MachineRole: RoleMemberServer
Microsoft.UserSignIn.DesktopSsoEnabled: True
Microsoft.UserSignIn.SignOnMethod: PassThroughAuthentication
Microsoft.UserWriteBack.Container:
Microsoft.UserWriteBack.Forest:
Microsoft.Version.SynchronizationRuleImmutableTag: V1


Azure AD Connect Sync Scheduler - Settings


AllowedSyncCycleInterval: 00:30:00
CurrentlyEffectiveSyncCycleInterval: 00:30:00
CustomizedSyncCycleInterval:
NextSyncCyclePolicyType: Delta
NextSyncCycleStartTimeInUTC: 22.10.2020 15:37:03
PurgeRunHistoryInterval: 7.00:00:00
SyncCycleEnabled: True
MaintenanceEnabled: True
StagingModeEnabled: False
SchedulerSuspended: False
SyncCycleInProgress: False



On 10/22/2020 1:19 PM, Michal Bruncko via samba wrote:> hello mj
>
> we are school - we are syncing approx 500 users and several groups.
>
> we use pass-through authentication because AAD authentication was not 
> working (that time we used samba 4.8.12, but since moving to 4.12 I 
> didnt tested this), we have (as recommended) one AAD connector plus 
> two authentication agents deployed in our environment.
>
> the thing what I dont understand is the amount of data which is 
> connector getting each day (~50GB). I have to check AAD connector 
> tshoot tool to understand whether there are some issues (like sync 
> will not get properly completed and therefore it is always retrying).
>
> but from AAD point of view all looks fine - in AAD portal I see the 
> sync is done every 30 minutes without errors.
>
> but this is definitely not correct behaviour regards to CPU and BW 
> utilization...
>
> regards
> michal
>
>
> On 10/21/2020 9:16 PM, mj via samba wrote:
>> Hi Michal,
>>
>> Seems we are doing similar things at the moment: getting samba to 
>> work with azure AD.
>>
>> We also see the high CPU usage on the DC that the Azure AD Connect 
>> server connected to. Between 70 - 100 percent in our case.
>>
>> We are not seeing any replication issues after azure AD Connect, and 
>> I have a script that automatically checks replication every few
minutes.
>>
>> I was the one reporting the highwatermark back in 2017, but we're
not
>> getting those now. Between 2017 and now, we stopped testing azure, 
>> and I took it up again only last week.
>>
>> On the samba DC side, we observe that samba is very buzy talking to 
>> the Azure AD Connect machine, even though we filter sync using one 
>> group containing only three members, and none of them are actually 
>> changing. (this is just for testing now) How many users are you
syncing?
>>
>> Are you using pass-through authentication, or syncing password 
>> hashes? And is what you chose working for you, authentication-wise?
>>
>> Regards,
>> MJ
>>
>> On 10/21/20 7:10 PM, Michal Bruncko via samba wrote:
>>> ups, seems pictures (attachments in general) are not accepted here,
>>> screen (graph) is available here: 
>>> https://i.postimg.cc/xCk6k038/image-2020-10-21-190940.png
>>>
>>> On 10/21/2020 6:00 PM, Michal Bruncko wrote:
>>>> hello
>>>>
>>>> our AD domain is hosted by two samba AD domain controllers
version
>>>> 4.12.6
>>>> - replication between controllers is fine, no problems.
>>>> - no schema errors.
>>>> - no database errors, all fine.
>>>> - no CPU utilizations
>>>> - wthout noticeable bandwidth utilization
>>>>
>>>> Recently we have deployed Azure AD connector on dedicated
windows
>>>> system (system is domain member server). since this deployment
we
>>>> are observing following issues on DCs:
>>>> - CPU utilization issue (one CPU core fully utilized)
>>>> - high BW utilization
>>>> - replication issue messages:
>>>> [2020/10/21 17:41:55.043563,? 0] 
>>>>
../../source4/rpc_server/drsuapi/getncchanges.c:2910(dcesrv_drsuapi_DsGetNCChanges)
>>>>
>>>> ? ../../source4/rpc_server/drsuapi/getncchanges.c:2910: 
>>>> DsGetNCChanges 2nd replication on DN DC= older highwatermark 
>>>> (last_dn CN=userXYZ,OU=Users,DC=)
>>>>
>>>>
>>>> and this is happening only on one DC server in time - the one,
to
>>>> which this AD connector is connected for doing AD to AAD sync
tasks.
>>>>
>>>> More details:
>>>>
>>>> CPU: mostly only one CPU core from all system-assigned cores is
>>>> utilized at 100%:
>>>>
>>>>
>>>> BW utilization: you can see example here (peak starts once the 
>>>> Azure AD connector connects to particular DC server) (notice
the
>>>> "uploaded" data - 54GB - value from DC system):
>>>>
>>>>
>>>>
>>>> Replicaton errors: repeating messages (example above) every
each
>>>> 4-5 seconds. the "last_dn" is changing during time
slowly: it is
>>>> changed to another (user) object each several hours.
>>>>
>>>> no other issues observed.
>>>>
>>>> - If we deactivate this Azure connector, all issues stopped
(but of
>>>> course we are out of sync with AAD)
>>>> - if we reboot/stop DC1 services (serving for Azure connector),
the
>>>> Azure connector switch to DC2 and same story happen again 
>>>> (CPU/bandwidth/replication logs)
>>>>
>>>> I've found similar issue reported back in 2017: 
>>>> https://lists.samba.org/archive/samba/2017-October/211756.html 
>>>> ([Samba] samba getting stuck, highwatermark replication issue?)
>>>>
>>>> seems this issue is still in place now. no difference.
>>>>
>>>>
>>>> does anyone else have similar issues? does anyone else how to 
>>>> resolve them? either on Azure AD connector side (there are
various
>>>> confiuration option available) or (possibly) on samba side?
>>>>
>>>>
>>>> thank you
>>>> michal
>>>>
>>>
>>
>
>

Michal Bruncko

2020-Oct-25 16:11 UTC

head link

[Samba] Azure AD Connect and replication issues / WORKAROUND

hello all

just to summarize this topic for anybody else.
after some time spent with tshooting there seems is a sufficient 
workaround for this issue: disable "sync of password hashes" option 
within Azure AD connect tool.
with disabling this option all three issues (CPU/Bandwidth 
utilization/replication logs) are gone!
seems that for now the only working option for samba and Azure connect 
tool is to:
- use pass-through authentication
- deploy at least one authentication agent (agent is installed a part of 
Azure AD connect tool), but two agents are recommended
- and disable "sync of password hashes" option in Azure AD connect
tool

seems that syncing password hahes is still problem for samba: 
https://bugzilla.samba.org/show_bug.cgi?id=10635
and it looks like Azure connect tool keeps trying (and endlessly 
retrying) to get them synced which is causing all three issues in my 
environment (CPU/Bandwidth utilization/replication logs).

Michal


On 10/22/2020 5:40 PM, Michal Bruncko via samba wrote:> just small update:
>
> - idfix tool (Directory Synchronization Error Remediation Tool / 
> https://github.com/microsoft/idfix) shows just small issues like 
> empty/missing displayName attrib in some of objects which I have 
> corrected and no more issues present at all.
> - no errors from AAD connect event viewer: final log message is 
> "Scheduler::SchedulerThreadMain : Completed configured scheduler 
> operations." (no errors before)
> - aad connect info:
>
> Azure AD Tenant - Settings
>
> PasswordHashSync: True
> ForcePasswordChangeOnLogOn: False
> UserWriteback: False
> DeviceWriteback: False
> UnifiedGroupWriteback: False
>
>
> Azure AD Connect - Global Settings
>
> Microsoft.AADFilter.ApplicationList:
> Microsoft.AADFilter.AttributeExclusionList:
> Microsoft.Configuration.ImportDate: 2020-10-19 17:08:19Z
> Microsoft.ConnectDirectories.WizardDirectoryMode: AD
> Microsoft.DeviceWriteBack.Container:
> Microsoft.DeviceWriteBack.Forest:
> Microsoft.DirectoryExtension.SourceTargetAttributesMap:
> Microsoft.OptionalFeature.AutoUpgradeState: Enabled
> Microsoft.OptionalFeature.AutoUpgradeSuspensionReason:
> Microsoft.OptionalFeature.DeviceWriteBack: False
> Microsoft.OptionalFeature.DeviceWriteUp: True
> Microsoft.OptionalFeature.DirectoryExtension: True
> Microsoft.OptionalFeature.DirectoryExtensionAttributes:
> Microsoft.OptionalFeature.ExchangeMailPublicFolder: False
> Microsoft.OptionalFeature.ExportDeletionThreshold: True
> Microsoft.OptionalFeature.ExportDeletionThresholdValue: 500
> Microsoft.OptionalFeature.FilterAAD: True
> Microsoft.OptionalFeature.GroupFiltering: False
> Microsoft.OptionalFeature.GroupWriteBack: False
> Microsoft.OptionalFeature.HybridExchange: False
> Microsoft.OptionalFeature.UserWriteBack: False
> Microsoft.SynchronizationOption.AnchorAttribute: mS-DS-ConsistencyGuid
> Microsoft.SynchronizationOption.CustomAttribute:
> Microsoft.SynchronizationOption.JoinCriteria: AlwaysProvision
> Microsoft.SynchronizationOption.UPNAttribute: userPrincipalName
> Microsoft.Synchronize.MaintenanceEnabled: True
> Microsoft.Synchronize.NextStartTime: Thu, 22 Oct 2020 15:37:03 GMT
> Microsoft.Synchronize.RunHistoryPurgeInterval: 7.00:00:00
> Microsoft.Synchronize.ServerConfigurationVersion: 1.5.45.0
> Microsoft.Synchronize.SchedulerSuspended: False
> Microsoft.Synchronize.StagingMode: False
> Microsoft.Synchronize.SynchronizationPolicy: Delta
> Microsoft.Synchronize.SynchronizationSchedule: True
> Microsoft.Synchronize.TimeInterval: 00:30:00
> Microsoft.SystemInformation.MachineRole: RoleMemberServer
> Microsoft.UserSignIn.DesktopSsoEnabled: True
> Microsoft.UserSignIn.SignOnMethod: PassThroughAuthentication
> Microsoft.UserWriteBack.Container:
> Microsoft.UserWriteBack.Forest:
> Microsoft.Version.SynchronizationRuleImmutableTag: V1
>
>
> Azure AD Connect Sync Scheduler - Settings
>
>
> AllowedSyncCycleInterval: 00:30:00
> CurrentlyEffectiveSyncCycleInterval: 00:30:00
> CustomizedSyncCycleInterval:
> NextSyncCyclePolicyType: Delta
> NextSyncCycleStartTimeInUTC: 22.10.2020 15:37:03
> PurgeRunHistoryInterval: 7.00:00:00
> SyncCycleEnabled: True
> MaintenanceEnabled: True
> StagingModeEnabled: False
> SchedulerSuspended: False
> SyncCycleInProgress: False
>
>
>
> On 10/22/2020 1:19 PM, Michal Bruncko via samba wrote:
>> hello mj
>>
>> we are school - we are syncing approx 500 users and several groups.
>>
>> we use pass-through authentication because AAD authentication was not 
>> working (that time we used samba 4.8.12, but since moving to 4.12 I 
>> didnt tested this), we have (as recommended) one AAD connector plus 
>> two authentication agents deployed in our environment.
>>
>> the thing what I dont understand is the amount of data which is 
>> connector getting each day (~50GB). I have to check AAD connector 
>> tshoot tool to understand whether there are some issues (like sync 
>> will not get properly completed and therefore it is always retrying).
>>
>> but from AAD point of view all looks fine - in AAD portal I see the 
>> sync is done every 30 minutes without errors.
>>
>> but this is definitely not correct behaviour regards to CPU and BW 
>> utilization...
>>
>> regards
>> michal
>>
>>
>> On 10/21/2020 9:16 PM, mj via samba wrote:
>>> Hi Michal,
>>>
>>> Seems we are doing similar things at the moment: getting samba to 
>>> work with azure AD.
>>>
>>> We also see the high CPU usage on the DC that the Azure AD Connect 
>>> server connected to. Between 70 - 100 percent in our case.
>>>
>>> We are not seeing any replication issues after azure AD Connect,
and
>>> I have a script that automatically checks replication every few 
>>> minutes.
>>>
>>> I was the one reporting the highwatermark back in 2017, but
we're
>>> not getting those now. Between 2017 and now, we stopped testing 
>>> azure, and I took it up again only last week.
>>>
>>> On the samba DC side, we observe that samba is very buzy talking to
>>> the Azure AD Connect machine, even though we filter sync using one 
>>> group containing only three members, and none of them are actually 
>>> changing. (this is just for testing now) How many users are you 
>>> syncing?
>>>
>>> Are you using pass-through authentication, or syncing password 
>>> hashes? And is what you chose working for you, authentication-wise?
>>>
>>> Regards,
>>> MJ
>>>
>>> On 10/21/20 7:10 PM, Michal Bruncko via samba wrote:
>>>> ups, seems pictures (attachments in general) are not accepted
here,
>>>> screen (graph) is available here: 
>>>> https://i.postimg.cc/xCk6k038/image-2020-10-21-190940.png
>>>>
>>>> On 10/21/2020 6:00 PM, Michal Bruncko wrote:
>>>>> hello
>>>>>
>>>>> our AD domain is hosted by two samba AD domain controllers
version
>>>>> 4.12.6
>>>>> - replication between controllers is fine, no problems.
>>>>> - no schema errors.
>>>>> - no database errors, all fine.
>>>>> - no CPU utilizations
>>>>> - wthout noticeable bandwidth utilization
>>>>>
>>>>> Recently we have deployed Azure AD connector on dedicated
windows
>>>>> system (system is domain member server). since this
deployment we
>>>>> are observing following issues on DCs:
>>>>> - CPU utilization issue (one CPU core fully utilized)
>>>>> - high BW utilization
>>>>> - replication issue messages:
>>>>> [2020/10/21 17:41:55.043563,? 0] 
>>>>>
../../source4/rpc_server/drsuapi/getncchanges.c:2910(dcesrv_drsuapi_DsGetNCChanges)
>>>>>
>>>>> ? ../../source4/rpc_server/drsuapi/getncchanges.c:2910: 
>>>>> DsGetNCChanges 2nd replication on DN DC= older
highwatermark
>>>>> (last_dn CN=userXYZ,OU=Users,DC=)
>>>>>
>>>>>
>>>>> and this is happening only on one DC server in time - the
one, to
>>>>> which this AD connector is connected for doing AD to AAD
sync tasks.
>>>>>
>>>>> More details:
>>>>>
>>>>> CPU: mostly only one CPU core from all system-assigned
cores is
>>>>> utilized at 100%:
>>>>>
>>>>>
>>>>> BW utilization: you can see example here (peak starts once
the
>>>>> Azure AD connector connects to particular DC server)
(notice the
>>>>> "uploaded" data - 54GB - value from DC system):
>>>>>
>>>>>
>>>>>
>>>>> Replicaton errors: repeating messages (example above) every
each
>>>>> 4-5 seconds. the "last_dn" is changing during
time slowly: it is
>>>>> changed to another (user) object each several hours.
>>>>>
>>>>> no other issues observed.
>>>>>
>>>>> - If we deactivate this Azure connector, all issues stopped
(but
>>>>> of course we are out of sync with AAD)
>>>>> - if we reboot/stop DC1 services (serving for Azure
connector),
>>>>> the Azure connector switch to DC2 and same story happen
again
>>>>> (CPU/bandwidth/replication logs)
>>>>>
>>>>> I've found similar issue reported back in 2017: 
>>>>>
https://lists.samba.org/archive/samba/2017-October/211756.html
>>>>> ([Samba] samba getting stuck, highwatermark replication
issue?)
>>>>>
>>>>> seems this issue is still in place now. no difference.
>>>>>
>>>>>
>>>>> does anyone else have similar issues? does anyone else how
to
>>>>> resolve them? either on Azure AD connector side (there are
various
>>>>> confiuration option available) or (possibly) on samba side?
>>>>>
>>>>>
>>>>> thank you
>>>>> michal
>>>>>
>>>>
>>>
>>
>>
>
>

2020-Oct-26 09:45 UTC

head link

[Samba] Azure AD Connect and replication issues / WORKAROUND

Hi Michal,

Thanks for updating, I can confirm your findings on our site. 
Pass-through auth works nicely here as well.

However, when reading the ticket, we understand that Ralph B?hme claims 
that password-hashes should also work, after making the sync account 
MSOL_604447e... a member of "domain admins".

(instead of keeping the default rights that are granted by the 
installer, which directory ACLs are not honored by the Samba DRSR RPC 
service)

Buzy today, but in the coming days I will spend some time trying to see 
if I can make the password hash sync work as well. (if it works for 
Ralph B?hme, we should also be able to make it work) :-)

Thanks for updating here!

MJ


On 25/10/2020 17:11, Michal Bruncko via samba wrote:> hello all
> 
> just to summarize this topic for anybody else.
> after some time spent with tshooting there seems is a sufficient 
> workaround for this issue: disable "sync of password hashes"
option
> within Azure AD connect tool.
> with disabling this option all three issues (CPU/Bandwidth 
> utilization/replication logs) are gone!
> seems that for now the only working option for samba and Azure connect 
> tool is to:
> - use pass-through authentication
> - deploy at least one authentication agent (agent is installed a part of 
> Azure AD connect tool), but two agents are recommended
> - and disable "sync of password hashes" option in Azure AD
connect tool
> 
> seems that syncing password hahes is still problem for samba: 
> https://bugzilla.samba.org/show_bug.cgi?id=10635
> and it looks like Azure connect tool keeps trying (and endlessly 
> retrying) to get them synced which is causing all three issues in my 
> environment (CPU/Bandwidth utilization/replication logs).
> 
> Michal
> 
> 
> On 10/22/2020 5:40 PM, Michal Bruncko via samba wrote:
>> just small update:
>>
>> - idfix tool (Directory Synchronization Error Remediation Tool / 
>> https://github.com/microsoft/idfix) shows just small issues like 
>> empty/missing displayName attrib in some of objects which I have 
>> corrected and no more issues present at all.
>> - no errors from AAD connect event viewer: final log message is 
>> "Scheduler::SchedulerThreadMain : Completed configured scheduler 
>> operations." (no errors before)
>> - aad connect info:
>>
>> Azure AD Tenant - Settings
>>
>> PasswordHashSync: True
>> ForcePasswordChangeOnLogOn: False
>> UserWriteback: False
>> DeviceWriteback: False
>> UnifiedGroupWriteback: False
>>
>>
>> Azure AD Connect - Global Settings
>>
>> Microsoft.AADFilter.ApplicationList:
>> Microsoft.AADFilter.AttributeExclusionList:
>> Microsoft.Configuration.ImportDate: 2020-10-19 17:08:19Z
>> Microsoft.ConnectDirectories.WizardDirectoryMode: AD
>> Microsoft.DeviceWriteBack.Container:
>> Microsoft.DeviceWriteBack.Forest:
>> Microsoft.DirectoryExtension.SourceTargetAttributesMap:
>> Microsoft.OptionalFeature.AutoUpgradeState: Enabled
>> Microsoft.OptionalFeature.AutoUpgradeSuspensionReason:
>> Microsoft.OptionalFeature.DeviceWriteBack: False
>> Microsoft.OptionalFeature.DeviceWriteUp: True
>> Microsoft.OptionalFeature.DirectoryExtension: True
>> Microsoft.OptionalFeature.DirectoryExtensionAttributes:
>> Microsoft.OptionalFeature.ExchangeMailPublicFolder: False
>> Microsoft.OptionalFeature.ExportDeletionThreshold: True
>> Microsoft.OptionalFeature.ExportDeletionThresholdValue: 500
>> Microsoft.OptionalFeature.FilterAAD: True
>> Microsoft.OptionalFeature.GroupFiltering: False
>> Microsoft.OptionalFeature.GroupWriteBack: False
>> Microsoft.OptionalFeature.HybridExchange: False
>> Microsoft.OptionalFeature.UserWriteBack: False
>> Microsoft.SynchronizationOption.AnchorAttribute: mS-DS-ConsistencyGuid
>> Microsoft.SynchronizationOption.CustomAttribute:
>> Microsoft.SynchronizationOption.JoinCriteria: AlwaysProvision
>> Microsoft.SynchronizationOption.UPNAttribute: userPrincipalName
>> Microsoft.Synchronize.MaintenanceEnabled: True
>> Microsoft.Synchronize.NextStartTime: Thu, 22 Oct 2020 15:37:03 GMT
>> Microsoft.Synchronize.RunHistoryPurgeInterval: 7.00:00:00
>> Microsoft.Synchronize.ServerConfigurationVersion: 1.5.45.0
>> Microsoft.Synchronize.SchedulerSuspended: False
>> Microsoft.Synchronize.StagingMode: False
>> Microsoft.Synchronize.SynchronizationPolicy: Delta
>> Microsoft.Synchronize.SynchronizationSchedule: True
>> Microsoft.Synchronize.TimeInterval: 00:30:00
>> Microsoft.SystemInformation.MachineRole: RoleMemberServer
>> Microsoft.UserSignIn.DesktopSsoEnabled: True
>> Microsoft.UserSignIn.SignOnMethod: PassThroughAuthentication
>> Microsoft.UserWriteBack.Container:
>> Microsoft.UserWriteBack.Forest:
>> Microsoft.Version.SynchronizationRuleImmutableTag: V1
>>
>>
>> Azure AD Connect Sync Scheduler - Settings
>>
>>
>> AllowedSyncCycleInterval: 00:30:00
>> CurrentlyEffectiveSyncCycleInterval: 00:30:00
>> CustomizedSyncCycleInterval:
>> NextSyncCyclePolicyType: Delta
>> NextSyncCycleStartTimeInUTC: 22.10.2020 15:37:03
>> PurgeRunHistoryInterval: 7.00:00:00
>> SyncCycleEnabled: True
>> MaintenanceEnabled: True
>> StagingModeEnabled: False
>> SchedulerSuspended: False
>> SyncCycleInProgress: False
>>
>>
>>
>> On 10/22/2020 1:19 PM, Michal Bruncko via samba wrote:
>>> hello mj
>>>
>>> we are school - we are syncing approx 500 users and several groups.
>>>
>>> we use pass-through authentication because AAD authentication was
not
>>> working (that time we used samba 4.8.12, but since moving to 4.12 I
>>> didnt tested this), we have (as recommended) one AAD connector plus
>>> two authentication agents deployed in our environment.
>>>
>>> the thing what I dont understand is the amount of data which is 
>>> connector getting each day (~50GB). I have to check AAD connector 
>>> tshoot tool to understand whether there are some issues (like sync 
>>> will not get properly completed and therefore it is always
retrying).
>>>
>>> but from AAD point of view all looks fine - in AAD portal I see the
>>> sync is done every 30 minutes without errors.
>>>
>>> but this is definitely not correct behaviour regards to CPU and BW 
>>> utilization...
>>>
>>> regards
>>> michal
>>>
>>>
>>> On 10/21/2020 9:16 PM, mj via samba wrote:
>>>> Hi Michal,
>>>>
>>>> Seems we are doing similar things at the moment: getting samba
to
>>>> work with azure AD.
>>>>
>>>> We also see the high CPU usage on the DC that the Azure AD
Connect
>>>> server connected to. Between 70 - 100 percent in our case.
>>>>
>>>> We are not seeing any replication issues after azure AD
Connect, and
>>>> I have a script that automatically checks replication every few
>>>> minutes.
>>>>
>>>> I was the one reporting the highwatermark back in 2017, but
we're
>>>> not getting those now. Between 2017 and now, we stopped testing
>>>> azure, and I took it up again only last week.
>>>>
>>>> On the samba DC side, we observe that samba is very buzy
talking to
>>>> the Azure AD Connect machine, even though we filter sync using
one
>>>> group containing only three members, and none of them are
actually
>>>> changing. (this is just for testing now) How many users are you
>>>> syncing?
>>>>
>>>> Are you using pass-through authentication, or syncing password 
>>>> hashes? And is what you chose working for you,
authentication-wise?
>>>>
>>>> Regards,
>>>> MJ
>>>>
>>>> On 10/21/20 7:10 PM, Michal Bruncko via samba wrote:
>>>>> ups, seems pictures (attachments in general) are not
accepted here,
>>>>> screen (graph) is available here: 
>>>>> https://i.postimg.cc/xCk6k038/image-2020-10-21-190940.png
>>>>>
>>>>> On 10/21/2020 6:00 PM, Michal Bruncko wrote:
>>>>>> hello
>>>>>>
>>>>>> our AD domain is hosted by two samba AD domain
controllers version
>>>>>> 4.12.6
>>>>>> - replication between controllers is fine, no problems.
>>>>>> - no schema errors.
>>>>>> - no database errors, all fine.
>>>>>> - no CPU utilizations
>>>>>> - wthout noticeable bandwidth utilization
>>>>>>
>>>>>> Recently we have deployed Azure AD connector on
dedicated windows
>>>>>> system (system is domain member server). since this
deployment we
>>>>>> are observing following issues on DCs:
>>>>>> - CPU utilization issue (one CPU core fully utilized)
>>>>>> - high BW utilization
>>>>>> - replication issue messages:
>>>>>> [2020/10/21 17:41:55.043563,? 0] 
>>>>>>
../../source4/rpc_server/drsuapi/getncchanges.c:2910(dcesrv_drsuapi_DsGetNCChanges)
>>>>>>
>>>>>> ? ../../source4/rpc_server/drsuapi/getncchanges.c:2910:
>>>>>> DsGetNCChanges 2nd replication on DN DC= older
highwatermark
>>>>>> (last_dn CN=userXYZ,OU=Users,DC=)
>>>>>>
>>>>>>
>>>>>> and this is happening only on one DC server in time -
the one, to
>>>>>> which this AD connector is connected for doing AD to
AAD sync tasks.
>>>>>>
>>>>>> More details:
>>>>>>
>>>>>> CPU: mostly only one CPU core from all system-assigned
cores is
>>>>>> utilized at 100%:
>>>>>>
>>>>>>
>>>>>> BW utilization: you can see example here (peak starts
once the
>>>>>> Azure AD connector connects to particular DC server)
(notice the
>>>>>> "uploaded" data - 54GB - value from DC
system):
>>>>>>
>>>>>>
>>>>>>
>>>>>> Replicaton errors: repeating messages (example above)
every each
>>>>>> 4-5 seconds. the "last_dn" is changing during
time slowly: it is
>>>>>> changed to another (user) object each several hours.
>>>>>>
>>>>>> no other issues observed.
>>>>>>
>>>>>> - If we deactivate this Azure connector, all issues
stopped (but
>>>>>> of course we are out of sync with AAD)
>>>>>> - if we reboot/stop DC1 services (serving for Azure
connector),
>>>>>> the Azure connector switch to DC2 and same story happen
again
>>>>>> (CPU/bandwidth/replication logs)
>>>>>>
>>>>>> I've found similar issue reported back in 2017: 
>>>>>>
https://lists.samba.org/archive/samba/2017-October/211756.html
>>>>>> ([Samba] samba getting stuck, highwatermark replication
issue?)
>>>>>>
>>>>>> seems this issue is still in place now. no difference.
>>>>>>
>>>>>>
>>>>>> does anyone else have similar issues? does anyone else
how to
>>>>>> resolve them? either on Azure AD connector side (there
are various
>>>>>> confiuration option available) or (possibly) on samba
side?
>>>>>>
>>>>>>
>>>>>> thank you
>>>>>> michal
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
> 
>

Possibly Parallel Threads

Search for more reasonably related threads

samba - Oct 2020 - Azure AD Connect and replication issues / WORKAROUND

[Samba] Azure AD Connect and replication issues

[Samba] Azure AD Connect and replication issues / WORKAROUND

[Samba] Azure AD Connect and replication issues / WORKAROUND

Possibly Parallel Threads