|3.10 and Later||How-to||
THIS PROCEDURE IS STRICTLY INTENDED TO BE EXECUTED BY A DOMAIN ADMINISTRATOR WHO WORKS FOR THE CUSTOMER, AND NOT BY A NOVICE; THIS IS A PROCEDURE FOR AD EXPERTS. SCALEARC CAN ONLY PROVIDE GUIDANCE; THE RESPONSIBILITY FOR THE OUTCOME MUST REST WITH THE CUSTOMER'S EXPERT DOMAIN ADMINISTRATION STAFF.
DOUBLE-CHECK YOUR WORK; A MISTAKE HERE WILL CAUSE SERIOUS PROBLEMS BOTH ON SCALEARC AND IN YOUR AD DOMAIN. CONFIRMATION STEPS ARE NOT OPTIONAL.
Once Scalearc is joined as an RODC to an Active Directory domain then you may see messages with Event ID 1645 or other permission errors being logged in the Directory Service Event logs on your AD Domain Controllers for that AD domain.
The text of these events will be similar to the following:
Active Directory Domain Services did not perform an authenticated remote procedure call (RPC) to another directory server because the desired service principal name (SPN) for the destination directory server is not registered on the Key Distribution Center (KDC) domain controller that resolves the SPN.
Destination directory server:
Verify that the names of the destination directory server and domain are correct. Also, verify that the SPN is registered on the KDC domain controller. If the destination directory server has been recently promoted, it will be necessary for the local directory server’s account data to replicate to the KDC before this directory server can be authenticated.
In addition to this error, which contains information vital to properly address this problem, 2883 and 2896 errors have also been observed, and are corrected by the solution given below.
These errors are caused by a problem in Windows Active Directory RODC registration which causes a conflict on the DC in writing the SPN for the system. This conflict causes the SPN to be immediately deleted after it is created on the domain. This procedure is based upon Microsoft's article on this problem, with elaborations allowing the correct SPN for ScaleArc to be determined, checked, and if necessary registered. The Microsoft Technet document is available here.
0. If you have joined the domain from the HA Primary of an HA pair, then please see #4 below. The instructions there will allow you to decide which SPN needs to be registered for which ScaleArc server. The need for this will become apparent as you follow this procedure. You will need to complete the procedure for both systems in the HA pair.
1. First check whether you can resolve the RODC name on the ScaleArc server. You will need to login to ScaleArc on a console or SSH session to give this command.
$ host -t A SC10.phil.edu
This command is run from the ScaleArc SSH or console command line; replace SC10.phil.edu with your ScaleArc server's FQDN.
You will get a message like this with your ScaleArc server's IP address:
SC10.phil.edu has address 192.168.11.58
or else you will get an error. If you get an error, you will either need to add the FQDN for your ScaleArc server into your AD Domain Controller's DNS configuration, or configure ScaleArc to use the AD Domain's DNS server. You will need to do this before you proceed. Instructions for configuring AD DNS using the Windows AD DNS Manager are included in Microsoft AD documentation. See the ScaleArc Administrator's Guide for instructions on configuring DNS on ScaleArc.
Confirmation: Do the host command again. It should return your ScaleArc server's IP address.
2. Next try to ping the Destination directory server FQDN mentioned in the 1645 event; this will probably fail. Copy and paste the exact name as it is logged in the 1645 event.
C:\> ping c90d198b-0019-4651-95fc-668443cb22a5._msdcs.phil.edu
This must be done using the "ping" command on your AD DC. It will not work on the ScaleArc server since it cannot resolve the _msdcs.<your domain> zone. Do not attempt to enable resolution in the _msdcs.<your domain> zone on the ScaleArc server; this is Microsoft-specific DNS configuration and is not supported in Linux Bind (which is the DNS resolver on ScaleArc). Specifically, the _ character is illegal in the Internet RFCs and STDs for DNS, to which Bind conforms.
Carefully note the hostname part (before the _msdcs zone name); this is your ScaleArc's GUID.
If the command fails, this indicates a missing CNAME entry for the Scalearc RODC in the DNS of the AD domain. To fix this, you need to login to Scalearc on a console or SSH session, acquire root privileges using the sudo su - command, and on the command line give following command:
$ sudo samba-tool dns add <dc> _msdcs.phil.edu c90d198b-0019-4651-95fc-668443cb22a5 CNAME SC10.phil.edu -Uadministrator
Replace <dc> with the DNS name of your AD Primary DC (which is the first one that joined the domain). Substitute the phil.edu part of the _msdcs subdomain name with your domain's name. Substitute the GUID from the FQDN you pinged above (without the domain name!) from the 1645 message for the GUID c90d198b-0019-4641-95fc-668443cb22a5. Substitute the true FQDN of your ScaleArc for SC10.phil.edu. You must use the same administrator name you used when you joined the domain on ScaleArc. You may be asked for the password for the AD administrator, but in our tests it was not needed since the ScaleArc Samba RODC installation had already joined the domain. This command will create the CNAME entry for your ScaleArc server's GUID. If this command does not work, you will need to manually add this entry on your DC's DNS to the _msdcs.<your domain> zone.
If you have multiple DNS servers (and by default all DCs are DNS servers), you will need to reload the zone and refresh, using DNS Manager, on each subsidiary DC node; otherwise, you should be prepared to wait for the zones to expire and be automatically reloaded. Reload timing is configured in the Properties of the _msdcs.<your domain> zone in the DNS Manager, but ScaleArc recommends that you do zone reloads for the _msdcs.<your domain> zone on all DCs in your domain due to the operational constraints of DNS propagation in Microsoft AD, and for immediate propagation to ensure you can test successfully.
Confirmation: Ping again as above on your AD DC to confirm connectivity, remembering that the ping must be from a DC that has had the zone reloaded on it. You should also check for the DNS entry in the Forward Lookup Zones for your domain's DNS server, in the _msdcs.<your domain> zone.
Note: This operation can also be completed using the DNS Manager on the DNS server in Windows. For instructions on this see the Microsoft DNS Manager documentation. In this case, make sure you add the entry in the _msdcs.<your domain> zone, as a CNAME for the FQDN of your ScaleArc server.
3. Now that we can look up the Directory Server CNAME record for the ScaleArc system in the _msdcs.<your domain> zone, we need to set the Service Principal Name (SPN) for your ScaleArc system. This is done on your AD DC's command line.
C:\> setspn -a E3514235-4B06-11D1-AB04-00C04FC2DCD2/c90d198b-0019-4651-95fc-668443cb22a5/phil.edu SC10
Use the SPN from the 1645 message, leaving off the @ sign and domain name following it; the last parameter is the hostname (without the domain name!) of your ScaleArc server. Replace the GUID (here c90d198b-0019-4651-95fc-668443cb22a5) with the GUID for your ScaleArc, the domain name (here phil.edu) with your domain name, and the hostname (here SC10) with your ScaleArc's hostname.
The E3514235-4B06-11D1-AB04-00C04FC2DCD2 portion of the SPN is required by AD for the SPNs of all RODCs, and is the missing entry that was updated but deleted by the conflicting updates on the DC immediately after RODC registration.
Confirmation: A message that the SPN is updated will be returned.
4. If you have done a domain join on the HA Primary of an HA pair, then you will have to execute this procedure for each ScaleArc server, and you may have trouble deciding which system has which GUID, and therefore which SPN is associated with which ScaleArc server. This can be resolved by checking on the DC in the Users and Computers Manager. Select Domain Controllers in the left pane, then check the Properties for each ScaleArc server's hostname; on the General tab, choose the NTDS Settings button, and on the NTDS Settings popup, in the General tab, you will find the DNS Alias is <ScaleArc server GUID>._msdcs.<your domain>. This will allow you to determine the correct SPN for each ScaleArc server from the SPNs in the 1645 events. Note that this information is also available in the Sites and Services Manager. Expand the hostnames of the ScaleArcs and choose Properties for the NTDS Settings under the ScaleArc servers. WARNING: the correct association of the GUID and SPN is essential for proper operation of both ScaleArc and your AD domain. Triple check that you have this correct before making the CNAME entries and registering the SPNs.
You will also note that the ping command on the DC of the Destination directory server prints the FQDN of the ScaleArc server associated with that GUID as part of the resolution of the CNAME record, for confirmation. This is a good check for correct configuration.
The 1645 messages and other permission errors should stop being posted.
if you are experiencing issues with ScaleArc or with any of it's features, please contact ScaleArc Support.