Local Host Cache sizing and scaling
NOTE:
This article provides size and scale recommendations for XenApp and XenDesktop deployments using Local Host Cache (LHC). For size and scale recommendations for Citrix DaaS, see Size and scale considerations for Cloud Connectors.
LHC provides high availability by allowing connection brokering to continue during an outage. Users of LHC should observe the following design considerations:
- During an outage, a single broker per zone will handle Virtual Delivery Agent (VDA) registrations and broker sessions.
- An election process, which decides which broker will be active, does not take broker resources into account.
- If any single broker in a zone cannot handle all logons during normal operation, then it will not work well in outage mode.
- No site management is available during an outage.
- A highly available SQL Server is still the recommended design.
- For intermittent database connectivity scenarios, it is still better to isolate the SQL Server and leave the site in outage mode until all underlying issues are fixed.
- There is a limit of 10,000 VDAs per zone.
- Pooled desktops are not supported in outage mode, in the default configuration.
Architecture
LHC uses a local SQL Server database that is more efficient than connection leasing in disk space usage, but requires considerably more CPU and memory. LHC has phases of synchronization in which details from the main site database are synced to the brokers (Controllers). LHC uses a SQL database that still requires IOPS, and has the advantage of SQL optimizing these writes.
LHC employs a single broker that is elected to broker all connections and handle VDA registrations. All VDAs in the site will re-register with this single broker, which will then experience higher demand for resources (compared to a multi-broker site in normal operation), especially in sites with large numbers of VDAs.
LHC uses Microsoft LocalDB, which appears in Task Manager as the sqlserver.exe process. It has been configured to use up to 1 GB of memory for database buffer pool caching. However, the process will grow beyond this as the SQL engine needs memory for itself and other smaller caches. In general, the longer the outage and the more resources accessed during outage mode, the more LocalDB memory use will increase. However, when the site database connectivity is restored, sqlserver.exe will hold on to this memory and not immediately return it to the main pool.
Effect of CPU sockets and cores during outage mode
LHC uses a runtime version of SQL Server called LocalDB which has specific licensing that limits it to the lesser of four cores or a single socket. This can have a significant performance effect when the physical or virtual machine has been configured with multiple sockets with only a single or dual core. A broker machine with four sockets and one core per socket will limit LocalDB to using a single core, whereas the same VM configured as a 1-socket, 4-core machine means that LocalDB can access all four cores (albeit sharing them with other processes). During outage mode, LocalDB will be running the same broker and SQL code as during normal operation. Many of the SQL queries can be CPU-intensive and have a direct impact on the performance of brokering during outage mode. For details, see Size and scale considerations for cloud connectors and also Compute capacity limits by edition of SQL Server.
Other factors include the site configuration itself, such as the following:
- The number of published applications
- The number of users being brokered
- The rate at which users are attempting to launch sessions
- Active Directory performance
As the total broker CPU utilization approaches 100%, brokering response time will increase, logons will take longer to process and some logon attempts may fail.
Sites with multiple brokers
During site outage mode, only a single broker processes registration and logon requests. In a multi-broker site an election process takes place to nominate the broker that will be active during the outage. However, this election process does not consider the physical resources available to the brokers. This means that in a site where brokers have different amounts of resources, the elected broker will not necessarily be the most powerful in terms of CPU or RAM, which could potentially lead to poor performance during outage mode. It is important that each broker meets the additional requirements of LHC, in case it is elected.
Synchronization with the site database
The CitrixConfigSync Service handles the importing of data from the site database into a local copy on the brokers. It monitors the site database for changes to the site configuration, and triggers a new import when changes occur. A copy of the current local database is made before the import starts. The larger the number of resources (such as VDAs) in a site, the longer the import will take, but it should be less than ten minutes for a site with 10,000 VDAs.
Database location
The local database is stored in:
C:\Windows\ServiceProfiles\NetworkService\HaDatabaseName.mdf
To ensure reliability, the CitrixConfigSync Service makes a backup of the previously successful synchronized database import, before starting a new site database sync. If for any reason the sync does not successfully complete, the backup is used until a successful sync completes. You should not copy the database manually.
Local Host Cache Technical Specifications
Architecture | Spec |
---|---|
Disk Space | Depends on site configuration. For 1,000 RDS hosts + 9,500 VDI with 65 K users, 75 MB is used. |
RAM | 3 GB, ~1GB for SQL Server, 2 GB for High Availability Service and CitrixConfigSync Service. |
Time to sync configuration | 10,000 VDAs: ~ 7 minutes |
Time to activate during outage | Depends on number of VDAs and last registration sync with the broker. Only a single broker will be available for VDA registration during outage mode, so for large numbers of VDAs it can be many minutes before all VDAs are registered. |
Time to restore normal operations | As above, VDAs will need to deregister from secondary broker and re-register with the primary broker. |
Number of VDAs supported | 10,000. A site can have more than this, but the time needed to sync the site database will grow with the number of VDAs. Performance of a single broker with a large number of VDAs might result in some connections not being brokered during the outage. |
Site management during outage | No |
Enabling or disabling Local Host Cache
LHC can be enabled or disabled as required.
Set-BrokerSite –LocalHostCacheEnabled $True | $False |
Limitations
Desktops must have been assigned before they can be used in outage mode. Unassigned desktops will not be available for brokering. This can lead to desktops being unavailable and reporting “in maintenance mode” if an outage occurs before all assignments have been synchronized, despite a user having actually being assigned a desktop.
Pooled desktops are not supported in outage mode, in the default configuration. There is a workaround, but it has potential security and performance implications. If you configure a Delivery Group containing pooled desktops to not restart on logoff, any powered-on pooled desktops in that group will be available in outage mode. However, after a user logs off, the desktop will not be in a clean state because the desktop is not restarted. This could be a security concern in any scenario. If the next user of that desktop is a local administrator of that desktop, data from a previous user might be accessible. And although that risk is less of a concern for standard (non-admin) users, keep in mind that applications could behave improperly and cause performance issues over time. Important: Administrators should carefully consider the potential implications of using this workaround for using non-restarted pooled desktops in outage mode.
During an outage, no site changes can be made; the database is effectively a snapshot of the main site database and is discarded every time a new synchronization occurs.
Database size
For the 10,000 VDI configuration (that is, 10,000 single-session VDI desktops), the LocalDB was around 75 MB. For the 100,000 RDS configuration (that is, 100,000 users), the LocalDB varied between 100-300 MB, depending on the number of applications and logons. As a copy of the database is taken before a new import starts, allow 1 GB of space for LocalDB.
User sizing considerations
While 10,000 VDAs is the maximum per zone, because sessions will contribute to the load on the elected broker during outage, Citrix recommends that you consider the peak sessions per zone as well. Session density comes into play when using multisession VDAs, as multiple sessions can be initiated to a single VDA.
During an outage, the recommended peak is 25,000 users per zone, which may be able to reach 1-2k resource launches per minute if properly sized.
Application launches are treated similarly to Desktop launches. The same recommendations apply for both; however, Citrix recommends that you also consider the launch rate. A single user may launch multpile applications, thereby increasing the load effected per user on the broker during outage.
When calculating application capacity, stay aware of the average number of applications launched per user and maintain that within the same recommendation of 25,000 per zone.
Summary
During a site database outage, LHC supports a wide range of resources and conditions, but requires planning and consideration of CPU and memory when operating.
In multiple broker sites, any broker might be elected to be the outage broker, and therefore all must have enough resources to cope in outage mode. No evaluation of broker resources is done, so in a site with brokers having varying amounts of resources, it is possible for the least powerful broker to be elected during an outage.
The layout of cores and sockets must be considered as part of the design of the brokers.
The number of published applications and desktops will have an effect on logon response times and the maximum logon throughput.
Brokers with insufficient CPU resource may experience failed launches.
Citrix recommends that you define your antivirus exclusions. For more information, see Tech Paper: Endpoint Security, Antivirus and Antimalware Best Practices.
An additional two cores and 2 GB of RAM is a good starting point for testing performance in LHC outage mode.
1 GB of disk space will be sufficient for the LocalDB database.
An overloaded broker will result in failed connections.