Troubleshoot machines

Note:

Citrix Health Assistant is a tool to troubleshoot configuration issues in unregistered VDAs. The tool automates a number of health checks to identify possible root causes for VDA registration failures and issues in session launch and time zone redirection configuration. The Knowledge Center article, Citrix Health Assistant - Troubleshoot VDA Registration and Session Launch contains the Citrix Health Assistant tool download and usage instructions.

The Filters > Machines view in the Monitor tab displays the machines configured in the site. The Multi-session OS Machines tab includes the load evaluator index, which indicates the distribution of performance counters and tooltips of the session count if you hover over the link.

Click the Failure Reason column of a failed machine to get a detailed description of the failure and actions recommended to troubleshoot the failure. The failure reasons and the recommended actions for machine and connection failures are available in the Citrix Director Failure Reasons Troubleshooting Guide.

Click the machine name link to go to the Machine Details page.

The Machine Details page lists the machine details, infrastructure details, and details of the hotfixes applied on the machine.

Support for HDX Plus for Windows 365 Cloud PCs and Azure Virtual Desktops:

Note:

For HDX Plus for Windows 365 Cloud PCs, only the Restart and Force Restart Power Control options are available. For Azure Virtual Desktops (AVD) all Power Control options are available.

You can view the available Power Control options using one of the following methods:

Click Filters >Sessions > View Details > Machine Details > Power Control drop-down list and select an option to assign the required power control option for a machine.

Or,

Click Filters > Machine > Machine Details > Power Control drop-down list and select an option to assign the required power control option for a machine.

Machine-based real-time resource utilization

The Machine Utilization panel displays graphs showing real-time utilization of CPU and memory. In addition, disk and GPU monitoring graphs are available for sites with VDA versions 7.14 and later.

Disk monitoring graphs, average IOPS, and disk latency are important performance measurements that help you monitor and troubleshoot issues related to VDA disks. The Average IOPS graph displays the average number of reads and writes to a disk. Select Disk Latency to see a graph of the delay between a request for data and its return from the disk, measured in milliseconds.

GPU Utilization

Select GPU Utilization to see the percentage utilization of the GPU, the GPU memory, and of the Encoder and the Decoder to troubleshoot GPU-related issues on multi-session and single-session OS VDAs.

Supported GPU versions:

• NVIDIA Tesla M60 GPUs running Display Driver version 369.17 or later. For more information, see NVIDIA vGPU Software.
• AMD Radeon Instinct MI25 GPUs and AMD EPYC 7V12(Rome) CPUs. For more information, see AMD Drivers and Support.

Drivers:

The appropriate drivers or extensions must be installed on the VDAs.

• For NVIDIA GPUs, install GRID drivers manually or via extensions. For more information, see NVIDIA vGPU Software.

  • Note that for NVIDIA, only GRID drivers are supported. CUDA drivers do not work with the NVadsA10 v5-series and are not supported.
  • For a sample process to install Nvidia Grid GPU drivers via extensions on Azure based machines, see NVIDIA GRID drivers. NVIDIA GPU Driver Extension - Azure Windows VMs - Azure Virtual Machines.
  • For a sample process to install Nvidia Grid GPU drivers manually, see Azure N-series NVIDIA GPU driver setup for Windows - Azure Virtual Machines.

• For AMD GPUs, install AMD graphics drivers manually or via extensions. For more information, see AMD Drivers and Support.

  • For a sample process to install AMD GPU drivers via extensions on Azure based machines, see AMD GPU Driver Extension - Azure Windows VMs - Azure Virtual Machines.
  • For a sample process to install AMD GPU drivers manually on Azure machines, see Install AMD GPU drivers on N-series VMs running Windows.

Usage Notes:

• The GPU Utilization graphs are available only for VDAs running 64-bit Windows.
• The AMD GPU Utilization graphs are available only for VDAs running Citrix Virtual Apps and Desktops 7 2212 or later.
• The VDAs must have HDX 3D Pro enabled to provide GPU acceleration. For more information, see GPU acceleration for Windows Single-session OS and GPU acceleration for Windows Multi-session OS.
• When a VDA accesses more than one GPU, the utilization graph displays the average of the GPU metrics collected from the individual GPUs. The GPU metrics are collected for the entire VDA and not for individual processes.
• For AMD, encoder and decoder usage are not supported separately. Any encoding/ decoding workload using the GPU will be reported as the general 3D load on GPU usage.
• Ensure that you install the NVIDIA WMI during installation. This window is available only during manual installation.
• If drivers are installed but Director does not detect GPU
  • Check Task Manager. If drivers are installed properly, the GPU should show up in Task Manager.
  • Check if the machine is registered. Sometimes machines may take some time to be detected as online.
• If the GPU usage shows no activity in Director, make sure that the workload you are running is using the GPU. For graphics workloads, this can be enabled from Settings > System > Display > Graphics Settings > Choose the app to set preference. Make sure to turn on High Performance. Sometimes, Windows defaults to using the CPU for graphics workloads when this is set to system default or power saving, based on other settings.
• The data is updated every minute and the data visualization starts within a minute of selecting GPU Utilization.

Machine-based historical resource utilization

In the Machine Utilization panel, click View Historical Utilization to view the historical usage of resources on the selected machine. The utilization graphs include critical performance counters of CPU, memory, peak concurrent sessions, average IOPS, and disk latency.

Note:

The Monitoring policy setting, Enable Process Monitoring, must be set to Allowed to collect, and display data in the Top 10 Processes table on the Historic Machine Utilization page. The collection is prohibited by default.

The CPU and memory utilization, average IOPS, and disk latency data is collected by default. You can disable the collection by using the Enable Resource Monitoring policy setting.

1. From the Machine Utilization panel in the Machine Details view, select View Historical Utilization.

2. In the Historical Machine Utilization page, set Time Period to view usage for the last 2 hours, 24 hours, 7 days, month, or year.

   Note:

   Average IOPS and disk latency usage data are available only for the last 24 hours, month, and year ending now. Custom end time is not supported.

3. Click Apply and select the required graphs.
4. Hover over different sections of the graph to view more information for the selected time period.

For example, if you select Last 2 hours, the baseline period is the 2 hours prior to the selected time range. View the CPU, memory, and session trend over the last 2 hours and the baseline time. If you select Last month, the baseline period is the previous month. Select to view the Average IOPS and disk latency over the last month and the baseline time.

1. Click Export to export the resource utilization data for the selected period. For more information, see Export reports section in Monitor Deployments.

2. Below the graphs, the table lists the top 10 processes based on CPU or memory utilization. You can sort by any of the columns, which show Application Name, User Name, Session ID, Average CPU, Peak CPU, Average Memory, and Peak Memory over the selected time range. The IOPS and Disk Latency columns cannot be sorted.

   Note:

   • The session ID for system processes is displayed as “0000”.
   • If a site that belongs to the Citrix Cloud Japan or the Citrix Cloud Government plane contains more than 5000 machines, process data is available for up to 2000 machines only. The Process Monitoring policy must be enabled on these machines.

3. To view the historical trend on the resource consumption of a particular process, drill into any of the Top 10 processes.

Machine Console access

You can access the consoles of Desktop and Multi-session OS machines hosted on XenServer Version 7.3 and later directly from Monitor. This way, you don’t require XenCenter to troubleshoot issues on XenServer hosted VDAs. For this feature to be available, the XenServer hosting the machine must be of Version 7.3 or later and must be accessible from the Monitor.

To troubleshoot a machine, click the Console link in the corresponding Machine Details panel. After authentication of the host credentials you provide, the machine console opens in a separate tab using noVNC, a web-based VNC client. You now have keyboard and mouse access the console.

Note:

• This feature is not supported on Internet Explorer 11.
• If the mouse pointer on the machine console is misaligned, see CTX230727 for steps to fix the issue.
• Console access is launched on a new tab, ensure that your browser settings allow pop-ups.
• For security reasons, Citrix recommends that you install SSL certificates on your browser.

Inspect machines with recent power actions

You can now inspect machines with the status of the success and failed power actions. This feature helps you to analyze the following:

• Power on failure which causes user issues
• Power off failure which increases cost

Note:

Data is available only for the power-managed machine. Data isn’t available for the power actions taken before the feature was supported.

You can view the power action status of the machines using the following ways:

From Filters -> Machines tab. In this case, by default the Power Action Time and the Power Action Result columns are visible. You can also select the columns that you want to visible.

From Cost Optimization tab. In this case, the default filter is Power Action Triggered By is set to Autoscale and the Power Action Result is set to Failed.

With this feature, you can view the details of the power action controls. For example, you can view who triggered the action, which action changed the power state, the reason for failure, and the time when the action is completed. You can also export these details.

The following filters are added to view the power action status:

Microsoft RDS license health

You can view the status of Microsoft RDS license in the Machine Details panel in the Machine Details and the User Details page for Multi-session OS machines.

One of the following messages is displayed:

• License available
• Not configured properly (warning)
• License error (error)
• Incompatible VDA version (error)

Note:

The Microsoft RDS license health status for machines under grace period with valid license displays a License available message in green. Renew your license before they expire.

For warning and error messages, hover over the info icon to view additional information as given in the following table.

Note:

This feature is applicable only for Microsoft RDS CAL (Client Access License).

PVS target device metrics

You can view the status of PVS target devices for single-session and multi-session OS machines on the Machine Details page in Monitor. Several metrics for Network, Boot, and Cache are available on this panel. These metrics help you monitor and troubleshoot PVS target devices to ensure that they are up and running.

Network:

• Network Bandwidth Utilization: Average bandwidth utilization across all NICs.

• Server Reconnect Count: Number of times the server has reconnected due to network issues or server rebalancing or shutdowns and restarts of the Citrix Provisioning Stream Service.

• Total UDP Retry Count: Number of times the provisioning target device has tried to reconnect to the provisioning server using UDP. This metric helps you to know if there are any network issues in the Citrix Provisioning Stream Service (for example, bad switch configurations).

Boot:

• Boot Bytes Read MB: Bytes read while booting.

• Boot Byes Written MB: Bytes written while booting.

• Boot From: Boot medium (vDisk, local disk, and so on).

• Boot Retry Count: Number of retries to boot the machine.

• Boot Time: Time taken to boot the machine, in seconds. By default, there is a 5 second delay between retries. If this delay grows into double digits, there is a significant increase in boot time. Check your provisioning configuration to resolve this issue.

• Target Software Version: Version of the Provisioning target device software.

• vDisk Name: vDisk from which the Provisioning target device is booting.

Cache:

• Write Cache Type: vDisk can be set to different types of cache. For more information, see Knowledge Center article CTX119469.

• Write Cache Volume Drive Letter: Drive letter for write cache types involving drives.

• Write Cache Volume Size MB: Total configured volume size for write cache.

• Cache File Size MB: Current cache file size (cache on device RAM with overflow on hard disk).

• Ram Cache Usage MB: Current RAM cache size (cache on device RAM with overflow on hard disk). Use Overflow to disk only if necessary. This metric is useful when setting or optimizing the proper size of RAM cache.

For more information, see Using the Status Tray on a target device.

Provisioning target device metrics is available only on:

• Provisioning machines.

• Provisioning target device version 7.19 and later.

• VDA version 2003 and later.

Note:

Metrics for Server Reconnect Count and UDP Retry Count are available only for Provisioning target version 1912 CU2 and later.