.. _HA-install:

Configuration of ThinLinc for HA Operations
-------------------------------------------

In this section, we describe how ThinLinc is configured for High
Availability.

Installation of a New HA Cluster
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In this section, we will describe how to setup a new HA cluster. In the
examples we will use a primary node with the hostname **tlha-primary**
and IP address **10.0.0.2**, a secondary node with the hostname
**tlha-secondary** and IP address **10.0.0.3**, and a resource IP
address of **10.0.0.4** with the DNS name **tlha**.

1. Begin by installing ThinLinc as described in :ref:`installation` on
   both nodes.

2. Both nodes in the HA cluster must have the same SSH host key. Copy
   :file:`/etc/ssh/ssh_host_*` from the primary host to the secondary
   host, and restart ssh on the secondary host.

3. Install and configure the system-level high-availability software,
   for example the software provided by the ClusterLabs project, which
   can be found at https://clusterlabs.org. This and other
   high-availability software may also be provided as part of your
   distribution, so check for the solution which best fits your
   requirements before proceeding.

4. Configure the system's high-availability software to watch the status
   of the other machine via the network, and to enable the resource IP
   address **10.0.0.4** on the active node. The machine with the
   hostname **tlha-primary** should normally be active.

5. Configure each VSM agent to allow privileged operations both from
   **tlha-primary** and **tlha-secondary**:

   .. code:: console

      $ sudo tl-config '/vsmagent/allowed_clients=tlha-primary tlha-secondary'

   Also, set the :servconf:`/vsmagent/master_hostname` to the DNS name
   of the HA interface:

   .. code:: console

      $ sudo tl-config /vsmagent/master_hostname=tlha

   Restart all VSM agents after changing the configuration values.

6. Verify operations of VSM Server on both nodes. Make sure you can
   start the VSM server properly on both hosts, and connect to the
   respective hosts when VSM server is running (i.e., it should be
   possible to connect, using :program:`tlclient`, to both
   **tlha-primary** and to **tlha-secondary**).

   Both nodes should be configured with the same subcluster
   configuration.

   .. warning::

      It is **very important** that 127.0.0.1 is not in the list of
      agent servers of any subcluster. If the machines running VSM
      server are also VSM agents, their unique hostnames or IP addresses
      must be added to :servconf:`/vsmserver/subclusters/<name>/agents`
      instead of 127.0.0.1. The reason for this is that 127.0.0.1 will
      be a different server based on which VSM server is currently
      active.

7. After verifying that normal ThinLinc connections work as intended
   when using both the primary and the secondary VSM server's hostname,
   it is time to enable HA in the VSM servers. This is done by setting
   :servconf:`/vsmserver/HA/enabled` to ``1``, and by specifying the
   nodes in the cluster in :servconf:`/vsmserver/HA/nodes`. For example:

   .. code:: console

      $ sudo tl-config /vsmserver/HA/enabled=1
      $ sudo tl-config '/vsmserver/HA/nodes=tlha-primary.example.com tlha-secondary.example.com'

   Configuration should be identical on both nodes. Restart the VSM
   server on both nodes after configuration.

8. If vsmserver can't safely determine which of the two nodes in
   :servconf:`/vsmserver/HA/nodes` is the remote node, and which is the
   local node, it will start without HA enabled, and log a message. If
   this happens, validate your hostname and DNS setup. One of the
   entries of :servconf:`/vsmserver/HA/nodes` must match the local
   machine. Either the resolved IP of one of the entries in
   :servconf:`/vsmserver/HA/nodes` must match the local IP, or one entry
   must exactly match the local hostname as returned by
   :command:`uname -n`.

9. Once HA has been configured, tests should be performed in order to
   confirm that the failover works as expected. This can normally be
   done by simply removing the network cable from the primary node, and
   ensuring that the secondary node then takes over. Check also that any
   active ThinLinc sessions have been synchronized from the primary to
   the secondary node, and that logging in to such a session on the
   secondary node succeeds once the primary node has been disabled.

Your ThinLinc HA cluster is now configured! When sessions are created,
changed or deleted on the currently active node, the information about
them will be transferred to the other node using a inter-VSM server
protocol. If the other node has to take over service, its copy of the
session data will be up to date, and it can start serving new requests
instantly. When the primary node comes up again, the secondary node will
resynchronise with the master.

Reconfiguring an existing ThinLinc Installation into HA mode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you have an existing ThinLinc installation and want to eliminate the
single point of failure (the VSM server), the procedure is very much
like the procedure for installing a new HA cluster.