www.cendio.com
Bug 5241 - upgrade Xorg
: upgrade Xorg
Status: CLOSED FIXED
: ThinLinc
VNC
: trunk
: PC Unknown
: P2 Normal
: 4.10.0
Assigned To:
:
:
: 7158 7224 7245
: 400 4834 6177 6234 7139
  Show dependency treegraph
 
Reported: 2014-09-04 16:00 by
Modified: 2019-03-01 15:30 (History)
Acceptance Criteria:
* Our Xvnc should be based on the latest stable Xorg release * Xorg dependencies should be upgraded (e.g. all the X11 proto libs), see the list in the serverdeps spec file * The bundled swrast_dri.so should be upgraded to the latest Mesa * The xkb tools and data should be updated to the latest version


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From cendio 2014-09-04 16:00:06
We are currently using version 1.14 of the Xorg server for our Xvnc. There has
been at least two releases since then, so we should look at upgrading to make
sure we get new bug fixes and functionality.
------- Comment #1 From cendio 2014-09-05 10:24:15 -------
Remember to check for new extensions which might require more dependencies in
our build system.
------- Comment #2 From cendio 2014-12-08 15:00:35 -------
They changed the integration with Mesa in 1.15 that might cause problems for
us. See upstream commit be6680967a479eedbcab2fe1718c5f981e1029c7.
------- Comment #3 From cendio 2016-06-21 08:01:13 -------
We are now quite a bit behind everyone else. Even RHEL 6 has been upgraded to
1.17 (from 1.15).
------- Comment #7 From cendio 2018-08-29 13:26:14 -------
We require a newer libdrm, which in turn needs a newer glibc. So we need to do
bug 7224 first.
------- Comment #10 From cendio 2018-08-30 13:32:08 -------
A newer mesa is also required, which also needs a newer glibc.
------- Comment #14 From cendio 2018-08-31 10:15:44 -------
All dependencies are now updated and I'm able to do a prototype build with an
upgraded Mesa. However I'm not able to get a properly linked Xvnc because of
changes to the Xorg/Mesa interaction:

* Xorg no longer includes some Frankenstein copy of parts of Mesa. Instead it
expects to link to libGL and get what it needs.

* Xorg still manually loads swrast_dri.so though. (because it cannot use the
standard context setup in libGL?)

* Mesa refuses to do a static build with DRI enabled. Not sure how that will
interact with loading swrast_dri.so, or the DRI the X11 applications need to
use.

So right now I have a runtime dependency on libGL. It seems to work on my
Fedora though, but it's unknown what problems that dependency might cause.

More investigation is needed.
------- Comment #16 From cendio 2018-08-31 15:32:41 -------
I tried a static build of Mesa, even though I had to disable a whole bunch of
things to get it through. It actually seems to work properly despite the lack
of DRI code.

I tested the following:

 * Direct rendering:

   Works well. I can see llvmpipe being reported as the renderer and I see a
bunch of llvmpipe threads being active

 * Indirect rendering:

   Also works well. I don't see any traces of llvmpipe though. But then again,
I can't see any traces of it in 4.9.0 either.

Since this seems to work I'm not going to try to dig deeper in to the DRI
magic.
------- Comment #17 From cendio 2018-08-31 15:41:34 -------
(In reply to comment #16)
> 
>    Also works well. I don't see any traces of llvmpipe though. But then again,
> I can't see any traces of it in 4.9.0 either.
> 

Seems like there are two "swrast", llvmpipe and an simpler older one. We've
always used the older one in ThinLinc. So there might be room for improvement.
But it would only affect indirect rendering, so it's probably not worth the
effort.
------- Comment #24 From cendio 2018-09-04 09:28:11 -------
The linking of swrast_dri.so required some adjustments. Normally it would link
to libglapi.so. However we linked that statically in to Xvnc, and we don't want
a second copy in swrast_dri.so as that library contains dispatch logic for the
OpenGL calls.

So we do a manual link where all the glapi symbols are left undefined and will
get resolved as swrast_dri.so is loaded in to Xvnc at runtime. The downside of
this is that we can't have the linker check for unresolved symbols and get an
early error of any link issues.
------- Comment #25 From cendio 2018-09-04 09:29:08 -------
I had to revert the fix for bug 6044 as that code had gotten a large rewrite
upstream. The principle seems to have been altered though, so hopefully the
issue is still gone. Will have to re-test.
------- Comment #27 From cendio 2018-09-04 13:25:23 -------
This upgrade also disables indirect rendering by default as upstream consider
that code untrustworthy:

    Almost every situation of someone running indirect GLX is a mistake
    that results in X Server crashes.  Indirect GLX is the cause of
    regular security vulnerabilities, and rarely provides any capability
    to the user.  Just disable it unless someone wants to enable it for
    their special use case (using +iglx on the command line).

We should probably mention this in the release notes, but I don't think we need
to be proactive with documenting the flag in the TAG (just like most arguments
to Xvnc aren't documented).
------- Comment #31 From cendio 2018-09-06 15:13:49 -------
(In reply to comment #25)
> I had to revert the fix for bug 6044 as that code had gotten a large rewrite
> upstream. The principle seems to have been altered though, so hopefully the
> issue is still gone. Will have to re-test.

I retested this on RHEL 6 with Chromium 68. I could reproduce the issue with
ThinLinc 4.6.0, but after an upgrade to the nightly build the problem went
away.
------- Comment #32 From cendio 2018-09-10 09:34:34 -------
Everything should be done here. Ready for regression testing.
------- Comment #33 From cendio 2018-09-12 09:58:30 -------
I can not find any issues in general usage with normal applications. I've also
run some conformance tests and compared results to 4.9.0:


 - rendercheck: no change

42336 tests passed of 42948 total
Successful Groups:
    fill, dcoords, scoords, mcoords, tscoords
    tmcoords, blend, composite, cacomposite, repeat
    bug7366, gtk_argb_xbgr, libreoffice_xrgb

 - xts: Slight improvement

========================
109 of 1078 tests failed
(339 tests were not run)
========================

vs

========================
112 of 1078 tests failed
(337 tests were not run)
========================

 - piglit: Much better

[39411/39411] skip: 15897, pass: 23345, warn: 2, fail: 166, crash: 1         

vs

[39411/39411] skip: 31205, pass: 8112, warn: 2, fail: 88, crash: 4
------- Comment #34 From cendio 2018-09-12 09:59:26 -------
Piglit crashes Xvnc in indirect mode (see bug 3625), so I couldn't test that
the same way. I did however run mesa's demos in indirect mode and didn't see
any issues.
------- Comment #35 From cendio 2018-09-12 12:09:00 -------
Similar results when running tests on RHEL 6:

 - rendercheck:

42336 tests passed of 42948 total
Successful Groups:
    fill, dcoords, scoords, mcoords, tscoords
    tmcoords, blend, composite, cacomposite, repeat
    bug7366, gtk_argb_xbgr, libreoffice_xrgb

 - xts:

========================
110 of 1078 tests failed
(338 tests were not run)
========================

vs

========================
112 of 1078 tests failed
(337 tests were not run)
========================

 - piglit:

[26100/26100] crash: 9, fail: 382, pass: 11139, skip: 14568, warn: 2 |       

vs

[26100/26100] crash: 8, fail: 98, pass: 6266, skip: 19726, warn: 2 |
------- Comment #36 From cendio 2018-09-13 09:28:35 -------
Performance tests:

 - xrenderbenchmark:

   Mostly the same. Perhaps a few percentages better in some places.

   HOWEVER! We see a massive performance reduction in a bunch of operations.
Apparently this is per design and caused by upstream commit 15aa37ad. The old
code had rounding errors so they had to resort to a slower code path for these
things. Hopefully these are not common operations.

 - gtkperf

   Mostly the same. Text rendering is slower for some reason though. Seems to
have something to do with change tracking as I'm seeing a lot of CPU time in
the region code. Will investigate further.
------- Comment #37 From cendio 2018-09-13 10:57:57 -------
(In reply to comment #36)
>  - gtkperf
> 
>    Mostly the same. Text rendering is slower for some reason though. Seems to
> have something to do with change tracking as I'm seeing a lot of CPU time in
> the region code. Will investigate further.

I'm having a hard time pinpointing the cause of this. The cause seems to be
two-fold:

 - 25% more overhead going from 4.9.0 to new TigerVNC / old Xorg. However the
old version gets bogged down by the text rendering and fails to send updates to
the client. So the user experience is a lot better now. And more updates means
more CPU, so this is natural.

 - 25% more overhead going from old Xorg to new Xorg (both with new TigerVNC).
This is where it spends more CPU tracking changes. It is also spending more
time in select(). Perhaps the select() loop is rewritten, causing the pattern
of updates to change, in turn causing the complexity of the changed region
between updates to change?

It is still very fast, so I don't think it's worth digging further in to.
------- Comment #38 From cendio 2018-09-13 13:24:34 -------
Everything seems to work fine and I can't think of any more things to test.

Someone else should have a look that the release notes look fine, and then we
should be done here.
------- Comment #39 From cendio 2018-09-14 15:37:03 -------
(In reply to comment #38)
> Everything seems to work fine and I can't think of any more things to test.
> 
> Someone else should have a look that the release notes look fine, and then we
> should be done here.

Release notes look fine.
------- Comment #40 From cendio 2018-09-17 15:42:26 -------
> * Our Xvnc should be based on the latest stable Xorg release
> * Xorg dependencies should be upgraded (e.g. all the X11 proto libs)

Mesa, makedepend, libSM, libICE and libXxf86misc are not up to date.

| component    | cenbuild | latest | up to date? |
|--------------+----------+--------+-------------|
| libICE       |    1.0.4 |  1.0.9 | n           |
| libSM        |    1.1.0 |  1.2.2 | n           |
| libXxf86misc |    1.0.1 |  1.0.4 | n           |
| makedepend   |    1.0.4 |  1.0.5 | n           |
| mesa         |   18.1.7 | 18.1.8 | n           |
|--------------+----------+--------+-------------|
| libX11       |    1.6.6 |  1.6.6 | y           |
| libXau       |    1.0.8 |  1.0.8 | y           |
| libXdamage   |    1.1.4 |  1.1.4 | y           |
| libXdmcp     |    1.1.2 |  1.1.2 | y           |
| libXext      |    1.3.3 |  1.3.3 | y           |
| libXfixes    |    5.0.3 |  5.0.3 | y           |
| libXfont     |    2.0.3 |  2.0.3 | y           |
| libXrandr    |    1.5.1 |  1.5.1 | y           |
| libXxf86vm   |    1.1.4 |  1.1.4 | y           |
| libdrm       |   2.4.94 | 2.4.94 | y           |
| libxkbfile   |    1.0.9 |  1.0.9 | y           |
| libxshmfence |      1.3 |    1.3 | y           |
| pixman       |   0.34.0 | 0.34.0 | y           |
| xcb-proto    |     1.13 |   1.13 | y           |
| font-util    |    1.3.1 |  1.3.1 | y           |
| util-macros  |   1.19.2 | 1.19.2 | y           |
| xorgproto    |   2018.4 | 2018.4 | y           |
| xtrans       |    1.3.5 |  1.3.5 | y           |
|--------------+----------+--------+-------------|


> * The bundled swrast_dri.so should be upgraded to the latest Mesa

Mesa is not up to date.

| component |    ctc | latest | up to date? |
|-----------+--------+--------+-------------|
| vnc/mesa  | 18.1.7 | 18.1.8 | n           |
|-----------+--------+--------+-------------|


> * The xkb tools and data should be updated to the latest version

| component            |   ctc | latest | up to date? |
|----------------------+-------+--------+-------------|
| vnc/setxkbmap        | 1.3.1 |  1.3.1 | y           |
| vnc/xkeyboard-config |  2.24 |   2.24 | y           |
| vnc/xkbcomp          | 1.4.2 |  1.4.2 | y           |
|----------------------+-------+--------+-------------|
------- Comment #44 From cendio 2018-09-18 15:01:46 -------
(In reply to comment #40)
> > * Our Xvnc should be based on the latest stable Xorg release
> > * Xorg dependencies should be upgraded (e.g. all the X11 proto libs)
> 
> Mesa, makedepend, libSM, libICE and libXxf86misc are not up to date.
> 

Mesa has been updated. makedepend isn't actually needed anymore (I forgot to
update the serverdeps list when doing the initial upgrade).

The last three libraries are not used by Xvnc so they have not been changed.
------- Comment #45 From cendio 2018-09-27 14:28:48 -------
*** Bug 7252 has been marked as a duplicate of this bug. ***
------- Comment #47 From cendio 2018-09-27 14:54:23 -------
The required dependencies have now been upgraded and the release note looks
good.