www.cendio.com
Bug 4416 - Xvnc crashes on Gnome login (composite)
: Xvnc crashes on Gnome login (composite)
Status: CLOSED FIXED
: ThinLinc
VNC
: 3.4.0
: PC Unknown
: P2 Normal
: 4.1.0
Assigned To:
:
:
: 4417
:
  Show dependency treegraph
 
Reported: 2012-10-05 14:11 by
Modified: 2013-06-24 10:03 (History)
Acceptance Criteria:


Attachments


Note

You need to log in before you can comment on or make changes to this bug.


Description From cendio 2012-10-05 14:11:24
Reported in Issue 13451:

Xvnc crashes when Gnome starts up, but only for some users. Managed to get a
core file finally, and it gives us this backtrace:

(gdb) bt
#0  0x00000000006ae784 in sse2_composite_src_x888_8888 ()
#1  0x00000000006690b5 in pixman_image_composite32 ()
#2  0x00000000004578fd in fbComposite (op=1 '\001', pSrc=0x1294100, pMask=0x0, 
    pDst=0x12944e0, xSrc=<optimized out>, ySrc=<optimized out>, xMask=0, 
    yMask=0, xDst=0, yDst=0, width=30, height=26) at fbpict.c:185
#3  0x000000000054119c in vncHooksComposite (op=1 '\001', pSrc=0x1294100, 
    pMask=0x0, pDst=0x12944e0, xSrc=<optimized out>, ySrc=<optimized out>, 
    xMask=0, yMask=0, xDst=0, yDst=0, width=30, height=26) at vncHooks.cc:635
#4  0x00000000004effe6 in damageComposite (op=1 '\001', pSrc=0x1294100, 
    pMask=0x0, pDst=0x12944e0, xSrc=0, ySrc=0, xMask=0, yMask=0, xDst=0, 
    yDst=0, width=30, height=26) at damage.c:576
#5  0x000000000053c89d in compNewPixmap (pWin=<optimized out>, 
    x=<optimized out>, y=<optimized out>, w=30, h=26) at compalloc.c:522
#6  0x000000000053d56c in compReallocPixmap (pWin=0x1244320, 
    draw_x=<optimized out>, draw_y=<optimized out>, w=30, h=<optimized out>, 
    bw=<optimized out>) at compalloc.c:620
#7  0x000000000053b533 in compResizeWindow (pWin=<optimized out>, x=0, y=0, 
    w=30, h=26, pSib=0x1226a30) at compwindow.c:401
#8  0x00000000005e708a in ConfigureWindow (pWin=<optimized out>, 
    mask=<optimized out>, vlist=<optimized out>, client=<optimized out>)
    at window.c:2483
#9  0x00000000005b901e in ProcConfigureWindow (client=0x1049640)
    at dispatch.c:764
#10 0x00000000005beeac in Dispatch () at dispatch.c:454
#11 0x00000000005d4d5a in main (argc=22, argv=0x7fff8ac54b88, 
    envp=<optimized out>) at main.c:441
------- Comment #1 From cendio 2012-10-05 16:05:24 -------
Initial analysis:

The triggering condition is a window being resized, which in composite mode
results in a new backing pixmap being created. In order for this pixmap to have
sensible initial data, the code tries to copy the data from the old window
over.

The crash happens because of a read from a bad address (0x7f515433feb0).

Digging upwards in the stack, this seems to come from the frame buffer, which
is at address 0x7f5154039010. It is 1024x768 and has a stride of 4096 bytes.
Calculating the offset gives us a line number of 774, which is outside the
valid memory area.

The reason this happens seems to be that the source window is not the root
window, but one that is partially off screen. Hence it gets a height that isn't
cropped by the backing frame buffer.



Unable to reproduce it locally though, so there is some finer point to this
that I'm yet to discover.
------- Comment #2 From cendio 2012-10-17 10:37:47 -------
There was a lot of work done upstream to handle out-of-bounds access in this
specific scenario, so I'm just going to assume it is fixed there and an upgrade
of xorg will fix it for us.

I tried cherry-picking the most relevant commit, but it results in massive
rendering bugs.

Moving this forward until we can upgrade xorg.
------- Comment #3 From cendio 2012-11-12 14:07:32 -------
This bug happens on SLED11, about every 2 logins or so.
------- Comment #4 From cendio 2012-11-12 14:17:40 -------
Traceback on SLED11:

#0  sse2_composite_src_x888_8888 (imp=<optimized out>, op=<optimized out>,
src_image=<optimized out>, mask_image=<optimized out>, 
    dst_image=<optimized out>, src_x=<optimized out>, src_y=0, mask_x=0,
mask_y=0, dest_x=0, dest_y=0, width=1, 
    height=<optimized out>) at pixman-sse2.c:2923
#1  0x00000000006690b5 in pixman_image_composite32 (op=<optimized out>,
src=0x167bf80, mask=0x0, dest=0x167c0a0, src_x=2, 
    src_y=0, mask_x=0, mask_y=0, dest_x=0, dest_y=0, width=1, height=24) at
pixman.c:848
#2  0x00000000004578fd in fbComposite (op=1 '\001', pSrc=0x1669fa0, pMask=0x0,
pDst=0x166a040, xSrc=<optimized out>, 
    ySrc=<optimized out>, xMask=0, yMask=0, xDst=0, yDst=0, width=1, height=24)
at fbpict.c:185
#3  0x000000000054119c in vncHooksComposite (op=1 '\001', pSrc=0x1669fa0,
pMask=0x0, pDst=0x166a040, xSrc=<optimized out>, 
    ySrc=<optimized out>, xMask=0, yMask=0, xDst=0, yDst=0, width=1, height=24)
at vncHooks.cc:635
#4  0x00000000004effe6 in damageComposite (op=1 '\001', pSrc=0x1669fa0,
pMask=0x0, pDst=0x166a040, xSrc=2, ySrc=0, xMask=0, 
    yMask=0, xDst=0, yDst=0, width=1, height=24) at damage.c:576
#5  0x000000000053c89d in compNewPixmap (pWin=<optimized out>, x=<optimized
out>, y=<optimized out>, w=1, h=24) at compalloc.c:522
#6  0x000000000053d56c in compReallocPixmap (pWin=0x16693a0, draw_x=<optimized
out>, draw_y=<optimized out>, w=1, 
    h=<optimized out>, bw=<optimized out>) at compalloc.c:620
#7  0x000000000053b533 in compResizeWindow (pWin=<optimized out>, x=2, y=0,
w=1, h=24, pSib=0x1615110) at compwindow.c:401
#8  0x00000000005e708a in ConfigureWindow (pWin=<optimized out>,
mask=<optimized out>, vlist=<optimized out>, 
    client=<optimized out>) at window.c:2483
#9  0x00000000005b901e in ProcConfigureWindow (client=0x1556580) at
dispatch.c:764
#10 0x00000000005beeac in Dispatch () at dispatch.c:454
#11 0x00000000005d4d5a in main (argc=22, argv=0x7fff6f8d6048, envp=<optimized
out>) at main.c:441
------- Comment #5 From cendio 2012-11-12 14:24:31 -------
Unfortunately it also happens with the 32-bit version of Xvnc:

#0  sse2_composite_src_x888_8888 (imp=0x83ccc38, op=PIXMAN_OP_SRC,
src_image=0x8846248, mask_image=0x0, dst_image=0x882cb68, 
    src_x=0, src_y=0, mask_x=0, mask_y=0, dest_x=0, dest_y=0, width=48,
height=<optimized out>) at pixman-sse2.c:2923
#1  0x082ad6e2 in pixman_image_composite32 (op=<optimized out>, src=0x8846248,
mask=0x0, dest=0x882cb68, src_x=0, src_y=0, 
    mask_x=0, mask_y=0, dest_x=0, dest_y=0, width=48, height=24) at
pixman.c:848
#2  0x0808cc19 in fbComposite (op=1 '\001', pSrc=0x87ff0a0, pMask=0x0,
pDst=0x882bc18, xSrc=1263, ySrc=1155, xMask=0, yMask=0, 
    xDst=0, yDst=0, width=48, height=24) at fbpict.c:185
#3  0x0817daaa in vncHooksComposite (op=1 '\001', pSrc=0x87ff0a0, pMask=0x0,
pDst=0x882bc18, xSrc=0, ySrc=0, xMask=0, yMask=0, 
    xDst=0, yDst=0, width=48, height=24) at vncHooks.cc:635
#4  0x08124c1c in damageComposite (op=1 '\001', pSrc=0x87ff0a0, pMask=0x0,
pDst=0x882bc18, xSrc=0, ySrc=0, xMask=0, yMask=0, 
    xDst=0, yDst=0, width=48, height=24) at damage.c:576
#5  0x08179559 in compNewPixmap (pWin=<optimized out>, x=1263, y=1155, w=48,
h=24) at compalloc.c:522
#6  0x0817a407 in compReallocPixmap (pWin=0x882e350, draw_x=1263, draw_y=1155,
w=48, h=24, bw=0) at compalloc.c:620
#7  0x08177d08 in compResizeWindow (pWin=0x882e350, x=0, y=0, w=48, h=24,
pSib=0x8791538) at compwindow.c:401
#8  0x08225b8f in ConfigureWindow (pWin=0x882e350, mask=15, vlist=0x87aca00,
client=0x87682e8) at window.c:2483
#9  0x081f9ede in ProcConfigureWindow (client=0x87682e8) at dispatch.c:764
#10 0x081ffa7f in Dispatch () at dispatch.c:454
#11 0x0821751a in main (argc=22, argv=0xffac9494, envp=0xffac94f0) at
main.c:441
------- Comment #6 From cendio 2012-11-12 16:23:22 -------
Workaround that works on SLED11: disable Composite with:

-extension Composite
------- Comment #8 From cendio 2013-04-23 12:46:50 -------
This needs to be retested with our upgraded Xorg.
------- Comment #9 From cendio 2013-06-24 10:03:25 -------
I have been doing a lot of testing against SLED 11 Sp2 without stumble upon
this issue. I have also tried to reproduce it without success so we could
probably consider this issue being solved with the big Xorg update in 4.1.0