No HW cursor on oprange pi pc2 (sun50i H5 chip)

[marsohod4you]

hi
I am trying to force my "orange pi pc2" board work with armsoc driver.
Purpose is to make armsoc+sunxi-mali gpu+opengles2 but now even simply i cannot make armsoc work properly.
There are several issues like no HW cursor and very strange display output like not all colors displayed or something like that. First I want to understand HW cursor issues and then maybe other issues would disappear. So I dig into HW cursor implementation first.

Very strange thing.
My Xorg config in /etc/X11/xorg.conf is very simple:
`# cat /etc/X11/xorg.conf
Section "Device"
Identifier "sun4i-DRM"
Driver "armsoc"
Option "DRI2" "true"
Option "InitFromFBDev" "/dev/fb0"
Option "Debug" "true"
EndSection

Section "Screen"
Identifier "Screen0"
Device "sun4i-DRM"
EndSection
`
No cursor on screen.
During armsoc driver (I built myself) debug I found that function drmModeSetPlane() always fails with error -22 "invalid parameter".

Further investigations show that error probably happens at early step in function drmmode_cursor_init_plane()

This function takes planes with call
plane_resources = drmModeGetPlaneResources(drmmode->fd);
Then assumed that first plane should be cursor plane (am I right?):
ovr = drmModeGetPlane(drmmode->fd, plane_resources->planes[0]);

I was trying to print all formats supported by this plane. I modified armsoc source debug output like this code:
ovr = drmModeGetPlane(drmmode->fd, plane_resources->planes[0]);
if (!ovr) {
ERROR_MSG("HW cursor: drmModeGetPlane failed: %s",
strerror(errno));
drmModeFreePlaneResources(plane_resources);
return FALSE;
}
INFO_MSG("???CURSOR got plane, count formats %d",ovr->count_formats);
for(i=0; icount_formats; i++)
{
char* pf=&ovr->formats[i];
INFO_MSG(" ???CURSOR format %08X %c%c%c%c",ovr->formats[i],pf[0],pf[1],pf[2],pf[3]);
}

After armsoc rebuild and put armsoc_drv.so on target device it prints me following plane formats:
cat /var/log/Xorg.0.log | grep ???
[ 55.702] (II) ARMSOC(0): ???CURSOR count planes 3
[ 55.702] (II) ARMSOC(0): ???CURSOR got plane, count formats 32
[ 55.702] (II) ARMSOC(0): ???CURSOR format 35314241 AB15
[ 55.702] (II) ARMSOC(0): ???CURSOR format 32314241 AB12
[ 55.702] (II) ARMSOC(0): ???CURSOR format 35315241 AR15
[ 55.702] (II) ARMSOC(0): ???CURSOR format 32315241 AR12
[ 55.703] (II) ARMSOC(0): ???CURSOR format 36314742 BG16
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34324742 BG24
[ 55.703] (II) ARMSOC(0): ???CURSOR format 35314142 BA15
[ 55.703] (II) ARMSOC(0): ???CURSOR format 32314142 BA12
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34325842 BX24
[ 55.703] (II) ARMSOC(0): ???CURSOR format 36314752 RG16
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34324752 RG24
[ 55.703] (II) ARMSOC(0): ???CURSOR format 32314152 RA12
[ 55.703] (II) ARMSOC(0): ???CURSOR format 35314152 RA15
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34325852 RX24
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34324258 XB24
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34325258 XR24
[ 55.703] (II) ARMSOC(0): ???CURSOR format 3631564E NV16
[ 55.703] (II) ARMSOC(0): ???CURSOR format 3231564E NV12
[ 55.703] (II) ARMSOC(0): ???CURSOR format 3132564E NV21
[ 55.703] (II) ARMSOC(0): ???CURSOR format 3136564E NV61
[ 55.703] (II) ARMSOC(0): ???CURSOR format 59565955 UYVY
[ 55.703] (II) ARMSOC(0): ???CURSOR format 59555956 VYUY
[ 55.703] (II) ARMSOC(0): ???CURSOR format 56595559 YUYV
[ 55.703] (II) ARMSOC(0): ???CURSOR format 55595659 YVYU
[ 55.703] (II) ARMSOC(0): ???CURSOR format 31315559 YU11
[ 55.703] (II) ARMSOC(0): ???CURSOR format 32315559 YU12
[ 55.703] (II) ARMSOC(0): ???CURSOR format 36315559 YU16
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34325559 YU24
[ 55.703] (II) ARMSOC(0): ???CURSOR format 31315659 YV11
[ 55.703] (II) ARMSOC(0): ???CURSOR format 32315659 YV12
[ 55.703] (II) ARMSOC(0): ???CURSOR format 36315659 YV16
[ 55.703] (II) ARMSOC(0): ???CURSOR format 34325659 YV24

At this moment I realize that this plane does not support alpha channel?
I do not know why but later code
rmModeAddFB2(drmmode->fd, w + 2 * pad, h, DRM_FORMAT_ABGR8888,
handles, pitches, offsets, &cursor->fb_id, 0))

Does not give error, so driver thinks that no error.
But as I already wrote function drmModeSetPlane() for cursor always fails errno -22.

I continued to study DRM source code (I use Linux kernel 4.2.17)
I see that error happens in
drivers/gpu/drm/drm_plane.c function drm_mode_setplane() => setplane_internal() =>__setplane_check() function drm_check_pixel_format() fails.

Interesting is that if I modify armsoc driver source to choose second plane for cursor:
ovr = drmModeGetPlane(drmmode->fd, plane_resources->planes[1]);
then HW cursor appears on screen and i can move it with mouse!!!!

After that I was wondering what are those planes in my display.
So I begin to read sun4i sources.
I see that I have 3 planes created in sun8i-mixer: 1st plane is so called VI plane (not sure what is this) and 2 planes UI planes.
Interesting is that sun4i/sun8i_vi_layer.c has a list of formats exactly which I printed in my armsoc debug version, the same 32 format and they write that VI plane does not support ALPHA.
Contrary sun4i/sun8i_ui_layer.c has declared 20 formats and some of then support ALPHA.

At this moment I am totally confused. Of cause I do not understand fully how it ever works.
I do not understand why we choose cursor plane[0] which does not support alpha - alpha is necessary for cursor function.

Maybe issue is somewhere deeper like i have wrong dts or or something else.
But what I am doing wrong?
Why I have no normally HW cursor and should think to change cursor plane to 2nd?
Why display image is very strange like lack of colors or wrong color format?
No idea..