Hi,
First I would like to say that porting the 2.6 kernel version drivers to 4.x is not at all an easy operation and it will need some effort as kernels are VERY different.
Second the kernel and os (package manager manly) are 2 different things, and you can keep using exactly the same rootfs from the official image but use a new kernel without a problem.
There are already some patches for 3.x kernel version for the same issue ( or a similar one)
I can try to help you , if it works than OK, if it doesn't - that's a possibility as I don't have the HW here and I won't be able to do indeep kernel debugging - then you check someone else
Let me know if you are interested.
cheers