RAD Software

Hi there
I'm sure problem can be easily identified and fixed but I'm not sure where to start.
MB : Gigabyte Z170X Ultra Gaming
CPU : i7-7700k
GPU : Sapphire RX580 nitro+ (with dual bios)
Main drive for OSX : Samsung 970 Pro
Second drive windows : Samsung Evo 860
RAM : 32GB Corsair Dominator Platinium
I have 3 monitors (2x display port + 1 hdmi)
This glitch/freeze happens both in macos and Windows under 'heavy' load, either gaming on Windows or video editing on macos (mostly using final cut)
When it happens it freezes (the sound continues) and I have to hard reset.
Here is how it looks :
Again, this happen both in macos and windows so I doubt this is a hackintosh problem.
I'm pretty new to this so I'm not sure where to start.
Bios ? VBios ? NVRAM ?

Thanks
EDIT :
SOLVED - Turns out my gpu is probably dying, but the problem looked like a VRAM problem.
Changing the max memory frequency from 2000mhz to 1900mhz did the trick.
I used radeon software in windows to find the balance between performance and crashes (lol) and decreased the memory 50mhz at a time. Did the same tasks, test, games etc, instant freeze and glitch at 2000mhz and 1950mhz. No freezes and glitches at 1900mhz.
Runned a few more stress test and real life workloads, no glitches.
Now for those changes to be applied in macOs too j Iust tweaked the vbios of the card cuz radeon software is obviously not usable outside windows.
I used SBRPolaris to tweak the parameters and flashed the card using ATIflash.
Now the card is maxed out at 1900mhz and all the tasks that tiggered the freeze/artifacts before now work fine.
Did some super heavy render in final cut pro (I have a project with 10 different 4k footages stacked on top of each other using chroma key, masking and stuff, used to crash almost instantly everytime. Now the export went all the way without problem.
I bet I have lost some power and overall speed doing that but at least it works now.
Next stop is a Radeon VII, in a few months probably, or maybe a 5700 since radeon VII are discontinued.