Crunching in a Virtual Machine

DanNeely
DanNeely
Joined: 4 Sep 05
Posts: 1,358
Credit: 2,491,558,424
RAC: 3,202,931

RE: Guess you installed a

Quote:
Guess you installed a single CPU version of windows. Did it only have one vCPU when you installed windows?
Look at this thread

IIRC that's only an old version of Windows problem; Vista/Server08 and later only have multi-processor kernels.

Logforme
Logforme
Joined: 13 Aug 10
Posts: 332
Credit: 1,714,373,961
RAC: 0

Another beautiful theory

Another beautiful theory ruined by an ugly fact :)

robl
robl
Joined: 2 Jan 13
Posts: 1,687
Credit: 1,358,131,416
RAC: 734,023

Feeling like I was "rode hard

Feeling like I was "rode hard and put up wet", but I believe I have achieved GPU "pass through" of a Radeon HD 6450. Now just because we have "pass through" does not apparently mean we can crunch GPU WUs from E@H.

Let me show you what I have visually and why I believe I have achieved "pass through" on the device noted above.

The image that follows shows a vncserver login to the virtual machine. It seems that when you get pass through working you can no longer login through the "console" window in xencenter ( the large 3rd window down). In that window you will also notice that it states this fact with "This vm has a pass-through GPU assinged. You must connect to it using Remote Desktop". Remember U-soft remote desktop is not available on all Windows 7 releases, hence I chose VNC server and client. They are free but reasonably priced should you want a licensed version.

In the first window (Device manager on the VM) you will notice that 3 display adapters are listed: AMD RADEON HD 6450, the Standard VGA graphics (disabled by windows - yellow triangle, and VNC Mirror Driver. This indicated to me that the Radeon device is truly passed through to the windows VM. The 4th small window shows one CPU. This really has be confused. To get this far I threw away the original VM and started to create a new one - one with 2 CPUs. I had read somewhere that 2 CPUs is max under the FREE XenServer but I am not sure about this.

Pic follows:

What did I do differently this time around? Other than defining a VM with two CPUs (which did not take) I am not entirely sure. I do remember reading in other posts a need to define a file which defined a pass through file containing a "pass through device" but this did not seem to apply to XenServer. then I cam across the following and I did this on the XenServer host machine.

how to attach your video device to a vm

read and understand the logic in what follows. your values will change.

------cut here------

locate the device you want to attach

lspci -v | grep VGA
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06) (prog-if 00 [VGA controller])
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450] (prog-if 00 [VGA controller])

locate the uuid for the vm you want to attach the device to;

xe vm-list

uuid ( RO) : eb27edce-59e4-49c4-ade4-e0b0aa13a9fd
name-label ( RW): Control domain on host: dhyana
power-state ( RO): running

uuid ( RO) : 7bb323fc-012c-87c2-541e-a32dc82ecd70
name-label ( RW): win7_64
power-state ( RO): running

uuid ( RO) : 81bf0326-08f5-05a0-38fc-73a220ac4e0f
name-label ( RW): Ubuntu12_04
power-state ( RO): running

use the following command to set the pass through
from the above we want to "pass through" to uuid 7bb323fc-012c-87c2-541e-a32dc82ecd70 associated
with the win7_64 VM, a windows box.

xe vm-param-set other-config:pci=0/"PCI ID" uuid="UUID of VM"

xe vm-param-set other-config:pci=0/01:00.0 uuid=7bb323fc-012c-87c2-541e-a32dc82ecd70

-----cut here------

This above is the only thing I can think of that made a difference.

[EDIT] Why no gpu activity in BOINC? Not sure. Will have to investigate. Could it be that I only have one CPU and can't feed the Radeon Card. Or is this Radeon card not suitable for number crunching?

Logforme
Logforme
Joined: 13 Aug 10
Posts: 332
Credit: 1,714,373,961
RAC: 0

Looks like you got as far as

Looks like you got as far as I did. Having the GPU looking ok in the device manager, let's hope yours is actually working. The command line xe commands looks to be doing the same as I did by assigning the GPU to the VM using the XenCenter console.

Only difference between free and subscription xenserver is that the subscribers can get support and do automatic installation of updates. I run VMs with 4 vCPUs on the free version, no problem. I think your problem with only seeing one vCPU in your win 7 VM is that you have the "Home Premium" editon. Home editions seems to be limited to 1 CPU. Check out this wiki.

As for why boinc won't use the GPU. Since a GPU job does require some small CPU part and the one CPU is 100% busy with your CPU job the GPU jobs will have to wait. If you want to test only GPU jobs it might be best to turn off CPU jobs in your preferences.

Holmis
Joined: 4 Jan 05
Posts: 1,118
Credit: 1,055,935,564
RAC: 17,557

RE: As for why boinc won't

Quote:
As for why boinc won't use the GPU. Since a GPU job does require some small CPU part and the one CPU is 100% busy with your CPU job the GPU jobs will have to wait. If you want to test only GPU jobs it might be best to turn off CPU jobs in your preferences.

Don't know if it differs when running in a VM but my understanding of how Boinc schedules work is to first fill the fastest resource with work and that's usually the GPU.
Regarding fractional CPU/GPU usage I seem to remember from somewhere that Boinc schedules GPU task up to and equal to the number of GPUs, for example if you have on GPU it schedules task with fractional GPU usage up to and equal to one but not above, if you set the GPU utilization factor to 0.33 then Boinc will run x3 as 0.33x3=0.99 but if you set 0.34 Boinc will only run x2 as 0.34x3=1.02.
For the CPU Boinc will over commit the CPU if there is a GPU task to run that uses a fraction of the CPU, for example 1 CPU task and a GPU task that uses 0.2 CPU + 1 GPU for a total of 1.2 CPUs used even if there's only one CPU available.

Did Boinc ask for GPU work? What does the event log show? Can't check the server side log as there's been at least one scheduler contact without a work request since the first one.

robl
robl
Joined: 2 Jan 13
Posts: 1,687
Credit: 1,358,131,416
RAC: 734,023

RE: Looks like you got as

Quote:
Looks like you got as far as I did. Having the GPU looking ok in the device manager, let's hope yours is actually working. The command line xe commands looks to be doing the same as I did by assigning the GPU to the VM using the XenCenter console.

Regarding the xe commands I thought that might be the case. But I sure don't understand why that 1st VM I defined did not see the device at all.

What is interesting is that if you "view" my machines on the website it shows this VM as being defined with the Radeon card discussed so I think E&H sees the GPU.

Quote:
Only difference between free and subscription xenserver is that the subscribers can get support and do automatic installation of updates. I run VMs with 4 vCPUs on the free version, no problem. I think your problem with only seeing one vCPU in your win 7 VM is that you have the "Home Premium" editon. Home editions seems to be limited to 1 CPU. Check out this wiki.

Wow. I think your right. I bought the OEM home premium edition. I had no idea that Win distros had such limitations. It was like yesterday when I finally got stuff to work and rebooted the machine and learned I needed to use remote desktop to login. That distro also did not support this requirement.

Quote:
As for why boinc won't use the GPU. Since a GPU job does require some small CPU part and the one CPU is 100% busy with your CPU job the GPU jobs will have to wait. If you want to test only GPU jobs it might be best to turn off CPU jobs in your preferences.

I have "stopped new wus" and will burn off the ones I have. Then I will assign this VM node to a profile that supports only GPUs WUs. See how it handles that.

ExtraTerrestrial Apes
ExtraTerrestria...
Joined: 10 Nov 04
Posts: 770
Credit: 362,795,351
RAC: 698,486

Win 7 Home supports one CPU

Win 7 Home supports one CPU with as many cores as you have, that's why this limitation is hardly known. Enterprise (and probably Pro) support 2 physical CPUs, again with as many threads as you have. I don't know how Xen defines the CPUs - as cores of the same physical CPU (would be OK) or as separate sockets (would unneccesarily limit you to 1).

And Holmis is right, BOINC would schedule work on the GPUs first if possible. Look into the BOINC log upon startup - does it detect the GPU? For CAL and/or OpenCL? If not pass-through is obviously not working. If it does I'd launch GPU-Z and see which APIs are reported as support by the GPU.

MrS

Scanning for our furry friends since Jan 2002

Logforme
Logforme
Joined: 13 Aug 10
Posts: 332
Credit: 1,714,373,961
RAC: 0

RE: Win 7 Home supports one

Quote:
Win 7 Home supports one CPU with as many cores as you have, that's why this limitation is hardly known. Enterprise (and probably Pro) support 2 physical CPUs, again with as many threads as you have. I don't know how Xen defines the CPUs - as cores of the same physical CPU (would be OK) or as separate sockets (would unneccesarily limit you to 1).


Xenserver itself (the hypervisor) sees the actual physical CPUs and cores with hyper-threading. In my case I have a 4 core CPU with hyper-threading, so xenserver reports I have 8 CPUs.
XenServer delivers vCPUs to the VM. A vCPU seems to be a single core CPU with no hyper-threading. So I guess windows just sees a number of single core CPUs with no hyper-threading, checks it's edition, and just uses one core.

robl
robl
Joined: 2 Jan 13
Posts: 1,687
Credit: 1,358,131,416
RAC: 734,023

I have to admit to all of you

I have to admit to all of you that I am feeling a bit like a "pin head". Long story short: Crunching GPU WUs on a Win7 VM is now working as seen in the following screen capture:

Why is it now working? I have always used NVIDIA GPUs in the past and all of my E@H profiles had the "use ATI GPU" checkboxes "unchecked. I was so immersed in the virtualization side that I forgot completely about checking the profiles. It was only in checking the logs (someone had suggested I do this) that I realized GPU work was not being requested.

So I think that we can now say yes it is possible to crunch GPU work in a virtualized world using "pass through".

Aside: I defined an Ubunt12.04 VM yesterday with 7 vCPUs. I am happy to say that they are crunching 7 CPU jobs. There is a correlation of memory allocation to vCPUs. When I defined the vCPUs in the setup I assigned too little memory. That caused 1 CPU WU to run with 6 in a "wait state". Increasing the memory size caused all 7 vCPUs to crunch simultaneously.

Logforme
Logforme
Joined: 13 Aug 10
Posts: 332
Credit: 1,714,373,961
RAC: 0

That is great news!

That is great news! Congratulations.
Do you know how fast the card is outside a VM so you can compare speeds?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.