I'm responding to a message I posted 2 days ago (in this thread) to get some data from it. Use the 'in response to' link if you can't find the original message.
This new message contains a table I will update daily at about this time (unless I get distracted). The data comes from the server status page as at the listed time. It might be interesting to see on a daily basis how the distribution of GW tasks is going. This should work since ALL the remaining tasks for O1AST (Observation 1 All Sky Tuning) are in the database and the change in the 'Tasks to send' figure should tell us about the rate at which new hosts are being added to (or removed from) the run. If something 'really big' gets assigned/removed, it should be pretty obvious :-).
Date and Time UTC Status page item O1AS20-100T Change Comment
16 Feb 2016, 20:48:31 Tasks total 746,736
16 Feb 2016, 20:48:31 Tasks to send 741,997 0
17 Feb 2016, 21:05:01 Tasks to send 738,475 -3552 Initial cache fillup on hosts
18 Feb 2016, 21:35:02 Tasks to send 736,549 -1926 Hmmm.. these tasks chew sloooow
If there is ongoing interest in this, I could easily start a new thread and sticky it so it would be easier to find.
2016-02-18 22:48:49.1085 [PID=15715] Request: [USER#xxxxx] [HOST#11905468] [IP xxx.xxx.xxx.191] client 7.6.22
...
2016-02-18 22:48:49.4536 [PID=15715] [mixed] sending locality work second
2016-02-18 22:48:49.7973 [PID=15715] [version] Checking plan class 'AVX'
2016-02-18 22:48:49.7996 [PID=15715] [version] reading plan classes from file '/BOINC/projects/EinsteinAtHome/plan_class_spec.xml'
2016-02-18 22:48:49.7997 [PID=15715] [version] CPU [' fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr
sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu
pni pclmulqdq d family 6 model 60 stepping 3 '] lacks feature ' avx '
note the last line - it looks like boinc is not sending the correct CPU capabilities string as this part of the string is missing (avx of course in this part)
It must be what Juha said in his earlier answer. I've got other things going on so I haven't been able to respond until now (I'm taking a lunch break) :-).
The Linux version of BOINC is probably reporting the 'flags' line from /proc/cpuinfo. On my host there is no 'avx' on that line. So a BOINC upgrade won't fix it. I decided to try one of my i3-4160s and, lo and behold, they do have avx and avx2. I was puzzled by this so I did a bit of googling and found this on the Arch Linux forums (note same processor as mine G3258)
QUESTION:
My question is, why Haswell optimized linux-ck doesn't work on Haswell CPU - Intel G3258?
ANSWER:
Despite being labeled as such, it doesn't have the full Haswell feature set. Among other things, AVX and AVX2 are missing.
If the compiler generated code which uses AVX/AVX2 instructions, their lack can definitely cause the kernel not to boot.
So it would seem that Haswell Pentium dual core processors have AVX/AVX2 instructions deliberately disabled by Intel. I'll need to add an i3 to the test to see if that will finally get the AVX app :-).
Returning to why you don't get the AVX app with your i5-4690K. You show the example of the full string BOINC is sending as compared to the truncated string reported in the scheduler log. To me this looks like exactly that - a server-side bug causing the string to be truncated so that avx and avx2 are not being seen. That doesn't explain that seems a real puzzle. Why did the other host you linked to get the AVX app. I checked and it's what appears to be a older 8 core Xeon. Its latest contact resulted in getting FGRP CPU work so there was no test for avx. I'm just wondering if the capabilities string might be shorter for that CPU so there may not have been a truncation??
Of course this is all just conjecture which Bernd may be able to comment on.
The only other point of difference is that you are running kernel 3.13.0 whereas the other example you linked to was running 4.2.0. Could kernel version have anything to do with it?
I'll need to add an i3 to the test to see if that will finally get the AVX app :-).
I realize my answer does not answer the Linux/Einstein AVX question, but I do have a dual-core Haswell i3 host i3-4130 to be exact, running Windows 7 and BOINC 7.6.22. Early on it got a little SSE2 work, perhaps because the SSE2 flavor of v1.03 was released a little sooner than AVX, but it has been getting (and successfully processing) AVX flavor since 14 Feb 2016, 18:35:59 UTC with a nice little performance improvement.
I had spotted the remarkably low prices of the Pentium-flavored Haswells, and had not really seen a substantial reason to prefer the i3 for purposes of possible interest to me. Not having AVX, which is affirmed if one compares the official Intel summary characteristics site for the i3-4130 to the Pentium G3258 is more of a reason than I had previously known, though quite likely the deficiency would not hurt a system intended as a GPU host.
I have now added an i3-4130 + HD7850 GPU to the testing of O1AST. It got the non-AVX app.
To make sure of what was being sent to the server, I looked at the tag in the scheduler request rather than looking at the event log. Both avx and avx2 were present in the full string which is actually 522 characters in length. On consulting the scheduler log on the website, I can see exactly the same truncation that AgentB reported earlier. The truncated string (as shown by AgentB and as seen in my log entry) just happens to have exactly 256 characters. That seems to be a highly suspicious number :-).
Of course maybe the truncation is just for display purposes in the log output, but if so, it seems rather strange that the AVX capability in the scheduler request isn't being seen.
I'm thinking about ways to get avx and avx2 into earlier positions in the string. I could try stopping BOINC and editing the line in the state file to shift avx and avx2 to positions earlier than 256. Of course, I could just be wasting my time - BOINC may very well refresh all this stuff at startup rather than simply believing what was there when it was last shut down. An even less likely option would be to see if a scheduler request can be captured after it is created but before it is able to be sent to the servers - pull the network cable at the same time as triggering a work request :-). Then edit the scheduler request and plug the cable back in. When BOINC retries, will it create a new request or will it use the one it had created just before it noticed there was no network? :-).
On second thoughts, maybe I'll just hope Bernd fixes this soonish :-).
I remember that AVX detection was a little difficult on Windows and it took Rom several tries to get it right. So it is possible to have to use a version 7.4.x on Windows for BOINC to detect the AVX capability.
The truncation issue seems something we can fix on our side. I'll take a look into that shortly.
... though quite likely the deficiency would not hurt a system intended as a GPU host.
Yes, this is true and I guess it's a small consolation. I have Pentium Dual Cores from Sandy Bridge, Ivy Bridge and Haswell and all of them lack AVX - now that I know where it is easiest to look to see it :-). I'm quite happy with their ability to drive GPUs.
I also have i3s from all the above so as soon as the AVX detection for Linux gets fixed, I'll be able to see what difference it makes.
The truncation issue should be fixed now. I now see longer feature strings in the logfile where hosts are missing AVX. You should get work now if your CPU does support AVX.
I had to come home before your reply arrived but I have an Ivy Bridge i3 (i3-3240) here so I just added it to those doing the test. It's now running a task with the 64bit AVX app. There don't appear to be any issues.
The machine has a 4 day cache so there's quite a bit of FGRPB1 work to complete before it starts running all O1AST tasks. After allowing it to do the GW run, I triggered a small work fetch to get a single task. I was then going to suspend all the FGRPB1 tasks to get the new one to start but BOINC paused one of the running FGRPB1s and started the new one automatically. That was kind of BOINC to know what my intentions were :-).
I guess BOINC made the decision that with a 4 day cache and a 7 day deadline, the new task needed starting immediately. The machine does 4xBRP6 and 2xCPU tasks concurrently.
I'm responding to a message I
)
I'm responding to a message I posted 2 days ago (in this thread) to get some data from it. Use the 'in response to' link if you can't find the original message.
This new message contains a table I will update daily at about this time (unless I get distracted). The data comes from the server status page as at the listed time. It might be interesting to see on a daily basis how the distribution of GW tasks is going. This should work since ALL the remaining tasks for O1AST (Observation 1 All Sky Tuning) are in the database and the change in the 'Tasks to send' figure should tell us about the rate at which new hosts are being added to (or removed from) the run. If something 'really big' gets assigned/removed, it should be pretty obvious :-).
If there is ongoing interest in this, I could easily start a new thread and sticky it so it would be easier to find.
Cheers,
Gary.
RE: I guess it's got to be
)
Hmm...maybe - we're not there yet!.
I was not getting anything AVX on 7.2.42 so just upgraded to 7.6.22
still no AVX.
Eventlog (with some editing to highlight)
Thu 18 Feb 2016 22:26:44 GMT | | Starting BOINC client version 7.6.22 for x86_64-pc-linux-gnu Thu 18 Feb 2016 22:26:44 GMT | | log flags: file_xfer, sched_ops, task, coproc_debug Thu 18 Feb 2016 22:26:44 GMT | | Libraries: libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3 Thu 18 Feb 2016 22:26:44 GMT | | Data directory: /var/lib/boinc-client Thu 18 Feb 2016 22:26:44 GMT | | OpenCL: AMD/ATI GPU 0: Tahiti (driver version 1702.3 (VM), device version OpenCL 1.2 AMD-APP (1702.3), 2763MB, 2763MB available, 2688 GFLOPS peak) Thu 18 Feb 2016 22:26:44 GMT | | OpenCL: AMD/ATI GPU 1: Tahiti (driver version 1702.3 (VM), device version OpenCL 1.2 AMD-APP (1702.3), 3035MB, 3035MB available, 2688 GFLOPS peak) Thu 18 Feb 2016 22:26:44 GMT | | OpenCL CPU: Intel(R) Core(TM) i5-4690K CPU @ 3.50GHz (OpenCL driver vendor: Advanced Micro Devices, Inc., driver version 1702.3 (sse2,avx), device version OpenCL 1.2 AMD-APP (1702.3)) Thu 18 Feb 2016 22:26:44 GMT | | Host name: gluon Thu 18 Feb 2016 22:26:44 GMT | | Processor: 4 GenuineIntel Intel(R) Core(TM) i5-4690K CPU @ 3.50GHz [Family 6 Model 60 Stepping 3] Thu 18 Feb 2016 22:26:44 GMT | | Processor features: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm Thu 18 Feb 2016 22:26:44 GMT | | OS: Linux: 3.13.0-77-genericAsked for some AVX CPU tasks the scheduler request shows
2016-02-18 22:48:49.1085 [PID=15715] Request: [USER#xxxxx] [HOST#11905468] [IP xxx.xxx.xxx.191] client 7.6.22 ... 2016-02-18 22:48:49.4536 [PID=15715] [mixed] sending locality work second 2016-02-18 22:48:49.7973 [PID=15715] [version] Checking plan class 'AVX' 2016-02-18 22:48:49.7996 [PID=15715] [version] reading plan classes from file '/BOINC/projects/EinsteinAtHome/plan_class_spec.xml' 2016-02-18 22:48:49.7997 [PID=15715] [version] CPU [' fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq d family 6 model 60 stepping 3 '] lacks feature ' avx 'note the last line - it looks like boinc is not sending the correct CPU capabilities string as this part of the string is missing (avx of course in this part)
and replaced with
Thoughts?
RE: Thoughts? It must
)
It must be what Juha said in his earlier answer. I've got other things going on so I haven't been able to respond until now (I'm taking a lunch break) :-).
The Linux version of BOINC is probably reporting the 'flags' line from /proc/cpuinfo. On my host there is no 'avx' on that line. So a BOINC upgrade won't fix it. I decided to try one of my i3-4160s and, lo and behold, they do have avx and avx2. I was puzzled by this so I did a bit of googling and found this on the Arch Linux forums (note same processor as mine G3258)
QUESTION: My question is, why Haswell optimized linux-ck doesn't work on Haswell CPU - Intel G3258?ANSWER:
Despite being labeled as such, it doesn't have the full Haswell feature set. Among other things, AVX and AVX2 are missing.
If the compiler generated code which uses AVX/AVX2 instructions, their lack can definitely cause the kernel not to boot.
So it would seem that Haswell Pentium dual core processors have AVX/AVX2 instructions deliberately disabled by Intel. I'll need to add an i3 to the test to see if that will finally get the AVX app :-).
Returning to why you don't get the AVX app with your i5-4690K. You show the example of the full string BOINC is sending as compared to the truncated string reported in the scheduler log. To me this looks like exactly that - a server-side bug causing the string to be truncated so that avx and avx2 are not being seen. That doesn't explain that seems a real puzzle. Why did the other host you linked to get the AVX app. I checked and it's what appears to be a older 8 core Xeon. Its latest contact resulted in getting FGRP CPU work so there was no test for avx. I'm just wondering if the capabilities string might be shorter for that CPU so there may not have been a truncation??
Of course this is all just conjecture which Bernd may be able to comment on.
The only other point of difference is that you are running kernel 3.13.0 whereas the other example you linked to was running 4.2.0. Could kernel version have anything to do with it?
Cheers,
Gary.
RE: I'll need to add an i3
)
I realize my answer does not answer the Linux/Einstein AVX question, but I do have a dual-core Haswell i3 host i3-4130 to be exact, running Windows 7 and BOINC 7.6.22. Early on it got a little SSE2 work, perhaps because the SSE2 flavor of v1.03 was released a little sooner than AVX, but it has been getting (and successfully processing) AVX flavor since 14 Feb 2016, 18:35:59 UTC with a nice little performance improvement.
I had spotted the remarkably low prices of the Pentium-flavored Haswells, and had not really seen a substantial reason to prefer the i3 for purposes of possible interest to me. Not having AVX, which is affirmed if one compares the official Intel summary characteristics site for the i3-4130 to the Pentium G3258 is more of a reason than I had previously known, though quite likely the deficiency would not hurt a system intended as a GPU host.
I have now added an i3-4130 +
)
I have now added an i3-4130 + HD7850 GPU to the testing of O1AST. It got the non-AVX app.
To make sure of what was being sent to the server, I looked at the tag in the scheduler request rather than looking at the event log. Both avx and avx2 were present in the full string which is actually 522 characters in length. On consulting the scheduler log on the website, I can see exactly the same truncation that AgentB reported earlier. The truncated string (as shown by AgentB and as seen in my log entry) just happens to have exactly 256 characters. That seems to be a highly suspicious number :-).
Of course maybe the truncation is just for display purposes in the log output, but if so, it seems rather strange that the AVX capability in the scheduler request isn't being seen.
I'm thinking about ways to get avx and avx2 into earlier positions in the string. I could try stopping BOINC and editing the line in the state file to shift avx and avx2 to positions earlier than 256. Of course, I could just be wasting my time - BOINC may very well refresh all this stuff at startup rather than simply believing what was there when it was last shut down. An even less likely option would be to see if a scheduler request can be captured after it is created but before it is able to be sent to the servers - pull the network cable at the same time as triggering a work request :-). Then edit the scheduler request and plug the cable back in. When BOINC retries, will it create a new request or will it use the one it had created just before it noticed there was no network? :-).
On second thoughts, maybe I'll just hope Bernd fixes this soonish :-).
Cheers,
Gary.
Two thoughts: I remember
)
Two thoughts:
I remember that AVX detection was a little difficult on Windows and it took Rom several tries to get it right. So it is possible to have to use a version 7.4.x on Windows for BOINC to detect the AVX capability.
The truncation issue seems something we can fix on our side. I'll take a look into that shortly.
Thanks. Bernd is already
)
Thanks. Bernd is already looking at it - posted in Tech News.
Cheers,
Gary.
RE: ... though quite likely
)
Yes, this is true and I guess it's a small consolation. I have Pentium Dual Cores from Sandy Bridge, Ivy Bridge and Haswell and all of them lack AVX - now that I know where it is easiest to look to see it :-). I'm quite happy with their ability to drive GPUs.
I also have i3s from all the above so as soon as the AVX detection for Linux gets fixed, I'll be able to see what difference it makes.
Cheers,
Gary.
The truncation issue should
)
The truncation issue should be fixed now. I now see longer feature strings in the logfile where hosts are missing AVX. You should get work now if your CPU does support AVX.
Thank you very much. I had
)
Thank you very much.
I had to come home before your reply arrived but I have an Ivy Bridge i3 (i3-3240) here so I just added it to those doing the test. It's now running a task with the 64bit AVX app. There don't appear to be any issues.
The machine has a 4 day cache so there's quite a bit of FGRPB1 work to complete before it starts running all O1AST tasks. After allowing it to do the GW run, I triggered a small work fetch to get a single task. I was then going to suspend all the FGRPB1 tasks to get the new one to start but BOINC paused one of the running FGRPB1s and started the new one automatically. That was kind of BOINC to know what my intentions were :-).
I guess BOINC made the decision that with a 4 day cache and a 7 day deadline, the new task needed starting immediately. The machine does 4xBRP6 and 2xCPU tasks concurrently.
Cheers,
Gary.