How to Empty the Queue the Right Way (changing hard drive)
31 Dec 2015 17:07:49 UTC
Topic 198356
(moderation:
Sorry if this is FAQ, but I didn't see it...
I am changing out a hard drive on a machine soon, and set the client for "No New Tasks" and am emptying out the queue of tasks I have downloaded.
If these fail to complete before I must change the hard drive, which is better
1) abort the remaining tasks
2) just let the server figure it out when they time out without returning a result
Does it matter?
How to Empty the Queue the Right Way (changing hard drive)
)
Good that you have set No New Tasks, so are limiting the problem.
If you abort the remaining tasks just before the end, your quorum partners will on average wait a shorter time to get their results compared, which is good.
In your situation I'd say it is better to abort than to let the server figure it out. The thing I don't know is whether there is more server overhead incurred one way than the other, but Einstein seems to generally manage things so that server capacity is adequate to the offered work.
Both your quorum partners and the project will in the end be OK either way.
In special cases such as overclocking experiments, your quorum partner may be eager to get an answer one way or the other, so grateful for the quicker average resolution should you choose to abort.
If you are more ambitious, I believe it is often feasible to back up the BOINC data directories (for example to a memory stick), after shutting down BOINC on the old disk, then by following a correct startup protocol involving first installing BOINC and then copying in the old data to get BOINC going on the new disk without interruption. I think Gary Roberts has posted here with details on how he has successfully done this many times on the large fleet he manages.
Thanks for the great reply.
)
Thanks for the great reply. If I understood the last bit of advice correctly, you are suggesting I could also move my unprocessed work units to the new hard drive to continue processing them, yes?
I used to do that with SETI once-upon-a-time with a machine that was connection-challenged. I think I also did it with BOINC a while back. But I saw a thread here that discouraged me about this.
I am pretty sure everything would work out like this if I cloned my old hard drive onto my new one. I also changed a drive recently (the old one was a complete loss) and it *appeared* that the work units for that same machine re-downloaded automatically.
But in this case, the new drive will have a different OS, so I'm not optimistic about moving the work units, especially since the applications will be different. So I'm trying to minimize the damage.
Thanks again!
If you want to preserve the
)
If you want to preserve the now running tasks it's the other way around as to what archae86 wrote.
Start with backing up the complete data directory, the location is given in the start up messages in Boinc's Event log.
Then when the new disk is in place copy the backup to the desired location and then install Boinc, on the 3rd (I think) screen choose "Advanced" and point the installer to the data directory. This will set the permissions as they should be and Boinc should just pick up where it left off.
I've done this a couple of times without problems, the only thing to look out for is if you're changing the OS it will not work.
RE: ... the only thing to
)
It can still be done even if the OS is different. When I switched from Windows to Linux many years ago, I had no real difficulty taking a running Windows installation and converting it into a Linux installation whilst retaining all the in-progress tasks. I had a lot of machines to do and setting NNT and waiting for caches to drain became quite tedious so I experimented by changing over with tasks on board, some partly crunched. As I recall, nothing went wrong and all partly crunched tasks were valid even though the OS had changed.
If the OP would like to give some details about which current machine and what the new OS is going to be, I'll have a think about what would be the best option.
Cheers,
Gary.
Thank you for the great
)
Thank you for the great replies on this. My intent is to minimize disruption to the project so I am trying to drain the queue and not abort or transfer tasks if I can help it.
Right now the machine is a i7-5820k with a GTX960 video card running Ubuntu Linux. I originally built this machine to turn into a Hackintosh (a homebuilt PC that runs Mac OS X). (It will still be running Einstein@home). I will start with Yosemite (10.10) first to install a few things before upgrading to El Capitan (10.11).
I was going to pull the trigger on this conversion this weekend, since I now have the final pieces to proceed. But I had trouble booting the USB stick, so I haven't proceeded yet.
Fortunately, I didn't do anything to the Linux drive yet, and it is still cranking away. I set the client to keep about a day's work in the queue so I can drain it down more quickly if I succeed in the near future.
RE: Thank you for the great
)
Thank you for your thoughtfulness concerning the project. There is really no problem aborting excess tasks if you need to. They will be sent to someone else straight away with minimum inconvenience to anyone. The real thing to avoid if possible is shutting down without aborting. The project isn't particularly bothered, but the up to 2 week wait before a reissue can bother a 'wingman' waiting for the credit :-).
Nice machine and nice 'fate' for it :-). I'm reasonably familiar with OS X. My daughter's business uses iMacs and Mac minis and I get to advise on networking and sysadmin type stuff. I used to run Einstein on them all (CPU only) but decided not to risk premature failure due to overheating. I quite like OS X because of its unix/FreeBSD heritage.
One thing that did concern me was how much network bandwidth gets chewed with incessant (and largely invisible to the user) communication with Apple. The business is in an area where the distance to the exchange and congestion at the exchange was severely impacting on speeds to the point of making the large internet component of the business unworkable via ADSL. There was a 1TB monthly limit on the ADSL plan which was impossible to ever use.
The business transitioned to a mobile wireless broadband solution (there is a tower 400m away with a direct line-of-sight and not a single structure in between) with a factor of 20 improvement in speeds but a crippling daily usage now that everything could proceed this fast. This was often in the 15-20GB per day range. I setup caching via the improved server app that was available with Yosemite and this reduced the daily consumption to around 3-5GB, pretty much the true business consumption. The plan has a (specially negotiated - they wanted the business) 200GB monthly limit at affordable rates but real penalties after that. So far (quite a few months now) monthly usage has been well within the limit.
A couple of years ago, I toyed with the idea of building some Hackintoshes and did a lot of reading about how to do this. But then sanity returned when I thought through the advantages of the crunching farm running a single OS configuration that had become very comfortable and familiar and very easy to maintain.
Good luck with everything when you do start the job.
A good plan - very considerate of you to think it through.
Cheers,
Gary.
Thanks a lot! I suspected
)
Thanks a lot! I suspected that aborting would result in re-issue quicker, but either way I was afraid of what it could do to the server load or something.
As for the Hackintosh, most of my other machines are Macs, and until a few days ago, I was hoping to use the CPU optimizations for the Mac on the SETI project (SSE4.1, SSSE3, AVX) which run substantially faster than the plain vanilla application. That's overcome by events now with the release of SETI v8, so the urgency has died down a bit. I know some people run similar clients for Linux and Windows (particularly Lunatix(?)) but I didn't want to horse around with that every time a new client came out. And I like Mac OS X.
I was not so keen on Apple hardware with BOINC projects, however. I suspected BOINC was the culprit in a streak of hard drive failures I've had over the years. Until recently I thought it was the heat and played around with overriding fan settings and such. Then I suspected it was preventing the hard drives from sleeping. My BOINC activity dropped off for about a year when I was away and the last machine running BOINC blew a hard drive. That was the worst one to go (my wife's) which led to some...awkwardness and attempts to recover the data (not to mention the downtime).
Then I started loading Einstein on Raspberry Pi2's, thinking I could scale that under the budget RADAR, but quickly realized it wasn't even playing on the same field as regular PCs, regardless of the MIPS/MFLOPs. Which led to this latest project (and the re-seeding of my BOINC farm).