Amazing Numbers of New Hosts
30 Jul 2008 7:33:27 UTC
Topic 193788
(moderation:
The project is now past the 99% complete mark with little more than 1 days primary work left.
With that in mind why would there be over 4000 new hosts added in the last 24 hours? According to the current server status page, there were 179 new users in the last 24 hours, most of which probably had 1 or 2 hosts. Even if you were generous and said that every new user had 5 hosts, that still leaves well over 3000 unaccounted for.
To my way of thinking, this probably serves to add weight to my earlier thoughts that there has been a very big conversion of Windows hosts to Linux and that this is still continuing at a fairly frenetic pace. The conversion of hosts in bulk like this automatically generates new hostIDs and so would seem to account for the statistic we are seeing. Of course, it could have just been several clusters of new hosts being added for S5R4 but this trend of large numbers of new hostIDs has been going on for a while now.
The question is why is it being done now rather than three months ago? The obvious conclusion is that the new apps for S5R4 will continue to perform significantly better under Linux. If that's the case, I'm glad I've converted the bulk of my hosts already.
I can take a host crunching under Windows and in around 15 minutes (sometimes less) it is firing up BOINC and downloading tasks and data under Linux. I have a single remastered CD which is loaded with a complete BOINC template so that, irrespective of what the hardware actually is, the only configuration I actually have to do during the install is to type in the hostname I want to use for this particular new host. Everything else is automatic or simply OK'ing a default (eg US keyboard, or use the available spare space on the HDD, etc) that is presented.
I prepared the CD on a desktop machine with a Gigabyte mobo with integrated graphics and IDE drives and recently I've used it to convert servers with serverworks dual processor mobos, SCSI HDDs and smart array controllers, without any issues. The different hardware all gets correctly detected during installation and since all my particular customisations are built into the remastered CD, there is no after installation configuration to be done other than setting the hostname. So as the installation CD is ejected and the new system boots from the HDD, all I have to do is set the hostname and click the "StartBoinc" icon that is sitting there on the desktop. The only annoying bit is that after the new host downloads a bunch of large data files and its very first task, I have to come back at a later stage and click the "StopBoinc" icon so I can edit the DCF from 1.000000 to 0.250000 so that the time completion estimates for tasks are at least in the ballpark :-).
Whilst I do hope the Windows app will be soon on a par with the Linux one, I also hope it never becomes significantly better. I don't want to be even slightly tempted into considering going back to crunching under Windows :-).
Cheers,
Gary.
Amazing Numbers of New Hosts
)
I haven't looked into the DB yet, but my guess is that a large part of these 'new hosts' comes from the 'e-science GRID', where every node of every cluster they submit an Einstein@home job on is potentially recorded as a new host at the server side (currently this 'participant' has ~250.000 hosts registered).
BM
BM
RE: ... every node of every
)
OK, thanks for the explanation. It seems "inefficient" (maybe it's not) to have what must be only a relatively small number of thousands of hosts turn into 250,000 registrations. You would have thought there would be some way to "reuse" the previous registration and keep that seemingly unwieldy number down to more manageable proportions.
Cheers,
Gary.
RE: RE: ... every node of
)
I know that they now have their own management of hostids, but apparently this didn't work always as it should (remember the monster-host?).
BM
BM
So one alternative would be
)
So one alternative would be to "somehow" mark those cluster computers so that they can be treated differently, e.g. periodically removed from the database (forced detaching) and therefore the statistics after a relatively short period of inactivity (sort of garbage collection).
Gary already mentioned somewhere else that even the scheduler might want to treat those short lived cluster nodes differently, as they are unlikely to compute long sequences of WUs from the same series if they are re-incarnated all over again many times, leading probably to excessive "fragmentation" of the WU pool.
CU
Bikeman
RE: ... apparently this
)
Yes, I certainly do :-).
I didn't really understand it at the time and I wasn't inclined to delve any further when it seemed to be some sort of "own goal" :-).
Makes a bit more sense now ...
Cheers,
Gary.