Does RAID make a difference?
2 May 2006 21:16:39 UTC
Topic 191167
(moderation:
I was reading through the "S41.xx Observation Thread" when this post got me thinking. Even though it is not a SCSI controller I have in my computer but an onboard IDE RAID controller I was wondering if it makes any difference in crunching speed.
Does RAID make a difference?
)
Einstein is not disk I/O intensive application, most of it goes in CPU cache and RAM.
CPDN or WCG/FA&H may be a different case - GBS of reads/writes.
It may be worth do some testing - on one machine, I'm running BOINC on SW RAMdrive and hope to have i-RAM soon (it's a SATA disk to system but as a medium RAM modules are used).
Both are VERY fast in term of seek comparing to any HD (even 147GB Seagate Cheetah 15k on SCSI-320 I've tested recently), have constant data throughput, no problem with fragmentation etc.
Not sure about CPU usage of SW Ramdrive comparing to IDE/SATA/SCSI controllers.
RE: Einstein is not disk
)
I agree your first point. But I crunching CPDN and FAAH too and noticed not a heavy disk IO - so I don't belive, that you get more crunching performance by a faster disk or a RAID system.
I think, you would not get better performance with a RAM drive. Depending on your system, it could be (under rare cirucumstances, that a SW-RAM Drive will slow down it. For example you have a small physical memory and a big ramdrive and system needs memory, but now your system goes out of memory and starts with paging out segments to the disk.
Your OS has a buffer cache, in these buffer cache will all the data which would be readen or written to the disks cached, and if your system has time for IO or the buffer cache algorythms wan't it, the data will be written from the buffer cache to the disks.
The size of the WUs and result files are so small and often readen and written, so these data were permanently in the buffer cache. And buffer cache is memory - memory is fast !
But for other tasks, a HW RAM Drive will be cool. But I think, the CPU Usage on the SATA/SCSI/FC differs in the first line from the used controller. If you have an ASIC which handles the IO on the controller, you have these tasks not on your CPU.
Good discussion,
)
Good discussion, Dotsch.
Yes, system cache may speed-up file access.
How about fragmentation? HDs get pretty fragmented after couple of weeks/months running CPDN (or F&H).
I'm running system with 4GB RAM (and 2 x 300 GBs HD), so having large SW RAMDrive and low system memory is now my case.
All 32-bit Windows system are poor at memory management and actually can't use whole 4GB (you get stucked at about 3.3GB).
So, I decided to have ~1GB SW RAM drive, where I run swap file, temporary files from web browser, printer spooler (this makes GBs/day as I'm working with DTP), temp for zip files etc. All of those are aplication where (i) HD seek is intersive and/or (ii) large data are read/written.
This combination revealed to be VERY fast - much faster than single 15k SCSI-320 HD.
Perhaps it's not as good for BOINC, but overall system performance is MUCH better.
I'm planning to run BOINC from 4GB i-RAM so that HDs can sleep when I'm not working on computer. This makes the heat and not much lower as well.
So, overall performance, noise and heat can be somehow solved using SW-RAM Drive or i-RAM. Maybe we should count BOINC on (i)-RAM/Drive a side-effect benefit.
RE: I'm running system with
)
If I understand it correctly, you can't use the last 0.7 GB RAM, right?
Wouldn't it be wiser to switch off swapping completely and make the SW RAM drive a little bit smaller (for approx. the size of former swap file)?
Peter
RE: If I understand it
)
Yes, at least that...depends how one counts 4GB - as a 4.192 MBs?
Unfortunatelly not, Peter.
Windows can't run correctly without swap file - they are desinged to paging memory. More memory doesn't help - not only due to limitations of capacity to address. Standard setting it that swap file is 1.5x size of RAM. If Windows are stupid enought to make a hibernation file as well, you get 6GB swap file + 4GB for hibernation. Funny, heh?
It's not the way: more RAM, larger part of OS in memory as commond sense may suggest.
Apart from that, some appication yell if not swap file present. I may pick Adobe Photoshop among them. More then that - Photoshop needs a scratch file (kind of paging memory/file technique)...but it is not recommend to be on the same drive as swap file (another yelling...ehe, warming window).
So, I substitued a temp folder on i-RAM to make it different drive to OS (yeah, the old-good command-line apps back from DOS).
Now, Windows and Photoshop using same physical drive fro swap/scratch without warnings.
Sure, it would be 1000 times better is 64-bit OS wouldn't use swap when not needed and 64-bit version of Photoshop was on marked...none of it happened so far and doesn't seem to happen in near future.
Sorry for a bit OT.
My obvious point is - system and applcations are running faster if in memory. If noet, put them to a SW RAMDisk or an i-RAM. Same apply for BOINC. Regular backups are on place not only for CPDN of course...
Happy crunching :-)
(gotta hack BOINC to gimme more work from Einstein - would 3-CPUs on dual-core do the job...?)
RE: RE: Wouldn't it be
)
Fortunately yes, Honzo ;-) Win2K was very near, reqired at least 2MB swap. WinXP (dunno about 64-bit versions, apologize) has third possibility in Virtual Memory settings dialog - No paging file. (Not aware of it? Give it a try.) And it also works and pretty fast (without the overhead of swapping memory into memory). Few years ago, I was running few weeks this way, until various apps (including mentioned Photoshop, which yelled on startup, but worked :) could not fit in 1 GB RAM without additional swap.
i-RAM .... mmmmmmm (mnam lahodka) must be very fine thing...
:-) Sure would! For the hack, DIY or wake up trux (he is somehow busy last days).
Have fun
Peter
[edit] I see one point - with external 'SSD'-drive you can overcome the 3.3 GB limit and expand the RAM (through RAM-swap) by nearly another 4 or 8 or ... GB.
I'm aware of possibility of
)
I'm aware of possibility of turning-off swap file via GUI; it's also possible via registry editing. Win XP x64 has the same GUI as old Win XP - just includes SP2 functions. [to be more precise, Win XP SP1 actually was able to use whole 4GB; I was thinking to reinstall from my original CD which has only SP1 integrated...but some application refuse to run if SP2 is not present {to have more fun, some aplications doesn't want to run on SP2 without pathches or new app version}].
Concept of memory management ans swap-file is pretty much the same in x64 I guess same for 2K3 server)...but it breaks 3.3GB barrier. (it may be still a problem for some apps - for example I was unable to run Half-Life 2 under Win XP x64 with 4 GB RAM and got 'not enought memory error' (with a MINUS value; buffer overflaw?). Limiting /maxmem in boot.ini did the job).
mentioning boot.ini - it may be tempting to play with /PAE switch to break memory limit or /3GB switch to let Photoshop use 3GB (instead of regular 2GB per any 32-bit application)...performance loss was enormouns.
Perhaps I'll got x64 way once there is 64-bit BOINC app(s) for that (no problem with running 32-bit BOINC). But so far, akofs shows that there are still ways to make 32-bit more effecient than imagined!
With swap-file off, I have seen some issues couple years back.
Instead, it served good to have low values like 250MB.
No, it somehow solved with i-RAM. Yeah, very fine. But it's shame that one needs to adjust to Windows and not other way around (swap-file concept with even with huge amount of memory).
Will try to play a bit more with Wnidows setting when I install them on i-RAM; got a trial installation: takes about 4 sec from 'starting windows' to desktop from Win XP SP2 (no sounds and videa card drviers included...they may slow booting process).
But this must wait (later this months?) until I finish some top priority commintments...
I get much the same result by
)
I get much the same result by locking the size of the pagefile/swapfile and setting conservitive swapfile use in the registry.
98SE XP2500+ @ 2.1 GHz Boinc v5.8.8
