Einstein on self compiled Linux

Gray Handcock
Gray Handcock
Joined: 11 Mar 05
Posts: 211
Credit: 135567
RAC: 0

Hello Bert I got a nice

Hello Bert

I got a nice suprise recently: I have always set

nano -w /etc/portage/package.keywords

to

media-video/nvidia-glx ~x86
media-video/nvidia-kernel ~x86
sci-misc/boinc ~x86

(nvidia stuff done like that 'cos it works out less hassle for me...)

I had to do this to get Gentoo to install BOINC correctly using Portage - in the past I used the sh command in my home directory, which works fine, but well gotta tweak... grin

anyhow Gentoo now has BOINC ~x86 at 5.5.6

also you might want to give fluxbox a whirl - not as user friendly as KDE but less to compile and a lot faster as long as you not in the habit of moving files every couple seconds (nano -w .fluxbox/menu in your home directory to add stuff to the "start" menu)

cheers - Gray

PS: being something of a control freak, all I have set to cron is a rootkit hunter and slocate - everything else I prefer to do manually :)

Lt. Cmdr. Daze
Lt. Cmdr. Daze
Joined: 19 Apr 06
Posts: 756
Credit: 82361
RAC: 0

Hi

Message 41915 in response to message 41912

Hi Gray,

Quote:
USE="-3dfx -3dnow -Xaw3d X -a52 aac aalib -accessibility -acl -afs -aim akode
alsa -altivec -apache -apache2 -apm -arts asf -audiofile bash-completion bcmath -bidi -birdstep -bluetooth bzip2 -canna -caps -cdb -chasen -cjk -clamav -crypt -cscope -cups -db2 -debug dri -dv -dvb -dvdr -eds -emacs -emacs-w3 -empress -empress-bcs esd ffmpeg -foomaticdb gif -gnome -gphoto2 gpm -gps -gstreamer -gtk -gtk2 hal -ieee1394 -imagemagick -ipv6 -jabber javascript -joystick jpeg -junit kde libwww mad -mng -mono mp3 mpeg -msession -msn -mule -nas -netboot -neXt ncurses -ocaml -oci8 -ofx -oss -pcmia -pda ogg opengl pdf png qt4 readline -scanner sdl -slp -smartcard -speex spell sse2 svg svga symlink tiff truetype unicode -v4l vcd vorbis -wifi win32codecs xine xml xmms -xosd -xprint xv xvid -yahoo"


So this is the weekend of the big transition. At the moment I'm looking at the USE variables. In "use.desc", I found a variable fftw, to "use FFTW library for computing Fourier transforms". That description looks like a good variable to set as, AFAIK, these transforms are used. But I didn't spot that one in your USE list. Does that mean it is not used?

BTW: I already like what I see. I moved from windows to Debian to have a better control of my OS, and so far Gentoo even improves that. Definitely worth the time!

Thanks and regards,
Bert

Somnio ergo sum

Gray Handcock
Gray Handcock
Joined: 11 Mar 05
Posts: 211
Credit: 135567
RAC: 0

RE: Hi

Message 41916 in response to message 41915

Quote:

Hi Gray,

Quote:
USE="-3dfx -3dnow -Xaw3d X -a52 aac aalib -accessibility -acl -afs -aim akode
alsa -altivec -apache -apache2 -apm -arts asf -audiofile bash-completion bcmath -bidi -birdstep -bluetooth bzip2 -canna -caps -cdb -chasen -cjk -clamav -crypt -cscope -cups -db2 -debug dri -dv -dvb -dvdr -eds -emacs -emacs-w3 -empress -empress-bcs esd ffmpeg -foomaticdb gif -gnome -gphoto2 gpm -gps -gstreamer -gtk -gtk2 hal -ieee1394 -imagemagick -ipv6 -jabber javascript -joystick jpeg -junit kde libwww mad -mng -mono mp3 mpeg -msession -msn -mule -nas -netboot -neXt ncurses -ocaml -oci8 -ofx -oss -pcmia -pda ogg opengl pdf png qt4 readline -scanner sdl -slp -smartcard -speex spell sse2 svg svga symlink tiff truetype unicode -v4l vcd vorbis -wifi win32codecs xine xml xmms -xosd -xprint xv xvid -yahoo"

So this is the weekend of the big transition. At the moment I'm looking at the USE variables. In "use.desc", I found a variable fftw, to "use FFTW library for computing Fourier transforms". That description looks like a good variable to set as, AFAIK, these transforms are used. But I didn't spot that one in your USE list. Does that mean it is not used?

BTW: I already like what I see. I moved from windows to Debian to have a better control of my OS, and so far Gentoo even improves that. Definitely worth the time!

Thanks and regards,
Bert

Hello Bert

Well it is Sunday evening - haven't been back here for quite a while: sorry, as by now chances are you have system built already...

I must admit I have not tried that one as yet - I've been fiddling around quite a bit doing "emerge -e --update --newuse --deep world" with each change. yup very patient, me !! Thus far I have gone back to what "might" be regarded as a sensible make.conf - see below:

CFLAGS="-O3 -march=pentium4 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
USE="alsa -cups -debug -gnome -kde ncurses nptl opengl qt4 sse2 symlink"
INPUT_DEVICES="keyboard mouse"
VIDEO_CARDS="nv"

There is quite a bit in the Gentoo Forums regarding CFLAGS and LDFLAGS, and the one I have chosen seem to be the ones least likely to cause issues - certainly I have had none so far.

I have been doing multiple tests using the BOINC benchmark - not valid for Einstein these days, but useful for tweaking :) - and the compiled version of BOINC plus associated libraries, definitely outdoes the sh version sitting in the user's home directory on my system.

My system is quite clean anyway here - removed xmms and music/video/e-mail apps and the dependencies for them (the plan is to run Einstein while I am at work - so no need), hence the few use keywords - it is more a matter of a couple percent increase here and there (grin).

Cheers - Gray

Lt. Cmdr. Daze
Lt. Cmdr. Daze
Joined: 19 Apr 06
Posts: 756
Credit: 82361
RAC: 0

Hi Gray! Just enjoy your

Hi Gray!

Just enjoy your weekend, I just HAD to spend the weekend on Gentoo. I actually did include it in USE.

Everything is running smoothly as far as I can see. Just started running boinc again (lost my old WU's though..). First benchmark run turned out to be 5 to 10 % worse than on Debian. However, the completion time, as far as it is viable to estimate it in under 5% done, seems to be well under 12h, where Debian had over 17h. But that's probably because of the different WU's as well: I have long WU's from l1 now, where I used to have long WUs from h1.

My make.conf:

CFLAGS="-O2 -march=pentium4 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j2"

USE="alsa avi bash-completion bcmath -bluetooth bzip2 -cups -debug dvd -emacs -emacs-w3 ffmpeg fftw gif gnome gpm hal java javascript -joystick jpeg -kde mmx mozilla mp3 mpeg -msn -mule -netboo opengl pcmcia pdf png quicktime samba -scanner -smartcard sse sse2 svg svga symlink tetex tiff truetype unicode usb wifi win32codecs xine xml xmms xvid -yahoo"

INPUT_DEVICES="keyboard mouse"
VIDEO_CARDS="i810"

But, although it's mainly crunching ;), I do use it for my study as well..

Again, many thanks for your advice.

Regards,
Bert

Somnio ergo sum

Gray Handcock
Gray Handcock
Joined: 11 Mar 05
Posts: 211
Credit: 135567
RAC: 0

RE: Hi Gray! Just enjoy

Message 41918 in response to message 41917

Quote:

Hi Gray!

Just enjoy your weekend, I just HAD to spend the weekend on Gentoo. I actually did include it in USE.

Everything is running smoothly as far as I can see. Just started running boinc again (lost my old WU's though..). First benchmark run turned out to be 5 to 10 % worse than on Debian. However, the completion time, as far as it is viable to estimate it in under 5% done, seems to be well under 12h, where Debian had over 17h. But that's probably because of the different WU's as well: I have long WU's from l1 now, where I used to have long WUs from h1.

My make.conf:

CFLAGS="-O2 -march=pentium4 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
MAKEOPTS="-j2"

USE="alsa avi bash-completion bcmath -bluetooth bzip2 -cups -debug dvd -emacs -emacs-w3 ffmpeg fftw gif gnome gpm hal java javascript -joystick jpeg -kde mmx mozilla mp3 mpeg -msn -mule -netboo opengl pcmcia pdf png quicktime samba -scanner -smartcard sse sse2 svg svga symlink tetex tiff truetype unicode usb wifi win32codecs xine xml xmms xvid -yahoo"

INPUT_DEVICES="keyboard mouse"
VIDEO_CARDS="i810"

But, although it's mainly crunching ;), I do use it for my study as well..

Again, many thanks for your advice.

Regards,
Bert

Hiya Bert

Congrats on getting all working - now the trick is to not screw it up by tweaking (and this IS Gray talking here - grin - any of my friends seeing this can just quietly pick themselves up off the floor....)

You might want to add the use keyword "mad" - for any mp3s (I see you have indicated you want mp3 sounds. You can update this by "emerge -pv --update --newuse --deep world" and check to see if it has been picked up. If all is OK then just "emerge --update --newuse --deep world" to get the new stuff installed.

From the docs online:

mad - Adds support for mad (high-quality mp3 decoder library and cli frontend)

I have dropped in your suggestion to the use keywords - hehe, maybe it'll give me that so very important extra percentage in the benchmarks - grin. Busy doing yet another "emerge -e --update --newuse --deep world" as I type this: only another 250 progs or so to go - lucky I'm on leave, tho I doubt my cpu has ever worked this hard...

Once that is done I'm gonna continue crunching Einstein again for a while to let things settle down (and I get my nerve back - maybe some social life too), while I do some more research, and maybe rejig my rather out-of-date website with Bluefish. I assume you turned on hdparm by the way ?

Just by way of comparison, timewise, I usually finish a big WU in or around 9 hours 45 minutes, so I guess you could do a large Einstein around maybe 10 hours or thereabout.

Good luck with the sparkly new system - Gray

Lt. Cmdr. Daze
Lt. Cmdr. Daze
Joined: 19 Apr 06
Posts: 756
Credit: 82361
RAC: 0

Yeah, I've read about hdparm.

Yeah, I've read about hdparm. But not implemented yet. I didn't have the impression it would make such a big difference. After your post, I gathered more info. Definitely worth a try!

I also see in your USE list the compilation flag LDFLAGS. A thread on Gentoo suggest that -z would give better runtime performance at the expense of a slower startup and possibly more memory usage (you probably already read it). Looks like a future step as well. But I'd first like to get a bit more familiar with these flags.

I still think it's a bit strange that running benchmarking suggests that Debian would be a better choice (4 to 14 % actually), but my WU finished in 11h37m, so that would be an improvement of around 30%. Apparently too many variables are changed to have a good comparison of the boinc benchmarks (other OS, other BOINC version, other WUs). Do you also use some other benchmarking (I read you use BOINC benchmarking as well)?

BTW: I used the source from boinc, as the ebuild is masked.

Quote:
and the compiled version of BOINC plus associated libraries, definitely outdoes the sh version sitting in the user's home directory on my system.


Do you use another source?

Regards,
Bert

(edit: typing flaws)

Somnio ergo sum

Lt. Cmdr. Daze
Lt. Cmdr. Daze
Joined: 19 Apr 06
Posts: 756
Credit: 82361
RAC: 0

By the way, I checked pmap on

By the way, I checked pmap on the einstein-app. Some of it:

:
:
081ba000 2900K rwx-- [ anon ]
b6872000 3312K rwx-- [ anon ]
b6bae000 4K ----- [ anon ]
b6baf000 8192K rwx-- [ anon ]
b73af000 4K ----- [ anon ]
b73b0000 8192K rwx-- [ anon ]
:
:
b7cc0000 364K r-x-- /usr/lib/opengl/xorg-x11/lib/libGL.so.1.2
b7d1b000 12K rwx-- /usr/lib/opengl/xorg-x11/lib/libGL.so.1.2
b7d1e000 4K rwx-- [ anon ]
:
:
total 27864K

I'll have to look up what it all means, but I noticed the usage of openGL. I assume that that's only used for the "show graphics" command (But that's for boinc. So that might be another point for improvement??

I'm definitely not a linux guru, but shouldn't that dependency have pop

The major part is however reserved for anon: "Memory not relating to any named object or file within the file system". Don't know yet what to think about that...

Regards,

Bert

Somnio ergo sum

Gray Handcock
Gray Handcock
Joined: 11 Mar 05
Posts: 211
Credit: 135567
RAC: 0

RE: Yeah, I've read about

Message 41921 in response to message 41919

Quote:

Yeah, I've read about hdparm. But not implemented yet. I didn't have the impression it would make such a big difference. After your post, I gathered more info. Definitely worth a try!

I also see in your USE list the compilation flag LDFLAGS. A thread on Gentoo suggest that -z would give better runtime performance at the expense of a slower startup and possibly more memory usage (you probably already read it). Looks like a future step as well. But I'd first like to get a bit more familiar with these flags.

I still think it's a bit strange that running benchmarking suggests that Debian would be a better choice (4 to 14 % actually), but my WU finished in 11h37m, so that would be an improvement of around 30%. Apparently too many variables are changed to have a good comparison of the boinc benchmarks (other OS, other BOINC version, other WUs). Do you also use some other benchmarking (I read you use BOINC benchmarking as well)?

BTW: I used the source from boinc, as the ebuild is masked.

Quote:
and the compiled version of BOINC plus associated libraries, definitely outdoes the sh version sitting in the user's home directory on my system.

Do you use another source?

Regards,
Bert

(edit: typing flaws)

Hiya Bert

Well if you cd to /etc/portage then "nano -w package.keywords" (note the dot between the 2 words). In another xterm (as root) do "emerge -s boinc", which should give you the category for the prog - ie: "sci-misc/boinc" and put those two words and the forward-slash into the "package.keywords" and add "~x86" - minus the quotes marks obviously - directly in front with a space between sci-misc/boinc and the ~x86. This will allow you to get the BOINC manager setup and installed by Gentoo, rather than doing it yourself. If you were able to copile from source then there doesn't seem much point in doing this, except that I prefer to let Gentoo's Portage control the packages - and all dependencies, and anything you install manually it will know nothing about. I had one issue. I had to remove the password in the /var/lib/boinc directory - file name: gui_rpc_auth.cfg .Also add your username to the boinc in /etc/group . Well to be absolutely accurate, I didn't have to remove the password, but I just got tired of retyping a password to connect to localhost to start the boinc manager. Boinc is also added to startup: "rc-update add boinc default"

Hope the above helps - you may need to stop boinc "/etc/init.d/boinc stop" and restart it again "/etc/init.d/boinc start" but coming from Windows, I just did a reboot and fired up "boinc_gui" to get the manager running...

One last comment - I don't get to see the screen-saver here, even tho I do have opengl working fine - but have had that working in the past with the sh version of BOINC

Gray

Gray Handcock
Gray Handcock
Joined: 11 Mar 05
Posts: 211
Credit: 135567
RAC: 0

Hiya Bert I had a look at

Hiya Bert

I had a look at your link - interesting reading. However I have decided to leave everything as is for now - I have recomplied entire system almost twice a day since I started my holidays, and I think maybe I'll take a break (at least for the next 24 hours until the urge to fiddle kicks in again - grin).

I'm actually looking for some pentium 4 processor kernel config info at the moment - that at least would be a quick boost if it works at all - kernel only takes around 8 minutes here, as compared to 6 hours for entire system (no KDE, thank goodness)

Cheers - Gray

Gray Handcock
Gray Handcock
Joined: 11 Mar 05
Posts: 211
Credit: 135567
RAC: 0

RE: Hello

Quote:

Hello fellow-Crunchers!

I am thinking of compiling my own kernel. I wondered if this would have impact on the crunching performance of my laptop.

I can imagine that a self-compiled kernel works more efficient, so more CPU could be available for crunching (isn't it?). But would it also have a direct effect on Einstein performance? And what about additional heat generation due to more efficient CPU usage( as p.e. seen by optimizations on the app for S4)?

Many thanks,
Bert

Hello Bert

Whether there will be a worthwhile difference depends on the amount of optimisation you do. My kernel is as tweaked as I am inclined to go, while using the standard gentoo-sources. My crunching times dropped from around 9 hours 50 minutes using KDE and SUSE, with around 80 processes started - not all running tho' - as compared to my present Fluxbox with optimised kernel and system (with some safe compiling optimisations) which seems to be able to crucnh a WU around 9 hours 30 minutes, if I am surfing, and a little less if I let it run undisturbed (9 hours 26 minutes was the most recent).

So after a week of fiddling and reading the forums etc, I have a speed-up of say 20 minutes per WU - on the other hand I also have a pretty snappy system and a fair bit of knowlege - also I backed up the kernel config, the make.conf and a couple other data files, so any future system will get optimised pretty darn quick.

If you use genkernel --menuconfig all, as I do then ensure that you only apply modules and support for what you actually have on the system. I have kept alsa compiled as a module, for sound, for example, but it doesn't start by default - I can use /etc/init.d/alsasound start if I want sound. I would suggest turning off preempt - keep it as for a server, and keep the amount of threads/interrupts to 100 mHz - back in XP at the moment, but both of these can be found under the processor configuration.

BOINC starts up in 2 seconds, Firefox in 3 secs - yeah, I'm happy (grin)

Hope the above helps answer the question - Gray

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.