Best Linux distro for crunching?

robl
robl
Joined: 2 Jan 13
Posts: 1709
Credit: 1454480596
RAC: 8837

RE: RE: When I referrred

Quote:
Quote:
When I referrred to the PCLinux MiniMe as being "advanced" those were not my words but the words used by the PCLinux website in referring to their MiniMe product.

I was specifically referring to 'bare bones' in case readers thought PCLinuxOS didn't have a fully functional DE. It's 'stripped' of non-essential stuff which is what you want for a pure crunching box. The only reason they call it "advanced" is that they really want users to get all the standard stuff in a standard install rather than have them come back complaining that they don't have the full gamut of apps you would expect on a fully outfitted machine.

In an earlier message in this thread, I posted a link to an installation howto which shows quite a bit of the disk partitioning stuff. Yes you can accept defaults and let the installer carve up the disk for you but I would strongly suggest you do what is shown in the howto and select the custom option. It's a full GUI process of creating partitions, selecting the type of filesystem you want, sizing them to your heart's content (dragging a slider) and selecting mount points from drop down menus. Sure, you need to know that you should have a least root, swap and home partitions (as is mentioned in the howto) and that root and home should be of type 'ext4', whilst the swap partition is of type 'linux swap'. If you play around with the options you find on these screens you will discover that it's a pretty simple procedure to get exactly what you want. If you make a mistake you can 'cancel' with a single click and go back to the start to try again. And, as you intimated, the first time you try something new and unfamiliar, it's bound to seem more difficult than it actually is.

I'd like to address a couple of individual points.

Quote:
Poking around I realized that there were two "configure options": 1. Configure your desktop and 2. Configure your computer.

"Configure your computer" is also known as the PCC (PCLinuxOS Control Centre) which is a really nice GUI way to configure a lot of (but not all) things on the system immediately after installation. You should always use this first to set up what you can. "Configure your Desktop" is a tool specific to KDE configuration. In most cases, the default configuration of KDE is quite satisfactory but if you want to change the look and feel of your KDE desktop and tweak a few things, this is the tool to use.

There is a very small amount of duplication between the two - setting the timezone is certainly one. You actually get asked what your BIOS clock is set to (UTC or local) and you get asked to set the timezone during the first reboot after the install. I used to set the BIOS clock to local but now I use UTC because I read somewhere that there are bugs in handling daylight saving correctly in some localities that are avoided with a UTC setting. I also select the option to use NTP (with an appropriate time server from a drop down list) to keep the time correct and it works really well on all hosts.

So the basic steps are I use are:

  • * Install the OS,
    * Reboot,
    * Use PCC to configure things associated with the entire machine including static IP address,
    * Use the CLI to edit /etc/hosts /etc/sysconfig/network /etc/apt/sources.list (you can easily do this last one later by GUI from within Synaptic),
    * Logout and immediately log back in to pick up the above config changes,
    * Use Synaptic to mark and apply all updates and extra stuff you want to install (including things like hardware drivers that weren't installed originally - like drivers for AMD GPUs),
    * Use PCC again (Hardware -> Video) if an AMD fglrx driver has been installed. This will 'see' the proprietary driver and ask you if you want to use it. Click 'yes',
    * Reboot again to ensure that all configuration and updates are fully operational, and finally,
    * Use the "Configure your Desktop" icon to explore all the things you can tweak to make your desktop look and feel the way you want it to.

Quote:
During the install I do not recall asking to provide a "hostname".

You aren't asked, a default is used which you need to deal with manually. It's something missing from the install procedure but fairly easy to rectify once you know how. You can use the hostname command to set a hostname but it won't survive a reboot. There is a file called "network" that lives in /etc/sysconfig. If you add a line at the top of the file saying "HOSTNAME=xxxx" then the name you choose there will become permanent.

Quote:
Yes you can do this from the command line, but a "man hostname" produces nothing. If fact a man of the basic Linux command line commands do not exist, i.e., ls, ping, df, etc. This is a loss for the new person running Linux.

This is in keeping with the philosophy of MiniMe. We are talking about a group of machines dedicated to crunching. Within that group, you can nominate one machine from which you will control the group. You could install the standard version of PCLinuxOS on that machine and I think you will have the man pages as part of that bigger install. I don't know that for certain as I've put the MiniMe version on everything. If I want to view a man page, I use an online man page service like this one.

If you want to view a man page installed on your machine, you have to open a terminal session and use a (shock, horror) CLI command of 'man xxxx' where xxxx is the name of the command in question. If it's a multi-page document, you need to know how to manipulate the pager that's doing the displaying of the information. For someone learning, I think it's easier to find and browse the online html pages using a standard browser of their choice. If you check out the link I provided you'll see a variety of ways to find info including a search box where you can type in the command name and get not only the command but also closely related stuff. When you actually open the html page, the presentation seems to be superior to the standard display in a terminal session.

Quote:

The default install produced the following output from a "df":
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 12G 2.5G 8.7G 23% /
/dev/sda6 131G 251M 131G 1% /home

What. A default install give 12 gig to "/" and 131 gig to /home. Really. I know there is a way to modify partioning during install but again a new guy would not necessarily have the ability to do this without much effort. I would not run with these defaults.


I would (for a crunching box) since there's plenty of room for the OS to grow with only 23% used. I've used around 5 - 7 GB for the root partition for years and never had problems. I want to have all my spare space for user files on /home. If you ever need to reinstall the OS, you will be able to leave everything in /home completely untouched.

In any case, as mentioned earlier, selecting custom partitioning and getting what you want size-wise is quite simple and straight forward.

Quote:
I had to hunt up a way to disable the touchpad on my Dell. There is a app/prog to do this and I installed it but you have to know this. In Ubu it is part of the options for mouse.

Try opening PCC -> Hardware and under "Configure Mouse and Keyboard" there is a "Setup pointer device (mouse and touchpad)". I don't have any touchpads so I've never used this.

If you don't get any joy there, try opening "Configure your Desktop" and select "Startup and Shutdown" in the System Administration section at the bottom of the page. On the menu at the left, click "Service Manager" and under "Startup Services" you will see a "KDE touchpad enabler daemon" which is enabled by default. Try unticking the box and see if that does what you want. I always disable a number of these services that I know I wont need for crunching.

There are a few other very useful things to know for improving the setup. Also there's a couple more hints for running BOINC. I'll deal with them in a future post.

I am not defending/promoting/criticizing one Linux distro over another. My point in responding at all has to do with whether a distro is suitable for someone with limited Linux experience/exposure. Every distro has a package that is scaled down and these are not necessarily ideal for beginners. For example, I would not recommend Ubuntu's "Kylin" distro to anyone, unless they were fluent in Chinese.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5842
Credit: 109383536146
RAC: 35947984

RE: I am not

Quote:
I am not defending/promoting/criticizing one Linux distro over another. My point in responding at all has to do with whether a distro is suitable for someone with limited Linux experience/exposure. Every distro has a package that is scaled down and these are not necessarily ideal for beginners. For example, I would not recommend Ubuntu's "Kylin" distro to anyone, unless they were fluent in Chinese.


Sure, and I really understand every point you made - all valid points.

Whilst I responded to those points, I wasn't really responding to you. I wanted the readership in general to know that every potential deal breaker had a solution and that I was prepared to talk through the solution if people asked about any problem.

A person trying Linux for the first time is going to have a steep learning curve no matter what distro they try - I remember what it was like in 2007 when I first experimented with putting Linux on multiple machines. I tried quite a few then (very briefly, I admit) before finding and being seduced by the relative ease of PCLinuxOS in comparison. The advice I would give is to find what suits you best and then push yourself to explore every corner of it until you are really comfortable. That all takes time.

I don't really care what a person chooses to use - that's their decision. In general, the best help is likely to come if that choice is used by and familiar to others willing to share knowledge.

Cheers,
Gary.

tullio
tullio
Joined: 22 Jan 05
Posts: 2118
Credit: 61407735
RAC: 0

I've used Slackware, Unifix,

I've used Slackware, Unifix, Corel and now use SuSE Linux with Ubuntu as a Virtual Machine running Test4Theory@home. SuSE is not very user friendly but I like its paucity of icons on its start page, while Ubuntu has too many. But I really like Solaris, a powerful OS for which there are few BOINC clients and none of the 7 BOINC projects I am running.
Tullio

robl
robl
Joined: 2 Jan 13
Posts: 1709
Credit: 1454480596
RAC: 8837

RE: I've used Slackware,

Quote:
I've used Slackware, Unifix, Corel and now use SuSE Linux with Ubuntu as a Virtual Machine running Test4Theory@home. SuSE is not very user friendly but I like its paucity of icons on its start page, while Ubuntu has too many. But I really like Solaris, a powerful OS for which there are few BOINC clients and none of the 7 BOINC projects I am running.
Tullio

Many years ago Sun Micro Systems had a PC version of Solaris. It ran quite well as I recall. I used it for quite sometime running RealPlayers video server stuff and it was quite good. But time marches on. When my Sparc 10 died with a bad power supply I pretty much left Sun/Solaris behind. But now you have me thinking. Hmm?....

tullio
tullio
Joined: 22 Jan 05
Posts: 2118
Credit: 61407735
RAC: 0

RE: Many years ago Sun

Quote:

Many years ago Sun Micro Systems had a PC version of Solaris. It ran quite well as I recall. I used it for quite sometime running RealPlayers video server stuff and it was quite good. But time marches on. When my Sparc 10 died with a bad power supply I pretty much left Sun/Solaris behind. But now you have me thinking. Hmm?....


I just installed VirtualBox 4.3.14 on my Solaris 11.2 Virtual Machine just for testing it and it runs well. But when I tried to install a second tier Virtual Machine with Ubuntu 14.04 on it the installation failed. But this may be a bug of Virtual Box. I am using VBox 4.3.12 on my vLHC@home project, running on 2 Linux boxes and it runs perfectly.
Tullio

mikey
mikey
Joined: 22 Jan 05
Posts: 11888
Credit: 1828050366
RAC: 206450

RE: I don't really care

Quote:

I don't really care what a person chooses to use - that's their decision. In general, the best help is likely to come if that choice is used by and familiar to others willing to share knowledge.

Over at gpugrid there was a user that was going to make a Boinc Linux setup cd that had all the amd and nvidia drivers in it for newbies, but he got angry at something and walked away from it all, even Boinc. The idea was that is was essentially a plug and play setup for Boinc only machines, similar to what you have, doing alot of the stuff with auto running scripts. So if you have an Nvidia card it would auto download an then install those drivers, likewise for those with AMD cards. When he quit he said he was in the testing phase, tweaking things. I have ALWAYS thought that Boinc should make one of these for newbies, picking a distro that is both easy to use and is robust enough to not need replacing as Boinc grows and the user gets more comfortable with it. I'm guessing that alot of people would even pay some minimal amount if it 'just worked' for them.

Gary Roberts
Gary Roberts
Moderator
Joined: 9 Feb 05
Posts: 5842
Credit: 109383556146
RAC: 35949862

RE: Over at gpugrid there

Quote:
Over at gpugrid there was a user that was going to make a Boinc Linux setup cd that had all the amd and nvidia drivers in it for newbies, but he got angry at something and walked away from it all, even Boinc.


I imagine he would have quit once he realised the magnitude of the maintenance burden he was exposing himself to. It's reasonably easy to make a live CD image that contains everything needed (including all drivers). It's quite possible for it to be ready to crunch immediately after an installation and some simple configuration. You could have a script to handle configuration that offered choices and options to the user. The real difficulty would be making it user proof. It's all very well to make a script where the author of the script is the user. In that case you can make a lot of assumptions about what the user understands and needs :-). But if you have to make a script that can cope with anybody (and any level of knowledge) running it, that task would be several orders of magnitude larger.

And then you would have to properly document it so that people new to Linux could make informed choices (if they could be convinced to properly read the documentation) about options they were being asked about. And on top of that, as the Linux distro you choose gets updated, or a new driver is released, or a different version of BOINC is needed, you would be caught in a cycle of releasing new versions of the CD image on a reasonably regular basis. It's a very much bigger project and commitment than you might think at first glance.

Quote:
The idea was that is was essentially a plug and play setup for Boinc only machines, similar to what you have, doing alot of the stuff with auto running scripts. So if you have an Nvidia card it would auto download an then install those drivers, likewise for those with AMD cards.


You can't have a completely "plug and play setup" - a sort of one size fits all that is smart enough to second guess every little option the user should be selecting. There are always configuration choices the user has to understand and make. You get to understand those choices by trying and failing and then asking specifically about the exact problem you had and why it failed. You digest the answer and then you try again - until you hit the next snag, or get to the desired goal.

There will be 'snags' with every distro that currently exists. If you find something (any distro that's been around for more than 5 years) that 'feels right' for you and you take the time to 'learn it' to the point that installation and initial configuration comes naturally to you, it will probably turn out to be the best choice for you. As well as maturity, the other thing that helps a lot is the availability of quality information and help - user forum, wiki, etc.

Cheers,
Gary.

mikey
mikey
Joined: 22 Jan 05
Posts: 11888
Credit: 1828050366
RAC: 206450

RE: RE: Over at gpugrid

Quote:
Quote:
Over at gpugrid there was a user that was going to make a Boinc Linux setup cd that had all the amd and nvidia drivers in it for newbies, but he got angry at something and walked away from it all, even Boinc.

I imagine he would have quit once he realised the magnitude of the maintenance burden he was exposing himself to. It's reasonably easy to make a live CD image that contains everything needed (including all drivers). It's quite possible for it to be ready to crunch immediately after an installation and some simple configuration. You could have a script to handle configuration that offered choices and options to the user. The real difficulty would be making it user proof. It's all very well to make a script where the author of the script is the user. In that case you can make a lot of assumptions about what the user understands and needs :-). But if you have to make a script that can cope with anybody (and any level of knowledge) running it, that task would be several orders of magnitude larger.

And then you would have to properly document it so that people new to Linux could make informed choices (if they could be convinced to properly read the documentation) about options they were being asked about. And on top of that, as the Linux distro you choose gets updated, or a new driver is released, or a different version of BOINC is needed, you would be caught in a cycle of releasing new versions of the CD image on a reasonably regular basis. It's a very much bigger project and commitment than you might think at first glance.


The scripts were going to be run using the 'sudo' tag, so he was in charge of the pc, this was something we had to agree to upfront.

Quote:
The idea was that is was essentially a plug and play setup for Boinc only machines, similar to what you have, doing alot of the stuff with auto running scripts. So if you have an Nvidia card it would auto download an then install those drivers, likewise for those with AMD cards.

Quote:

You can't have a completely "plug and play setup" - a sort of one size fits all that is smart enough to second guess every little option the user should be selecting. There are always configuration choices the user has to understand and make. You get to understand those choices by trying and failing and then asking specifically about the exact problem you had and why it failed. You digest the answer and then you try again - until you hit the next snag, or get to the desired goal.

There will be 'snags' with every distro that currently exists. If you find something (any distro that's been around for more than 5 years) that 'feels right' for you and you take the time to 'learn it' to the point that installation and initial configuration comes naturally to you, it will probably turn out to be the best choice for you. As well as maturity, the other thing that helps a lot is the availability of quality information and help - user forum, wiki, etc.

You are correct, but he was going to try and make something like the distro that GpuGrid already has available for their users, except this new one was going to require less user input to get it up and crunching initially, no matter what was in the actual pc as far as hardware goes. For example it would download, install and set the permission for the latest Boinc version, the latest hardware drivers etc, etc. A basic distro for dummies if you like. I know that would NOT work for the more experienced users, like yourself, but for a complete Linux newbie it was a way into the world of crunching without learning anything up front. We never got so far as updates later on, I am guessing that is where the learning phase would kick in, or he would put out some scripts that would do it automatically. YES it involved trusting him to NOT hijack our pc's, but you must trust someone else some time, I don't drive the trains or planes but I still get on both.

mikey
mikey
Joined: 22 Jan 05
Posts: 11888
Credit: 1828050366
RAC: 206450

I am trying a new Linux

I am trying a new Linux Distro I read about, Zorin, it is pretty cool, easy to setup, almost Windowsish, and based on Ubuntu. I do NOT have a gpu in the laptop that can crunch, but with a wired internet connection it did some updates, saw both my wireless networks, and even found the latest version of Boinc in the software repository. It is now up and running, but will take over 11 hours to finish an Asteroids unit. I will move it to another project, assuming it is still running, tomorrow after the one unit is done. It is an older dual core Dell D-830 Intel T-7700 cpu laptop with 4gb of ram. It is NOT fast, it never was even running Vista, which it came with, or Win7 which I installed later on, but it is perfectly usable as a laptop.

Hawkeye
Hawkeye
Joined: 15 Sep 14
Posts: 1
Credit: 9693871
RAC: 0

I'm a new cruncher for

I'm a new cruncher for Einstein, and after an initial run on Windows 7, I am moving my systems over to Linux.

Completed moves:
AMD Athlon 64 X2 6000 + GTX 650Ti
AMD Athlon X4 555 + GTX 650Ti

and tomorrow, I'll migrate my
AMD Athlon II X4 630 + GTX 750Ti (x2) over

I started with this post
http://www.overclock.net/t/1467918/ubuntu-server-12-04-4-64bit-boinc-using-gpu-from-geforce-gt610-to-crunch-data/80#post_22702255

and tailored it to Ubuntu Server 14.04.1 along with using Cuda 6.5 and newest drivers for x86_64

So far so good, with no issues to speak of.
If somebody needs a walk through, let me know.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.