Thushan Fernando Uncut

SHOCKING news today, Asustek has decided to…

Taiwan’s Asustek Computer on Friday said it would spin off Pegatron Technology, a subsidiary which makes more than half of the world’s computer motherboards, distancing itself from the business which made it into a familiar name in the computer industry.

The company is following a similar path to that taken 10 years ago by Acer, which has since gone on to become the world’s second-largest PC maker behind Hewlett-Packard. While its Asus brand is still only a small player in the global PC market, its successful Eee PC low-cost computer series, launched in 2007, helped to pioneer the netbook market and gave its Asus brand a position on electronic store shelves particularly in Asia and the US.

From XBIT Lab’s news article:

Asustek Computer, the world’s largest maker of mainboards, has decided to completely spin off its mainboards and graphics cards manufacturing arm Pegatron Technologies, the company said this week. The move will allow Asus to become more competitive in terms of branding, but will further withdraw the firm from the actual manufacturing.

According to a statement posted with Taiwan Stock Exchange, Asustek company had convened a board meeting to resolve the spin-off of its ODM business. As a result of the meeting, Pegatron Holding, the company that made virtually all Asus-branded motherboards, will issue two billion new shares to a number of shareholders.

I sure hope the quality and stability of the boards do not suffer. Read more from the Financial Times or XBIT Labs. When it comes to solid boards you can’t go past ASUS, XFX and Gigabyte (bang for buck!).

Now that I’ve decided what I want out of the server (and the hardware I’ve got), its time to workout what operating system to run the system on. Currently, ZEUS is running on Ubuntu Gutsy (7.10) which is running LVM with an XFS volume holding approximately 2.5Tb worth of data. There’s a cron job that defrags the XFS volume to keep things in order.

The Operating System

As the operating system is no longer maintained (my oversight into how long it would survive) I have to find an OS that supports the hardware platform without hacky hacky bits (and by this I mean avoiding buggy ACPI and issues with the NForce4 chipset and IRQ problems) and has a file system that will benefit long term.

There were a few considerations:

• Ubuntu 8.04.x LTS
  I like Ubuntu, I’m comfortable with the user land and find the Debian package system (in particular the dependency resolving) most impressive. Hardware is well supported and 8.04.3 (at the time of writing) boots on the hardware I originally selected (Intel) and the new configuration I recently selected (AMD). I could most definitely use Ext4 but the problems with data-loss (which I’ve reproduced on several occasions on desktop machines) scare me.FileSystem: I’d have to adopt either XFS or Ext4 on an LVM to factor in future-proofing, maybe get some fakeRAID happening for redundancy.
  Installation: comes with a Server edition that’s bare bones allowing it to be a minimalistic installation which is always nice!
• Ubuntu 9.04
  Initially when I started to rebuild Zeus back in April I wanted to use Ubuntu 9.04, I was really excited about Ext4 and the promise of a brand-spanking new file-system and what it would bring to the table. Unfortunately after using Ext4 with 9.04 I’ve come to realise its probably not the wisest to trust your data with it just yet – unless you get yourself a UPS! Laptop seems to be chugging nicely though.Installation: Like LTS, comes with a Server edition that’s bare bones allowing it to be a minimalistic installation which is always nice! (copy/paste!) Unfortunately picking 9.04 when 9.10 is just around the corner is not going to be ideal, I’ll be stuck with where I am right now in a year or so.

• CentOS 5.3 or wait for CentOS 5.4 coming soon!
  CentOS is based off Redhat Enterprise Linux and having only started using CentOS after joining HSD (though my first splash in Linux was with Bowtie Redhat Linux 6.0 – I still have the CD!) I find the RHEL clone to be stable and solid as a rock! CentOS 5.4 will (thanks to RHEL 5.4) bring some KVM goodness that is a high priority on my list.

  FileSystem: Ext4 is not in the RHEL5 stream (have to wait for RHEL6 – which is a good thing!) so the best bet here is to use XFS with LVM.
  Installation: Customised via kickstarts or setup a cobbler server to install via PXE.

• OpenSolaris
  OpenSolaris has some really exciting technologies that are worth a look at, but in particular is ZFS. ZFS breaks all the rules – and for a good reason. I’ve yet to really use Solaris – quite cosy with Linux, but there’s nothing like getting used to another OS by jumping straight into the deep end!

  FileSystem: ZFS all the way baby!
  Installation: Installation is via a LiveCD, I’ve looked into customising the installation so it removes some bits I don’t which I’ll discuss later but its quite fiddly.

So in case the sudden influx of OpenSolaris posts didnt give you the hint, I decided on OpenSolaris to power the new iZeus 2.0, actually no that sounds lame, zeusy will be the new ZEUS until ZEUS is retired in which case zeusy becomes zeus (confused?).

Why ZFS?

ZFS is one of those file-systems you look at and think, wow! Why didn’t anyone else think of that before?

• Very simple administration – you only use two commands, zpool and zfs.
• Highly scalable – 128-bit means we can hold 16 exabytes or 18 Million terabytes worth of data! More porn for you! XFS can no doubt handle the TBs we use for our home boxes now, but no-chance you can get the performance or benefits of ZFS in Ext3/Ext4 or XFS.
• Data integrity to heal a filesystem (no fsck’ing around!) – 256bit checksuming to protect data, if ZFS detects a problem it will attempt to reconstruct the bad block and continue on its merry way (utilising available redundancy)
• Compression – you can elect to compress a particular file-system or a hierarchy just by setting one command! I’m thinking things like logs here.
• No hardware dependency – JBOD on a controller, let ZFS maintain the RAID volumes in software. Checkout Michael Pryc’s crazy adventure with ZFS using USB thumb drives and Constantin’s original voyage with USB drives! RAID-Z is essentially RAID-5 without the write-hole problems has plagued it if power is lost during a write, it can also survive a loss of a drive (with RAIDZ-2 you can loose two drives).
• Happy snaps for free! Snapshot (a live) file-system as many times as you like, again one easy command. Its like that tendency to hit {CTRL+S} when your working in Windows from back in the days of Windows 9x, snapshot regularly!

So ZFS sounds much like marketing spiel right now, best thing since sliced bread, cooler than a cucumber, and you’d be right it is cool and the best thing since filesystems came to being. Over the coming days I’ll post some more on my musings with ZFS – keeping in mind that I’m still learning these things. It helps to have lots of hardware to play with, but even if you don’t, you can knock up a virtual version of OpenSolaris in VirtualBox, create some virtual disks and try it out.

There are a few caveats that I’ve come across though using ZFS, one is memory! ZFS will try and cache as much data as it can in RAM, so if you have 8Gb of RAM (as I have in this box) it will happily use as much of it as it can afford. Rightfully so, I was getting ~96MB/s transfering a 16Gb MPEG from one box to the other over our Gig link (thats from one end of the house to the other!) mind you this was just a test configuration using 2x 74Gb Western Digital Raptors (WD740ADFD) in a RAID-0 style hitting a single 150Gb Western Digital Raptor (WD1500ADFD). They could have gone much higher, but I was happy with that.

There are also (as of writing) no recovery tools for ZFS, but these are slated to arrive soon (Q4 2009) which is quite scary after you read this post about a guy loosing 10Tb worth of data, however a possible revert to an older uberblock may fix some problems.

Virtualisation

Initially I wanted to concentrate quite a bit on Virtualisation, I tried Xen on OpenSolaris. It was quite easy to setup a Xen Dom0 in OpenSolaris but with the 2009.06 release you had to tweak the Xen setup a bit. I wasn’t too enthusiastic about using Xen after seeing the performance lag in Windows in my musings. Instead I’m opting for my crush, VirtualBox.

So why use VirtualBox when you can get a bare-metal hypervisor? Firstly, performance seems to be sluggish with Xen for me (I didn’t investigate this too much), secondly I want to be able to run the latest and greatest OS’s out without worrying about upgrading Xen (I’m a sucker for OS’s!). VirtualBox development has accelerated at a feverish pace, I started with VirtualBox 1.3 in 2007 and its come an insanely long way since then. When a new release comes along, its as easy as updating VirtualBox and getting all the benefits. Plus with SunOracle‘s backing of VirtualBox you know things are going to work well on OpenSolaris, the Extras repository of VirtualBox makes it as easy as doing a pkg update.

I’m still quite intrigued by the way KVM is heading and how it will pan out, but for the future zeus, it will be VirtualBox.

After a bit of digging around, my original spec’d hardware I’ve decided is too much for a boxen that will be on 24×7, especially with the rates for electricity going up next year – every little Watt counts. The existing 65W CPU isn’t ideal, instead I’m opting for a 45W CPU instead and this means – looking at the lineup, its going to be a walk down AMD way. Less watts, less heat and less noise, noice! See AMD’s product roadmap for 2010-2011.

The original specifications I mentioned were:

• Motherboard: ASUS P5QL-PRO
• CPU: Intel Core-2 E6750 – 2.66Ghz (65W TDP, VT)
• RAM: Corsair TWIN2X4096-6400C5 (4Gb kit x 2 = 8Gb)
• Graphics: ASUS 9400GT PCI-Express

I’ve decided to change the CPU and Motherboard but keep the other bits and bobs – I could loose the graphics card and go onboard but I felt like leaving it there for now. The target budget is $250 maximum for both CPU+Mobo, so this means I’m sticking with DDR2 which implies AM2+ but it must also satisfy:

• CPU has to be 45W and be atleast 1.6Ghz, dual core no more, has to support Virtualization.
• Motherboard has to Support 8Gb (most boards doo!),  have atleast 2x  PCIe and a PCI slot, it would be nice if the network cards work (gigabit) but no fuss if it doesnt. No crazy shebangabang Wifi, remotes etc bling and if it has onboard Video great, otherwise its OK to use a crappy card.

I picked the AMD Athlon X2 5050e CPU because it was cheap (~$80), supports a 45W, has virtualisation and is an AM2. Next was the motherboard, looking at the ASUS, Gigabyte & XFX models as my target.

Chipset wise only the following fit the criteria for a possible match because others just don’t have the number of SATA ports available onboard. Primarily AMD boards are supplied by NVIDIA or AMD themselves.

• NVIDIA nForce 750a SLI
• NVIDIA GeForce 8200
• NVIDIA GeForce 8300
• AMD 790X Chipset

Initially I looked at the ASUS  boards (they’ve been nothing but rock solid for me in the past) but after a lot of research scouring through the manufacturer sites I ended up picking out the Gigabyte GA-MA790X-UD4P which is based on the AMD 790X Chipset. The board came with 8x SATA Ports, 3x PCIe and 2x PCI and a  Gigabit NIC all for a $137 from PCCaseGear. Not only was the power consumption lowered but the noise and heat generated was substantially lower too!

Coming in close was the ASUS M4N78 PRO or the ASUS M4A78 PRO, each of those unfortunately didn’t have as many SATA ports (2-less) nor the PCIe ports (1-less).

Just spent a fair chunk of today getting a rebuild of Zeus going – our affectionately dubbed Ubuntu server at home. This is the third rebuild (hardware wise) in the past 5 years (sheesh its been that long?), but I’m not complaining. First Ubuntu’fied version (5.10 – Breezy Badger) ran on an Pentium 4 3Ghz (Socket 478), noisey little guy that sucked quite a bit of power which was my old development box  that served me well.

Then with the release of the fornicating Feisty Fawn (Ubuntu 7.04) I moved over the server to an AMD box, a AMD 3200+ on a ASUS A8N-SLI Deluxe (which featured the incredibly shakey NForce 4 SLI chipset) with a modest 2Gb of DDR ram.

NVIDIA nForce4 APIC Woes

Unfortunately I didn’t realise that by using the NForce 4 chipset under Linux I’d have to wrestle with APIC issues due to an issue with the chipset and regressions.

If you fall into the above hole, edit your grub boot menu:

$ sudo vi /boot/grub/menu.lst

And change your booting kernel with two new options:

title           Ubuntu 7.10, kernel 2.6.22-14-generic
root            (hd0,5)
kernel          /vmlinuz-2.6.22-14-generic root=UUID=c7a7bf0a-714a-482e-9a07-d3ed40f519f5 ro quiet splash noapic nolapic
initrd          /initrd.img-2.6.22-14-generic
quiet

You may want to also add that to the recovery kernel just incase. This will effectively disable the onboard APIC Controller as its quite buggy. More information is available on Launchpad.

Its been chugging along nicely for the past 2 years – the time is always in accurate (about 8 minutes ahead) but the uptime right now is:

thushan@ZEUS:~$ uptime
19:54:06 up 147 days,  7:27,  7 users,  load average: 0.22, 0.43, 0.32

So I figured its time to put these issues behind and redo the server infrastructure at home.

Goals

There are some goals in this rebuild.

• Try out Ext4 and remove the use of ReiserFS and JFS which don’t seem to be going anywhere (JFS here and here). ZFS would be nice (but no FUSE!) to try out, but I’m hoping Btrfs brings some niceties to the table.
• The new Zeus needs to look at virtualisation a little more. Right now, alot of the QA for Windows builds of our stuff is done on several machines all over the place. Consolidate them to 1 Server with VT support, plenty of RAM and use a hypervisor (mentioned later) to manage testing.
• Provide the same services as the existing Zeus:
  • SVN + Trac
  • Apache
  • MySQL / Postgres
  • File hosting, storing vault sharing content across the computers around (the whole house is gigabitted).
  • Fast enough to run dedicated servers for Unreal Tournament, Quake, Call of Duty 4 and a few other games.
  • Profiles, user data needs to be migrated
• Messing about with the Cloud-Computing functionality in Jaunty.
• Provide a backend for the Mythbuntu frontends.
• Last another 2 years

Hardware

My previous workstation motherboard was the awesome ASUS P5W-DH Deluxe with a Intel QX6850 CPU, powered by the Intel 975 Chipset that has lasted for alot longer than anyone had predicted. But earlier this year I had a problem with the board that warranted a RMA request. As I had to have a machine I ended up buying an ASUS P5Q-Pro and did a re-install (same CPU). So instead of selling off the P5WDH I’ve decided to use that board coupled with a Intel E6750 which was picked because it supports Intel VT and it was lying around. Otherwise I _wouldnt_ consider using this setup – overkill!!! But I do want this setup to last and be beefy enough to support a little more than a few VM’s running concurrently.

• ASUS P5W-DH Deluxe – [Specs] Summary: Intel 975 Chipset, dual-gigabit (not really required!), 8-Phase Power, 5 usable SATA ports (more will be added via PCIe)
  
• Intel E6750 – 65W TDP, Virtualisation
• 8Gb of DDR2, in my initial messing about, I’m using Corsair TWIN2X2048-6400C4PRO but eventually they’ll be a couple of the much cheaper Corsair TWIN4096-6400C5‘s.
• Main drive is a  Seagate ST3320620AS 320Gb drive which will be running in RAID-1 (onboard the P5WDH’s EZ-RAID)
• Some crappy el-cheapo graphics card
• 600W PSU, theres going to be a few HDDs, this will be the same as the existing Zeus – CoolerMaster iGreen Power 600W
• Some optical drive lying around
• Same Lian-Li V1000+ II case.

Pretty shots are available here. Otherwise, the test bench, the tuniq and a pretty shot of my setup at home (no its not clean).

Software

Clearly Ubuntu  9.04 is where its at, its sleeker, blindingly fast to boot thanks to the boot time optimisations and sexier desktop thanks to the visual tweaking and the new Gnome 2.26 inclusion. The installer has matured greatly, gone is the plain old boring partition editor based on GParted and a sleek new timezone picker. To make the most of the RAM in the box, 64bit edition of Ubuntu-desktop is what I’m installing.

Installing Ubuntu, use a UNetbootin!

So you grabbed the latest ISO, burn and chuck it into an optical drive and way you go aye… *IF THIS WAS 2005*!!! As mentioned in an earlier post, grab a copy of UNetbootin, select the ISO you mustered from your local free ISP mirror and throw it inside your USB thumb drive. These days USB drives are dirt cheap, I picked up a Corsair Voyager 8Gb (non-GT) for AUD$39.

Why would you want to do that?  You wont need to use CD-RWs, delete and put another ISO and whats more, it will install in no time. With the VoyagerI got the core OS installed in 5 minutes – after selecting the iinet local software sources mirror. Funky?

Hypervisors

I got into the Virtualisation game early, VMWare 2.0 (2000-2001) is where it all began after seeing a close friend use it. Unfortunately I had to almost give up my kidney to afford to buy it. Then a brief time  I moved to Connectix VirtualPC when VMWare 4.0 arrived and messed up my networking stack, but went back to VMWare 3.0 for a little while. Then eventually moved back to VirtualPC 2004 after Microsoft acquired Connectix (it was free from the MSDN Subby) and back again on VMWare with version 5.

Fast forward to 2009, we have some ubber quality hypervisors. VMWare still has the behmoth marketshare but a little birdie got some extra power from the Sun and impressed everyone lately with its well roasted features. But the critical decision was which hypervisor to use, we have VMWare Server (1.0 or the 2.0 with its web interface – errr!), XenServer (which is now owned by Citrix) or VirtualBox.

After playing around with VMWare Server 1.0 last year I was left wanting more, so naturally I moved to VMWare Server 2.0 not knowing that the familiar client interface is GAWN, instead in its place is a web based implementation – VI Web Access.  It was slow and clunky and took a while to get used to – but the fact that it showed the status via the web was funky, but runnig an entire VM Session via a browser plugin (which hosed every so often) was far from impressive 🙁

It finally boiled down to deciding to go with VMWare Server 1.0 (released mid-2006), leaning onto XenServer (seems to include a bit of a learning curve) or to move to a brighter pasture with Sun VirtualBox – which is what I use on my development boxes. I’m still playing around with all three to see how they fair. I am a little biased towards VirtualBox (  I reckons its awesome ja! )  but as this is a long-term build I can’t knock out VMWare Server out just yet nor go the full para-virtualisation with XenServer which is probably what I’ll end-up doing.

I’ve only got a few days before the final release of Ubuntu 9.04 arrives and all this research prior is to make sure things go smoothly next weekend.

A new book titled The Race for a New Game Machine: Creating the Chips Inside the XBox 360 and the Playstation 3 was released on the 1st of Jannuary this year that looks into the development of the Microsoft Xbox 360 and the Sony Playstation 3 which, as it turned out in the end, were both developed by the IBM Corporation.

The authors of the book, David Shippy (who was the man behind the brains of the Cell) and his co-worker, Mickie Phipps goes into the depths of nerdisms to give an insight into the development of The Cell processor. From the Wall Street Journal review:

When the companies entered into their partnership in 2001, Sony, Toshiba and IBM committed themselves to spending $400 million over five years to design the Cell, not counting the millions of dollars it would take to build two production facilities for making the chip itself. IBM provided the bulk of the manpower, with the design team headquartered at its Austin, Texas, offices. Sony and Toshiba sent teams of engineers to Austin to live and work with their partners in an effort to have the Cell ready for the Playstation 3’s target launch, Christmas 2005.

But a funny thing happened along the way: A new “partner” entered the picture. In late 2002, Microsoft approached IBM about making the chip for Microsoft’s rival game console, the (as yet unnamed) Xbox 360. In 2003, IBM’s Adam Bennett showed Microsoft specs for the still-in-development Cell core. Microsoft was interested and contracted with IBM for their own chip, to be built around the core that IBM was still building with Sony.

All three of the original partners had agreed that IBM would eventually sell the Cell to other clients. But it does not seem to have occurred to Sony that IBM would sell key parts of the Cell before it was complete and to Sony’s primary videogame-console competitor. The result was that Sony’s R&D money was spent creating a component for Microsoft to use against it.

And here’s the real kicker.

Mr. Shippy and Ms. Phipps detail the resulting absurdity: IBM employees hiding their work from Sony and Toshiba engineers in the cubicles next to them; the Xbox chip being tested a few floors above the Cell design teams. Mr. Shippy says that he felt “contaminated” as he sat down with the Microsoft engineers, helping them to sketch out their architectural requirements with lessons learned from his earlier work on Playstation.

The deal only got worse for Sony. Both designs were delivered on time to IBM’s manufacturing division, but there was a problem with the first chip run. Microsoft had had the foresight to order backup manufacturing capacity from a third party. Sony did not and had to wait another six weeks to get their first chips. So Microsoft actually got the chip that Sony helped design before Sony did. In the end, Microsoft’s Xbox 360 hit its target launch in November 2005, becoming its own success. Because of various delays, the Playstation 3 was pushed back a full year.

The book (which arrived on Friday!) goes into all the juicy bits that lead up to the delivery of both processors, well worth the $14USD its listed for on Amazon. Whilst I havent finished the entire book yet, thus far its full of twists and corporate musings and tricks with an interesting look at the teams and people that made these two products possible in the end. You’ll be hooked from the first page – I guarantee it.

MSY(.com.au – dont go there yet!), one of the most competitive IT hardware stores in Australia recently got hacked and the site has embedded Net-Worm.JS.Aspxor.a worm. Only realised after I went to the site and Firefox blocked the page. You can read all about the hack and the effects on the Whirlpool Thread or Google Safe Browsing diagnostic page.

Its always nice when someones got your back. Who knows MSY might actually endup making a proper website now instead of the messy FrontPage site that was.