Not only that, you can also download a ready baked OpenIndiana distribution (based on ou_147) or if you’re like me and still using OpenSolaris DEV snv_134, you can upgrade via the IPS management tools. Having said that though, I’m not going to rush and upgrade my zeus box anytime soon as it will take time to settle in, but you can take the baked ISO’s for a spin in a VM I have found a few references to OpenSolaris still there and there is currently no xVM Xen (dom0) support nor lx (Linux) branded zones. Not to worry, keep an eye out on the roadmap and release schedule for what they’re going to deliver.

You can get a copy of the OpenIndiana announcement presentation slides as well or follow @openIndiana on twitter. Otherwise, see the Getting Involved guide on the OpenIndiana Wiki and join in!

In a way, its good to know that the beloved OpenSolaris will still live – thanks to the community, but at the same time, how long that community will be turned on by developing and maintaining it will be interesting – though other forks of OpenSolaris are backing it (via Illumos) – like Nexenta and Schillix which has just released a version based on Ilumos. All in all, WATCH THIS PROJECT!

Related posts:

1. OpenSolaris 2008.11 out the door!!!
2. ZFS gets deduplication
3. Next generation Linux file-systems

Server refused to allocate pty

DOH! This is caused by a known bug that has been around for a few builds now.

You’ll need to modify /etc/minor_perm and add the following to the bottom of the file.

clone:ptmx 0666 root sys

And what happens if your terminals don’t accept keyboard input? You could drop back into the shell *or* be lazy like me, find gText editor in your Accessories, add it to the panel and change the properties to run it as a privileged user:

pfexec gedit %U

Then run the file, open the /etc/minor_perm file, save and reboot. Make sure you change back the shortcut path

Related posts:

1. Next generation Linux file-systems
2. OpenSolaris 2008.11 out the door!!!
3. In the Zone, Creating OpenSolaris Zones.

To update to the latest BE, simply update your packages and image-update (after configuring the dev repository!).

# pkg refresh --full
# pkg image-update
# reboot

If you’ve created zones in your OpenSolaris system after upgrading to the latest BE you will need to upgrade your zones as well. Here’s a simple guide on how to update a zone named tomcat to the BE on the global zone.

# zoneadm -z tomcat halt
# zoneadm -z tomcat detach
# zoneadm -z tomcat attach -u
# zoneadm -z tomcat boot

The output of the attach and upgrade command appears below, here I am upgrading from 127 to 130.

Log File: /var/tmp/tomcat.attach_log.23aWZl
Attaching...

       Global zone version: entire@0.5.11,5.11-0.130:20091219T044839Z
   Non-Global zone version: entire@0.5.11,5.11-0.127:20091111T131831Z
           Publisher Check: Zone preferred publisher does not contain
                            entire@0.5.11,5.11-0.130:20091219T044839Z.
           Publisher Reset: Copying preferred publisher from global zone.
  Updating non-global zone: (Stage 1).  Output follows
DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                              130/130   6842/6842  191.0/191.0

PHASE                                        ACTIONS
Removal Phase                              3529/3529
Install Phase                              7108/7108
Update Phase                               5247/5247
  Updating non-global zone: (Stage 2).  Output follows
No updates necessary for this image.
  Updating non-global zone: Zone updated to entire@0.5.11,5.11-0.130:20091219T044839Z
Attach complete.

Thats it, the updated zones are now booted! Whilst I’m posting this, if you want to upgrade to a specific version of OpenSolaris you can do that too!

# pkg refresh --full
# pkg image-update --be-name opensolaris-128

This will upgrade your BE to 128 instead of the latest – 130.

Related posts:

1. Part II: Rebuilding ZEUS – The Operating System, FileSystem & Virtualisation
2. Setting up OpenSolaris Extras Repository for VirtualBox, True-Type Fonts, Flash & JavaFX SDK
3. ZFS gets deduplication

Related posts:

1. ZFS gets deduplication
2. OpenSolaris 2008.11 out the door!!!
3. Rebuilding Zeus – Part I.5: Change of heart, change of hardware.

You can read all about the Sun releases on InternetNews and catchup with whats new in Java EE 6 Overview from Suns site.

Next weekend its time to move Confluence & Jira (Glassfish 2) and TeamCity 5 (Tomcat) to Glassfish 3 in a opensolaris zone and see how things progress. Did I mention I love the zones in OpenSolaris?

Related posts:

1. Setting up OpenSolaris Extras Repository for VirtualBox, True-Type Fonts, Flash & JavaFX SDK
2. The Slick New DeveloperFusion.com is ‘soft-launched’
3. Java news: $10 Confluence, Jira & Atlassian products, and InteliJ goes opensource!

As I’ve come to realise, because of the way Solaris works in general, you can have multiple (isolated & secure) Zones for each application service exposed by the server – eg. one for Tomcat, one for Glassfish, maybe both Apache 1.3.x and 2.x, MySql, Postgres etc. Whats more, you can limit how much resources these Zones can utilise. They all have their own configuration including network routing (coupled with OpenSolaris Crossbow) and you can make for one kick ass setup that won’t break another area of the operating system.

In the Zones.

Here’s a guide on setting up a new Zone in OpenSolaris, configuring it and booting it.

Me Against the Music, its all in the global zone

When we first install OpenSolaris we’ve already got ourselves into a zone (the parent to all other zones) which is known as the global Zone.

You can find this by trying out the following to list all the available zones on a virgin install of OpenSolaris.

opensolaris# zoneadm list -vc
 ID NAME             STATUS     PATH                           BRAND    IP
 0 global           running    /                              native   shared

The output will be something like above. Now we can go about creating ourselves a zone for playing around in.

When working with zones, we only need to worry about three commands (damn I love that!). The zoneadm command to manage the physical zone, zonecfg command for configuring the zone and zlogin to login to the zone from the global zone.

First we have to do a bit of planning and thinking about what we’re going to do about this zone.

Here are few things to consider:

• What do you want to run in the zone?
• Will it need networking and have it exposed outside of the machine?
• Where will the zone reside on your disk?
• Would you like to limit the amount of CPUs the zone can see?
• Would you like to limit the amount of RAM the zone can utilise?
• Do you want to automatically boot the Zone when OpenSolaris starts?

For this post, we’re going to create a simple Zone (we won’t install anything).

Toxic Zone

Creating a zone we specify a zone to the zonecfg command.

opensolaris# zonecfg -z toxic

You’ll get something like this appearing because teh zone doesn’t exist, thats fine.

toxic: No such zone configured
Use 'create' to begin configuring a new zone.

Then you will be inside the zonecfg configuration.

Lets configure this zone to have the following:

• Reside in /base/zones/
• Autoboot with OpenSolaris
• Shared IP of 192.168.0.24 bound to physical interface e1000g1

Follow me:

zonecfg:toxic> create
zonecfg:toxic> set zonepath=/base/zones/
zonecfg:toxic> set autoboot=true
zonecfg:toxic> add net
zonecfg:toxic:net> set address=192.168.0.24
zonecfg:toxic:net> set physical=e1000g1
zonecfg:toxic:net> end
zonecfg:toxic> verify
zonecfg:toxic> commit
zonecfg:toxic> exit

This will create the configuration, verify, write it and exit. You can verify it was created by running the list command again:

opensolaris# zoneadm list -vc
ID NAME             STATUS         PATH
0 global           running        /
- toxic            configured     /base/zones

Its currently in a configured state, you can read more about the Non-Global State Model in the documentation. Next thing to do is to install the zone – this will get the base packages setup and configured for use.

opensolaris# zoneadm -z toxic install

Everytime, boot her up.

Next lets boot this bad baby up.

opensolaris# zoneadm -z toxic boot

Now if we do a list again we’ll see that our state has changed to running.

opensolaris# zoneadm list -vc
ID NAME             STATUS         PATH
0 global           running        /
- toxic            running        /base/zones

Now we have to configure the zone itself – just like a real machine. For this we use the zlogin command to login to the zone console.

opensolaris# zlogin toxic
[Connected to zone 'toxic' pts/5]
Last login: Sat Nov 21 17:52:43 on pts/5
Sun Microsystems Inc.   SunOS 5.11      snv_127 November 2008
root@toxic#

After that we’re now in the toxic zone. Anything we do inside here, stays within this zone and won’t affect our global or other zones. But before we continue we really should configure our networking.

First lets modify our /etc/nsswitch.conf file with vi.

...
passwd:     files
group:      files
hosts:      files dns
ipnodes:    files
networks:   file
...

Make sure the hosts entry has dns as above. Next we need to configure the nameservers.

toxic# echo 'nameserver 192.168.0.254' > /etc/resolv.conf

That will create a resolv.conf file with the nameserver which you can get from the global zone as it would be different for everyone:

opensolaris# cat /etc/resolv.conf
nameserver 192.168.0.254

Breath on me, reboot the zone.

Now we can access the networking like the global zone. So you can do a package refresh and update-image too.

toxic# pkg refresh && pkg image-update

If it succeeds we have correctly setup our zone and its ready for use – you may want to reboot the zone however. To do this, exit the toxic console.

toxic# exit
logout

[Connection to zone 'toxic' pts/5 closed]
opensolaris#

Then lets reboot the zone.

opensolaris# zoneadm -z toxic reboot
opensolaris# zlogin toxic
[Connected to zone 'toxic' pts/5]
Last login: Sat Nov 21 17:58:44 on pts/5
Sun Microsystems Inc.   SunOS 5.11      snv_127 November 2008
root@toxic#

Outrageous, removing the zones.

Now how about removing this zone and trying again? First get out of the zone console and back to your global zone. Issue the halt command to shutdown the zone.

root@toxic# exit
opensolaris# zoneadm -z toxic halt

Once stopped simply remove it.

opensolaris# zoneadm -z toxic uninstall
opensolaris# zonecfg -z toxic delete

You can make sure its gone by using the list command. That’s all there is to it!

Now you can consider yourself, In The Zone.

Related posts:

1. [XKCD] Security, in The Real World.
2. Boffins get 1,000,000 Linux Kernels running as virtual machines!
3. Think outside the box: Getting VirtualBox 2.x running in Jaunty

Here’s my final hardware specifications:

• CPU: AMD Athlon X2 5050e – 2.6Ghz (45W TDP, AMD-V)
• Motherboard: Gigabyte GA-MA790X-UD4P ( AMD 790X Chipset )
• RAM: 2x Corsair TWIN2X4096-6400C5 (4Gb kit x 2 = 8Gb)
• Graphics: ASUS 9400GT PCI-Express
• Hard Disks:
  • rpool – 2x WD740ADFD – 74Gb 10K RPM, 16Mb Cache (mirror’d)
  • tank – 6x WD1002FBYS – 1TB, 7200RPM, 32Mb Cache (raidz)
  • base – 2x WD7500AAKS – 750Gb, 7200RPM, 16Mb (mirror’d)
• Addon cards:
  • SATA – Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller
  • NICs – 2x Intel Corporation 82545GM Gigabit Ethernet Controller (e1000g)

I’ve finally managed to get the GA-MA790X-UD4P on the OpenSolaris HCL list – woo! Unfortunately the onboard NIC will not work in the 2009.06 release even though it is detected:

Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller

Maybe in a future release. Make sure you update the BIOS as OpenSolaris may have an issue with the USB controller being ‘mis-configured’ otherwise.

Just for kicks I went to Jaycar and bought myself a power usage meter to measure the watts used by the new boxen (see a review of the Mains Power Meter on DansData).

Old Zeus

• Idle: 380W
• Load: 413W

New Zeus

• Idle: 232W
• Load: 270W

Nice, with an Intel Atom based server it could go _a lot_ lower, but I’m happy with this.

Related posts:

1. Part II: Rebuilding ZEUS – The Operating System, FileSystem & Virtualisation
2. Rebuilding Zeus – Part I.5: Change of heart, change of hardware.
3. Rebuilding Zeus: Part 1 – Preliminary Research and Installing Ubuntu 9.04 RC1

$ zfs set dedup=on tank/src

Read Jeff Bonwick’s (the supremo source for ZFS) article as it covers everything you’d ever want to know about what deduplication is and the various strategies behind it. Can’t wait for it to be implemented in OpenSolaris 2010.02 its already been integrated into the ON source base.

Whilst on the subject of ZFS, there was a very good article posted on OSNews recently regarding the lack of fsck for ZFS, it gives you a very good overview of ZFS, what COW really implies, how it differs from journaling filesystems found in Linux and ofcourse regarding fsck.

And to compliment it all, a deeper look at the RAID-Z on disk format in ZFS. Light reading for Melbourne Cup Day!

Related posts:

1. OpenSolaris 2008.11 out the door!!!
2. Rebuilding Zeus – Part I.5: Change of heart, change of hardware.
3. Part II: Rebuilding ZEUS – The Operating System, FileSystem & Virtualisation

The Operating System

As the operating system is no longer maintained (my oversight into how long it would survive) I have to find an OS that supports the hardware platform without hacky hacky bits (and by this I mean avoiding buggy ACPI and issues with the NForce4 chipset and IRQ problems) and has a file system that will benefit long term.

There were a few considerations:

• Ubuntu 8.04.x LTS
  I like Ubuntu, I’m comfortable with the user land and find the Debian package system (in particular the dependency resolving) most impressive. Hardware is well supported and 8.04.3 (at the time of writing) boots on the hardware I originally selected (Intel) and the new configuration I recently selected (AMD). I could most definitely use Ext4 but the problems with data-loss (which I’ve reproduced on several occasions on desktop machines) scare me.FileSystem: I’d have to adopt either XFS or Ext4 on an LVM to factor in future-proofing, maybe get some fakeRAID happening for redundancy.
  Installation: comes with a Server edition that’s bare bones allowing it to be a minimalistic installation which is always nice!
• Ubuntu 9.04
  Initially when I started to rebuild Zeus back in April I wanted to use Ubuntu 9.04, I was really excited about Ext4 and the promise of a brand-spanking new file-system and what it would bring to the table. Unfortunately after using Ext4 with 9.04 I’ve come to realise its probably not the wisest to trust your data with it just yet – unless you get yourself a UPS! Laptop seems to be chugging nicely though.Installation: Like LTS, comes with a Server edition that’s bare bones allowing it to be a minimalistic installation which is always nice! (copy/paste!) Unfortunately picking 9.04 when 9.10 is just around the corner is not going to be ideal, I’ll be stuck with where I am right now in a year or so.

• CentOS 5.3 or wait for CentOS 5.4 coming soon!
  CentOS is based off Redhat Enterprise Linux and having only started using CentOS after joining HSD (though my first splash in Linux was with Bowtie Redhat Linux 6.0 – I still have the CD!) I find the RHEL clone to be stable and solid as a rock! CentOS 5.4 will (thanks to RHEL 5.4) bring some KVM goodness that is a high priority on my list.

  FileSystem: Ext4 is not in the RHEL5 stream (have to wait for RHEL6 – which is a good thing!) so the best bet here is to use XFS with LVM.
  Installation: Customised via kickstarts or setup a cobbler server to install via PXE.

• OpenSolaris
  OpenSolaris has some really exciting technologies that are worth a look at, but in particular is ZFS. ZFS breaks all the rules – and for a good reason. I’ve yet to really use Solaris – quite cosy with Linux, but there’s nothing like getting used to another OS by jumping straight into the deep end!

  FileSystem: ZFS all the way baby!
  Installation: Installation is via a LiveCD, I’ve looked into customising the installation so it removes some bits I don’t which I’ll discuss later but its quite fiddly.

So in case the sudden influx of OpenSolaris posts didnt give you the hint, I decided on OpenSolaris to power the new iZeus 2.0, actually no that sounds lame, zeusy will be the new ZEUS until ZEUS is retired in which case zeusy becomes zeus (confused?).

Why ZFS?

ZFS is one of those file-systems you look at and think, wow! Why didn’t anyone else think of that before?

• Very simple administration – you only use two commands, zpool and zfs.
• Highly scalable – 128-bit means we can hold 16 exabytes or 18 Million terabytes worth of data! More porn for you! XFS can no doubt handle the TBs we use for our home boxes now, but no-chance you can get the performance or benefits of ZFS in Ext3/Ext4 or XFS.
• Data integrity to heal a filesystem (no fsck’ing around!) – 256bit checksuming to protect data, if ZFS detects a problem it will attempt to reconstruct the bad block and continue on its merry way (utilising available redundancy)
• Compression – you can elect to compress a particular file-system or a hierarchy just by setting one command! I’m thinking things like logs here.
• No hardware dependency – JBOD on a controller, let ZFS maintain the RAID volumes in software. Checkout Michael Pryc’s crazy adventure with ZFS using USB thumb drives and Constantin’s original voyage with USB drives! RAID-Z is essentially RAID-5 without the write-hole problems has plagued it if power is lost during a write, it can also survive a loss of a drive (with RAIDZ-2 you can loose two drives).
• Happy snaps for free! Snapshot (a live) file-system as many times as you like, again one easy command. Its like that tendency to hit {CTRL+S} when your working in Windows from back in the days of Windows 9x, snapshot regularly!

So ZFS sounds much like marketing spiel right now, best thing since sliced bread, cooler than a cucumber, and you’d be right it is cool and the best thing since filesystems came to being. Over the coming days I’ll post some more on my musings with ZFS – keeping in mind that I’m still learning these things. It helps to have lots of hardware to play with, but even if you don’t, you can knock up a virtual version of OpenSolaris in VirtualBox, create some virtual disks and try it out.

There are a few caveats that I’ve come across though using ZFS, one is memory! ZFS will try and cache as much data as it can in RAM, so if you have 8Gb of RAM (as I have in this box) it will happily use as much of it as it can afford. Rightfully so, I was getting ~96MB/s transfering a 16Gb MPEG from one box to the other over our Gig link (thats from one end of the house to the other!) mind you this was just a test configuration using 2x 74Gb Western Digital Raptors (WD740ADFD) in a RAID-0 style hitting a single 150Gb Western Digital Raptor (WD1500ADFD). They could have gone much higher, but I was happy with that.

There are also (as of writing) no recovery tools for ZFS, but these are slated to arrive soon (Q4 2009) which is quite scary after you read this post about a guy loosing 10Tb worth of data, however a possible revert to an older uberblock may fix some problems.

Virtualisation

Initially I wanted to concentrate quite a bit on Virtualisation, I tried Xen on OpenSolaris. It was quite easy to setup a Xen Dom0 in OpenSolaris but with the 2009.06 release you had to tweak the Xen setup a bit. I wasn’t too enthusiastic about using Xen after seeing the performance lag in Windows in my musings. Instead I’m opting for my crush, VirtualBox.

So why use VirtualBox when you can get a bare-metal hypervisor? Firstly, performance seems to be sluggish with Xen for me (I didn’t investigate this too much), secondly I want to be able to run the latest and greatest OS’s out without worrying about upgrading Xen (I’m a sucker for OS’s!). VirtualBox development has accelerated at a feverish pace, I started with VirtualBox 1.3 in 2007 and its come an insanely long way since then. When a new release comes along, its as easy as updating VirtualBox and getting all the benefits. Plus with SunOracle‘s backing of VirtualBox you know things are going to work well on OpenSolaris, the Extras repository of VirtualBox makes it as easy as doing a pkg update.

I’m still quite intrigued by the way KVM is heading and how it will pan out, but for the future zeus, it will be VirtualBox.

Related posts:

1. Rebuilding Zeus – Part I.5: Change of heart, change of hardware.
2. Rebuilding Zeus: Part 1 – Preliminary Research and Installing Ubuntu 9.04 RC1
3. Sun ushers in VirtualBox 2.1 with cool new features!

thushan@zeusy:~$ pfexec pkg install virtualbox
Traceback (most recent call last):
  File "/usr/bin/pkg", line 2598, in ?
    __ret = main_func()
  File "/usr/bin/pkg", line 2541, in main_func
    return install(mydir, pargs)
  File "/usr/bin/pkg", line 710, in install
    update_index=update_index)
  File "/usr/lib/python2.4/vendor-packages/pkg/client/api.py", line 203, in plan_install
    self.log_operation_end(error=e)
  File "/usr/lib/python2.4/vendor-packages/pkg/client/api.py", line 1410, in log_operation_end
    self.img.history.log_operation_end(error=error, result=result)
  File "/usr/lib/python2.4/vendor-packages/pkg/client/history.py", line 680, in log_operation_end
    self.operation_result = result
  File "/usr/lib/python2.4/vendor-packages/pkg/client/history.py", line 279, in __setattr__
    raise AttributeError("'history' object attribute '%s' "
AttributeError: 'history' object attribute 'operation_result' cannot be set before 'operation_name'.

pkg: This is an internal error.  Please let the developers know about this
problem by filing a bug at http://defect.opensolaris.org and including the
above traceback and this message.  The version of pkg(5) is '87d6ba4c8e1c'.

Uh-oh, what the hell did I break now I thought? After some messing about I realised the time on the machine was a few hours behind – this was just installed on the new hardware I picked up the other day, the certificates were timestamped and I figured this was probably a clash of the space-time continuum. Instead of opting to manually set the time, I let it sync (periodically) with an NTP Server local to us here in Melbourne.

To do this, we enter our NTP server in /etc/inet/ntp.conf like so:

server ntp.mel.connect.com.au

Then we tell it to update itself:

thushan@zeusy:~$ pfexec ntpq -p
 remote           refid      st t when poll reach   delay   offset    disp
==============================================================================
*yarrina.connect mumnunah.csse.u  2 u   20   64  377    50.22  -11.747    0.44

Done, now you’ll find that your package manager will no longer fail with the stack-trace as the timestamps will match correctly!

UPDATE: There’s already a bug report about this in the OpenSolaris Bugzilla.

No related posts.