Archive

Archive for the ‘File Systems’ Category

OpenIndiana Announced, the fork to Oracle’s OpenSolaris!

September 15th, 2010 No comments

OpenIndianaEarlier today, we had the announcement for OpenIndiana. Aimed to be the de-facto OpenSolaris Distribution that tries to be binary and package compatible with Solaris 11 & Solaris 11 Express. Its apart Illumos Community with 20 core developers providing (eventually) a stable branch with 100% free & open source distribution.

Not only that, you can also download a ready baked OpenIndiana distribution (based on ou_147) or if you’re like me and still using OpenSolaris DEV snv_134, you can upgrade via the IPS management tools. Having said that though, I’m not going to rush and upgrade my zeus box anytime soon as it will take time to settle in, but you can take the baked ISO’s for a spin in a VM 🙂 I have found a few references to OpenSolaris still there and there is currently no xVM Xen (dom0) support nor lx (Linux) branded zones. Not to worry, keep an eye out on the roadmap and release schedule for what they’re going to deliver.

You can get a copy of the OpenIndiana announcement presentation slides as well or follow @openIndiana on twitter. Otherwise, see the Getting Involved guide on the OpenIndiana Wiki and join in!

In a way, its good to know that the beloved OpenSolaris will still live – thanks to the community, but at the same time, how long that community will be turned on by developing and maintaining it will be interesting – though other forks of OpenSolaris are backing it (via Illumos) – like Nexenta and Schillix which has just released a version based on Ilumos. All in all, WATCH THIS PROJECT!

{lang: 'en-GB'}
Share

Upgrading non-Global OpenSolaris Zone to latest BE

January 14th, 2010 1 comment

I’ve been tracking the latest dev version of OpenSolaris (as of writing I just upgraded to Nevada SNV 130 ) because of some issues surrounding CIFS in the 2009.06 image of OpenSolaris.

To update to the latest BE, simply update your packages and image-update (after configuring the dev repository!).

# pkg refresh --full
# pkg image-update
# reboot

If you’ve created zones in your OpenSolaris system after upgrading to the latest BE you will need to upgrade your zones as well. Here’s a simple guide on how to update a zone named tomcat to the BE on the global zone.

# zoneadm -z tomcat halt
# zoneadm -z tomcat detach
# zoneadm -z tomcat attach -u
# zoneadm -z tomcat boot

The output of the attach and upgrade command appears below, here I am upgrading from 127 to 130.

Log File: /var/tmp/tomcat.attach_log.23aWZl
Attaching...

       Global zone version: entire@0.5.11,5.11-0.130:20091219T044839Z
   Non-Global zone version: entire@0.5.11,5.11-0.127:20091111T131831Z
           Publisher Check: Zone preferred publisher does not contain
                            entire@0.5.11,5.11-0.130:20091219T044839Z.
           Publisher Reset: Copying preferred publisher from global zone.
  Updating non-global zone: (Stage 1).  Output follows
DOWNLOAD                                  PKGS       FILES    XFER (MB)
Completed                              130/130   6842/6842  191.0/191.0

PHASE                                        ACTIONS
Removal Phase                              3529/3529
Install Phase                              7108/7108
Update Phase                               5247/5247
  Updating non-global zone: (Stage 2).  Output follows
No updates necessary for this image.
  Updating non-global zone: Zone updated to entire@0.5.11,5.11-0.130:20091219T044839Z
Attach complete.

Thats it, the updated zones are now booted! Whilst I’m posting this, if you want to upgrade to a specific version of OpenSolaris you can do that too!

# pkg refresh --full
# pkg image-update --be-name opensolaris-128

This will upgrade your BE to 128 instead of the latest – 130.

{lang: 'en-GB'}
Share

OpenSolaris cheatsheet

December 15th, 2009 No comments

Most excellent cheatsheet for OpenSolaris.

{lang: 'en-GB'}
Share

VirtualBox 3.1 released!

December 1st, 2009 No comments

Just when you thought you can start a new month without some new software, Sun has blessed the world with a ray of VirtualBox 3.1 goodness on us all! All hail the Sun. I’ve been using the Betas and trying out the spanking awesome Teleportation feature in VirtualBox 3.1. So lets take a bit of a look at the new grub.

Beam me up Scotty!

You know, people say the catch phrase thinking its from Star Trek, but did you know that it was never actually mentioned in any episode?

Teleportation or ‘Live Migration‘ in Xen/KVM  or vMotion in VMWare allows you to move a running virtual machine to another host without any downtime. Sun brings us this ‘Enterprise’ feature to VirtualBox. Whats even cooler, is that you can teleport your running VM on different host platforms (Windows -> OpenSolaris or Linux, vice versa) but not from one hardware set (Intel) to another (AMD) unless they both have the same instruction-sets. The transport layer for the teleportation is TCP/IP, so as long as the agreed port is open and accessible you can even teleport it through the tubes! (assuming you have a fast link like those pesky Dutch)

There are a few conditions and caveats as I’ve found. Firstly you must ensure (as you’d expect) the target VM has to have the exact same configuration as the source VM (same RAM, graphics memory, storage, CD/DVD images etc) the other thing is to be weary of the CPUs the host computer has. As long as its between the same generations (different clock speeds are OK) it should work (I tried between a QX6850 -> E6600 but QX6850->AMD X2 4600+ wasn’t so pretty!).

Once you’ve configured the target host to match the source host, time to ask VirtualBox to keep its eyes open for an incoming beam.

VBoxManage modifyvm [VirtualMachineName] --teleporter on --teleporterport [Port]

Then on the source host, send out the beams to initiate the teleportation:

VBoxManage controlvm [VirtualMachineName] teleport --host [TargetIP] --port [Port]

Give it some time to think and if you tried a localhost migration, it should migrate seamlessly 🙂

Scotty doesn’t know

Scotty doesn’t know about the other little changes, but you will. The new VirtualBox has lots of refinements in the UI. For one, new icons for all the Guest operating systems. The settings window has had a make over and includes ‘optimal settings’ detection.

Windows 2003 VM in VirtualBox 3.1

Windows 2003 VM in VirtualBox 3.1

Here its telling me my Windows 2003 VM should have atleast 20Mb Video Memory assigned to it to work well in full-screen mode. Heading over to the Display options in VirtualBox 3.1 we find that the Video Memory selectors have got little indicators now, as well as the inclusion of 2D Video Acceleration.

Windows 2003 VM - VirtualBox 3.1 Display Settings

Windows 2003 VM - VirtualBox 3.1 Display Settings

Depending how ever many cores you have, it will highlight what you should set as the maximum number of cores available for your VirtualMachine as well as the recommended RAM allocation. This is what I see in my Intel QX6850 development workstation.

VirtualBox 3.1 System Processor Settings

VirtualBox 3.1 System Processor Settings

VirtualBox 3.1 - Motherboard Settings

VirtualBox 3.1 - Motherboard Settings

VirtualBox now also has experimental support for Extensible Firmware Interface (EFI) which will eventually replace the aging BIOS bootstrap (which is the default). Well known operating systems that boot via EFI include Windows Vista and Windows 7, Apple OS X and Fedora 11+.

The Storage controls in VirtualBox GUI has also had a bit of a make over. The options to select a disk and a controller have changed, CD/DVD drives can be attached to an arbitrary IDE controller too now!

VirtualBox 3.1 - Storage

VirtualBox 3.1 - Storage

The networking settings GUI in the new VirtualBox has change too, not only that but you can now configure the network interfaces whilst the guest is running – YAY!

VirtualBox 3.1 Network Settings

VirtualBox 3.1 Network Settings

Snapshots are a lot more flexible in this release (much like VMWare’s snapshot feature). Previously you can only restore from the last created snapshot, now any arbitrary snapshot can be restored too or branched off.

For those who use OpenSolaris (like yours truely!) the rewritten USB support (still experimental btw!) should mean we can interact with our USB devices in Solaris Nevada 124 or higher now – I’m running 127 and have USB devices appearing in my VMs.

If those don’t give you any indication on to the pure awesomeness of this release, there was a significant performance improvement for APE & AMD64 guests (VT-x/AMD-V) which will be quite noticeable from what I’ve been told by a college.

As Barack Obama said, tis time for a change..log.

He didn’t say that, I just reused 36 Mafia’s Lolli Lolli. The entire change log appears below from the website.

VirtualBox 3.1.0 (released 2009-11-30)

This version is a major update. The following major new features were added:

  • Teleportation (aka live migration); migrate a live VM session from one host to another (see the manual for more information)
  • VM states can now be restored from arbitrary snapshots instead of only the last one, and new snapshots can be taken from other snapshots as well (“branched snapshots”; see the manual for more information)
  • 2D video acceleration for Windows guests; use the host video hardware for overlay stretching and color conversion (see the manual for more information)
  • More flexible storage attachments: CD/DVD drives can be attached to an arbitrary IDE controller, and there can be more than one such drive (the manual for more information)
  • The network attachment type can be changed while a VM is running
  • Complete rewrite of experimental USB support for OpenSolaris hosts making use of the latest USB enhancements in Solaris Nevada 124 and higher
  • Significant performance improvements for PAE and AMD64 guests (VT-x and AMD-V only; normal (non-nested) paging)
  • Experimental support for EFI (Extensible Firmware Interface; see the manual for more information)
  • Support for paravirtualized network adapters (virtio-net; see the manual for more information)

In addition, the following items were fixed and/or added:

  • VMM: guest SMP fixes for certain rare cases
  • GUI: snapshots include a screenshot
  • GUI: locked storage media can be unmounted by force
  • GUI: the a log window grabbed all key events from other GUI windows (bug #5291)
  • GUI: allow to disable USB filters (bug #5426)
  • GUI: improved memory slider in the VM settings
  • GUI: the VirtualBox website couldn’t be opened from the help menu (bug #4559)
  • 3D support: major performance improvement in VBO processing
  • 3D support: added GL_EXT_framebuffer_object, GL_EXT_compiled_vertex_array support
  • 3D support: fixed crashes in FarCry, SecondLife, Call of Duty, Unreal Tournament, Eve Online (bugs #2801, #2791)
  • 3D support: fixed graphics corruption in World of Warcraft (#2816)
  • 3D support: fixed Final frame of Compiz animation not updated to the screen (#4653)
  • 3D support: fixed incorrect rendering of non ARGB textures under compiz
  • iSCSI: support iSCSI targets with more than 2TiB capacity
  • VRDP: fixed occasional VRDP server crash (bug #5424)
  • Network: fixed the E1000 emulation for QNX (and probably other) guests (bug #3206)
  • NAT: added host resolver DNS proxy (see the manual for more information)
  • VMDK: fixed incorrectly rejected big images split into 2G pieces (bug #5523, #2787)
  • VMDK: fixed compatibility issue with fixed or raw disk VMDK files (bug #2723)
  • VHD: fixed incompatibility with Hyper-V
  • Support for Parallels version 2 disk image (HDD) files; see the manual for more information
  • OVF: create manifest files on export and verify the content of an optional manifest file on import
  • OVF: fixed memory setting during import (bug #4188)
  • Mouse device: now five buttons are passed to the guest (bug #3773)
  • VBoxHeadless: fixed loss of saved state when VM fails to start
  • VBoxSDL: fixed crash during shutdown (Windows hosts only)
  • X11 based hosts: allow the user to specify their own scan code layout (bug #2302)
  • Mac OS X hosts: don’t auto show the menu and dock in fullscreen (bug #4866)
  • Mac OS X hosts (64 bit): don’t interpret mouse wheel events as left click (bug #5049)
  • Mac OS X hosts: fixed a VM abort during shutdown under certain conditions
  • Solaris hosts: combined the kernel interface package into the VirtualBox main package
  • Solaris hosts: support for OpenSolaris Boomer architecture (with OSS audio backend).
  • Shared folders: VBOXSVR is visible in Network folder (Windows guests, bug #4842)
  • Shared folders: performance improvements (Windows guests, bug #1728)
  • Windows, Linux and Solaris Additions: added balloon tip notifier if VirtualBox host version was updated and Additions are out of date
  • Solaris guests: fixed keyboard emulation (bug #1589)
  • Solaris Additions: fixed as_pagelock() failed errors affecting guest properties (bug #5337)
  • Windows Additions: added automatic logon support for Windows Vista and Windows 7
  • Windows Additions: improved file version lookup for guest OS information
  • Windows Additions: fixed runtime OS detection on Windows 7 for session information
  • Windows Additions: fixed crash in seamless mode (contributed by Huihong Luo)
  • Linux Additions: added support for uninstalling the Linux Guest Additions (bug #4039)
  • Linux guest shared folders: allow mounting a shared folder if a file of the same name as the folder exists in the current directory (bug #928)
  • SDK: added object-oriented web service bindings for PHP5

Overall this is a solid new release from Sun – unsure about its stability as I’ve only been running a few VMs (Windows 2003, CentOS and Fedora 12) for about 10-12hrs. Nothing bad as yet.

Download from the VirtualBox site:

  • VirtualBox 3.1.0 for Windows hosts x86/amd64
  • VirtualBox 3.1.0 for Solaris and OpenSolaris hosts x86/amd64

Enjoy!

{lang: 'en-GB'}
Share

Part III: Zeus rebuilt and configured!

November 21st, 2009 1 comment

I’ve spent the last month working with the newly built zeus server which is now powered by OpenSolaris (2009.06).

Here’s my final hardware specifications:

  • CPU: AMD Athlon X2 5050e – 2.6Ghz (45W TDP, AMD-V)
  • Motherboard: Gigabyte GA-MA790X-UD4P ( AMD 790X Chipset )
  • RAM: 2x Corsair TWIN2X4096-6400C5 (4Gb kit x 2 = 8Gb)
  • Graphics: ASUS 9400GT PCI-Express
  • Hard Disks:
    • rpool – 2x WD740ADFD – 74Gb 10K RPM, 16Mb Cache (mirror’d)
    • tank – 6x WD1002FBYS – 1TB, 7200RPM, 32Mb Cache (raidz)
    • base – 2x WD7500AAKS – 750Gb, 7200RPM, 16Mb (mirror’d)
  • Addon cards:
    • SATA – Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller
    • NICs – 2x Intel Corporation 82545GM Gigabit Ethernet Controller (e1000g)

I’ve finally managed to get the GA-MA790X-UD4P on the OpenSolaris HCL list – woo! Unfortunately the onboard NIC will not work in the 2009.06 release even though it is detected:

Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller

Maybe in a future release. Make sure you update the BIOS as OpenSolaris may have an issue with the USB controller being ‘mis-configured’ otherwise.

Just for kicks I went to Jaycar and bought myself a power usage meter to measure the watts used by the new boxen (see a review of the Mains Power Meter on DansData).

Old Zeus

  • Idle: 380W
  • Load: 413W

New Zeus

  • Idle: 232W
  • Load: 270W

Nice, with an Intel Atom based server it could go _a lot_ lower, but I’m happy with this.

{lang: 'en-GB'}
Share

Fedora 12 released

November 18th, 2009 No comments

It only feels last last month Fedora 11 was released, alas Fedora 12 is out now. Read about the changes and updates found in Fedora 12 or maybe just view the summary or a full list if your ubber keen.

Amongst the many changes, important ones to forward to (from the release notes):

* Optimized performance - All software packages on 32-bit (x86_32)
  architecture have been compiled for i686 systems, with special
  optimization for the Intel Atom processors used in many netbooks,
  but without losing compatibility with the overwhelming majority of
  CPUs.

* Smaller and faster updates - In Fedora 11, the optional yum-presto
  plugin, developed by Fedora contributor Jonathan Dieter, reduced
  update size by transmitting only the changes in the updated
  packages. Now, the plugin is installed by default. Also, RPMs now
  use XZ rather than gzip for compression, providing smaller package
  sizes without the memory and CPU penalties associated with
  bzip2. This lets us fit more software into each Fedora image, and
  uses less space on mirrors, making their administrators' lives a
  little easier. Thanks to the Fedora infrastructure team for their
  excellent work in setting up the infrastructure to generate delta
  RPMs on the fly for all the updates.

* NetworkManager broadband and other enhancements - NetworkManager,
  originally developed by Red Hat's Dan Williams, was introduced in
  Fedora 7 and has become the de facto network configuration solution
  for distributions everywhere. Enhancements to NetworkManager make
  both system-wide connections and mobile broadband connections easier
  than ever. Bluetooth PAN support offers a simple click through
  process to access the Internet from your mobile
  phone. NetworkManager can now configure always-on and static address
  connections directly from the desktop. PolicyKit integration has
  been added so configuration management can be done via central
  policy where needed. IPv6 support has also been improved.

* Next-generation (Ogg) Theora video - For several years, Theora, the
  open and free format not encumbered by known patents has provided a
  way for freedom-loving users to share video. Fedora 12 includes the
  new Theora 1.1, which achieves very high quality comparable to
  H.264, meeting the expectations of demanding users with crisp,
  vibrant media in both streaming and downloadable form. Thanks to the
  work of the Xiph.Org Foundation's Christopher "Monty" Montgomery,
  sponsored by Red Hat, other Xiph developers and the contribution of
  Mozilla.org, Theora videos now deliver much better quality primarily
  via enhancements in the encoder without any change in the format,
  making it available to all Theora users. Using Theora video and
  Vorbis audio formats, Firefox 3.5 and applications using the
  Gstreamer multimedia framework can deliver free media on the web out
  of the box even better than the previous release of Fedora. Theora
  is being rapidly adopted by several popular websites including
  Wikipedia, VideoPress and DailyMotion. Fedora Project is proud to
  support communities of free culture and open content as part of our
  mission. More details at
  http://hacks.mozilla.org/2009/09/theora-1-1-released/

* Graphics support improvements - Fedora 12 introduces experimental 3D
  support for AMD Radeon HD 2400 and later graphics cards. To try it
  out, install the mesa-dri-drivers-experimental package. On many
  cards, this support should allow desktop effects to be used. Kernel
  mode setting (KMS) support, which was introduced on AMD hardware in
  Fedora 10 and extended to Intel hardware in Fedora 11, is now
  extended to NVIDIA hardware as well, meaning the great majority of
  systems now benefit from the smooth, fully-graphical startup
  sequence made possible by KMS. The Fedora graphical startup sequence
  now works better on systems with multiple monitors. Also on multiple
  monitor systems, the desktop will now automatically be spread across
  all monitors by default, rather than having all monitors display the
  same output, including on NVIDIA chips (where multiple monitor
  spanning was not possible without manual configuration changes in
  Fedora 11). Systems with NVIDIA graphics chips also gain initial
  support for suspend and resume functionality via the default Nouveau
  driver. Initial support for the new DisplayPort display connector
  has been added for Intel graphics chips. Support for Nvidia and ATI
  systems is already under rapid development and will be included in
  the next release of Fedora. Thanks to the Red Hat Xorg team
  including Adam Jackson (X server), Kristian Høgsberg (Intel driver),
  Dave Airlie and Jerome Glisse (Radeon driver for AMD), and Ben
  Skeggs (Nouveau driver for NVIDIA).

* Virtualization improvements - Not content with all the improvements
  in Fedora 11, we've kicked virtualization based on KVM up another
  notch in Fedora 12. There are extensive improvements in performance,
  management, and resource sharing, and still more security
  enhancements. A new library (libguestfs) and an interactive tool
  (guestfish) are now available for directly accessing and modifying
  virtual machine disk images. Richard W.M. Jones from Red Hat's
  virtualization team has a list of extensive virtualization tools
  available and coming up for Fedora at
  http://rwmj.wordpress.com/2009/10/20/fedora-virt-commands/ 

* Automatic reporting of crashes and SELinux issues - Abrt, a tool to
  help non-power users report crashes to Bugzilla with a few mouse
  clicks, is now enabled by default. Abrt collects detailed
  information automatically and helps developers identify and resolve
  issues faster, improving the quality of individual upstream
  components and Fedora. The SELinux alert monitoring tool has also
  added the ability to report SELinux issues to Bugzilla quickly and
  easily with just a couple of clicks.

* New Dracut initrd generation tool - Up until Fedora 11, the boot
  system (initial ram disk or initrd) used to boot Fedora was
  monolithic, very distribution specific, and didn't provide much
  flexibility. This has been replaced with Dracut, an initial ram disk
  generation tool with an event-based framework designed to be
  distribution-independent. Dracut has been also adopted by OLPC which
  uses Fedora; OLPC modules for Dracut are available in the Fedora
  repository. Thanks to the Dracut team, including Harald Hoyer,
  Jeremy Katz, Dave Jones, and many others.

* PackageKit plugins - PackageKit now has a plugin which can install
  an appropriate package when a user tries to run a command from a
  missing package. Another new plugin allows installation of software
  packages from a web browser. Thanks to Red Hat's Richard Hughes and
  the PackageKit team.

* Bluetooth on-demand - Bluetooth services are automatically started
  when needed and stopped 30 seconds after last device use, reducing
  initial startup time and resource use when Bluetooth is not in
  active use. Thanks to Red Hat's Bastien Nocera.

* Moblin graphical interface for netbooks - In additional to special
  compiler optimization for netbooks in this release and the continued
  integration of Sugar interface, the Moblin graphical interface and
  applications are fully integrated thanks to Peter Robinson, a Fedora
  Project volunteer, and others. Collaboration between the Moblin
  project and Fedora was accelerated since Moblin itself is largely
  based on Fedora. To use it, just install the Moblin Desktop
  Environment package group using yum or the graphical software
  management tools, and choose Moblin from the login manager. A Moblin
  Fedora Remix (installable Live CD) for Fedora 12 will also be
  available.

* PulseAudio enhancements - Red Hat's Lennart Poettering and several
  others have made significant improvements to the PulseAudio
  system. Improved mixer logic makes volume control more fine-grained
  and reliable. Integration with the Rygel UPnP media server means you
  can stream audio directly from your system to any UPnP / DLNA
  client, such as a Playstation 3. Hotplug support has been made more
  intelligent, so if you configure a device as the default output for
  a stream, unplug that device -- causing the stream(s) to be moved to
  another output device -- and later reattach it, the stream is moved
  back to the preferred device. Finally, Bluetooth audio support means
  pairing with any Bluetooth audio device makes it available for use
  through PulseAudio.

* Lower process privileges - In order to mitigate the impact of
  security vulnerabilities, permissions have been hardened for many
  files and system directories. Also, process privileges have been
  lowered for a number of core components that require super user
  privileges. Red Hat's Steve Grubb has developed a new library,
  libcap-ng, and integrated it into many core system components to
  improve the security of Fedora.

* SELinux sandbox - It is now possible to confine applications' access
  to the system and run them in a secure sandbox that takes advantage
  of the sophisticated capabilities of SELinux. Dan Walsh, SELinux
  developer at Red Hat, explains the details at
  http://danwalsh.livejournal.com/31146.html 

* Open Broadcom firmware - The openfwwf open source Broadcom firmware
  is included by default. This means wireless networking will be
  available out of the box on some Broadcom chipsets. 

* Hybrid live images - The Live images provided in this release can be
  directly imaged onto a USB stick using dd (or any equivalent tool)
  to create bootable Live USB keys. The Fedora Live USB Creator for
  Windows and Fedora and the livecd-tools for Fedora are still
  recommended for data persistence, encryption and non-destructive
  writes. Thanks to Jeremy Katz. 

* Better webcam support - While Fedora 11 improved webcam support, in
  Fedora 12 you can expect even better video quality, especially for
  less expensive webcams. Red Hat's Hans de Goede, developer of the
  libv4l library, has more details on his continuous upstream webcam
  support enhancements at
  http://hansdegoede.livejournal.com/6989.html. 

* Polished Desktop - The latest version of the GNOME desktop includes
  the lighter Gnote replacement for Tomboy as the default note
  application, and Empathy replaces Pidgin as the default instant
  messenger. The new volume control application, first seen in Fedora
  11, has been improved to cover more advanced users. There are many
  nice tweaks from the desktop team for a polished user
  experience. More details at
  http://fedoraproject.org/wiki/Desktop_Enhancements_in_Fed... 

* GNOME Shell preview - Fedora 12 includes an early version of GNOME
  Shell, which will become the default interface for GNOME 3.0 and
  beyond. To try it, install the gnome-shell package, and use the
  Desktop Effects configuration tool to enable it. It will only work
  correctly from the GNOME desktop environment, not others such as KDE
  or Xfce. This is a preview technology, and some video cards may not
  be supported. Thanks to Owen Taylor from Red Hat and the GNOME Shell
  team. 

* KDE 4.3 - The new KDE features an updated "Air" theme and fully
  configurable keyboard shortcuts in Plasma, improved performance and
  new desktop effects in the window manager, a new bug reporting tool,
  and a configuration tool for the LIRC infra-red remote control
  system. 

* Cool new stuff for developers beginning with Eclipse Galileo, which
  includes more plugins than ever before. Perl 6 is now included,
  along with PHP 5.3. For Haskell developers, the Haskell Platform now
  provides a standardized set of libraries and tools. But one of the
  biggest changes for developers is that most of the nice new features
  of Fedora 12, from Bluetooth to webcams, are implemented through
  underlying libraries, and many of the improvements will be included
  simply by relinking your application. Also available in this release
  are SystemTap 1.0 for improved instrumenting and debugging of
  binaries, complete with Eclipse integration, and the newest NetBeans
  IDE for Java development. 

* Cool new stuff for sysadmins include added functionality for
  clustered Samba services (including active/active configurations)
  over GFS2; and the ability to boot a cluster of Fedora systems from
  a single, shared root file system. 

* Multi-Pointer X - The update to X.Org server 1.7 introduces the X
  Input Extension version 2.0 (XI2), with much work contributed by Red
  Hat's Peter Hutterer. This extension provides a new client API for
  handling input devices and also Multi-Pointer X (MPX)
  functionality. MPX functionality allows X to cope with many inputs
  of arbitrary types simultaneously, a prerequisite for (among others)
  multitouch-based desktops and multi-user interaction on a single
  screen. This is low-level work of which applications and desktop
  environments will incrementally take advantage in future
  releases. More details are available in the Release Notes and in the
  XI2 tag of Peter Hutterer's blog at
  http://who-t.blogspot.com/search/label/xi2 

Download them from Fedora or if your a local:

I’m torn between using the latest Ubuntu or Fedora on the client.

{lang: 'en-GB'}
Share

Next generation Linux file-systems

November 5th, 2009 No comments

If you’ve been reading this blog a couple of things are clear, I don’t like Apple much and I have a soft spot for file-systems. An article was posted on the IBM DeveloperWorks site that covers two file systems; NiLFS(2) and exofs that has some great information about these two beasts.

Linux® continues to innovate in the area of file systems. It supports the largest variety of file systems of any operating system. It also provides cutting-edge file system technology. Two new file systems that are making their way into Linux include the NiLFS(2) log-structured file system and the exofs object-based storage system. Discover the purpose behind these two new file systems and the advantages that they bring.

Read the full article on the Next-generation linux filesystems, there was an article on LWN.net a few years back discussing the (then emerging) Btrfs and NiLFS and how things may pan out. I’m quite happy and content with ZFS but in either case it’ll be interesting to see how all three go.

{lang: 'en-GB'}
Share

ZFS gets deduplication

November 3rd, 2009 No comments

ZFS now has deduplication support which is as easy as just setting a property on the file system.

$ zfs set dedup=on tank/src

Read Jeff Bonwick’s (the supremo source for ZFS) article as it covers everything you’d ever want to know about what deduplication is and the various strategies behind it. Can’t wait for it to be implemented in OpenSolaris 2010.02 its already been integrated into the ON source base.

Whilst on the subject of ZFS, there was a very good article posted on OSNews recently regarding the lack of fsck for ZFS, it gives you a very good overview of ZFS, what COW really implies, how it differs from journaling filesystems found in Linux and ofcourse regarding fsck.

And to compliment it all, a deeper look at the RAID-Z on disk format in ZFS. Light reading for Melbourne Cup Day!

{lang: 'en-GB'}
Share

CentOS 5.4 Released!

October 23rd, 2009 No comments

CentOS 5.4 has been released! Woo yeah, its been a while since RHEL 5.4 has been out but checkout the release notes for a list of changes.

Download mirrors are being updated but if your local, here are a couple of Australian Mirrors.

CentOS 5.4 x86

CentOS 5.4 x64

I just did a inplace 5.3->5.4 upgrade with a yum update. With a localised mirror, blindingly fast too!

{lang: 'en-GB'}
Share

Part II: Rebuilding ZEUS – The Operating System, FileSystem & Virtualisation

October 18th, 2009 No comments

Now that I’ve decided what I want out of the server (and the hardware I’ve got), its time to workout what operating system to run the system on. Currently, ZEUS is running on Ubuntu Gutsy (7.10) which is running LVM with an XFS volume holding approximately 2.5Tb worth of data. There’s a cron job that defrags the XFS volume to keep things in order.

The Operating System

As the operating system is no longer maintained (my oversight into how long it would survive) I have to find an OS that supports the hardware platform without hacky hacky bits (and by this I mean avoiding buggy ACPI and issues with the NForce4 chipset and IRQ problems) and has a file system that will benefit long term.

There were a few considerations:

  • Ubuntu 8.04.x LTS
    I like Ubuntu, I’m comfortable with the user land and find the Debian package system (in particular the dependency resolving) most impressive. Hardware is well supported and 8.04.3 (at the time of writing) boots on the hardware I originally selected (Intel) and the new configuration I recently selected (AMD). I could most definitely use Ext4 but the problems with data-loss (which I’ve reproduced on several occasions on desktop machines) scare me.FileSystem: I’d have to adopt either XFS or Ext4 on an LVM to factor in future-proofing, maybe get some fakeRAID happening for redundancy.
    Installation
    : comes with a Server edition that’s bare bones allowing it to be a minimalistic installation which is always nice!
  • Ubuntu 9.04
    Initially when I started to rebuild Zeus back in April I wanted to use Ubuntu 9.04, I was really excited about Ext4 and the promise of a brand-spanking new file-system and what it would bring to the table. Unfortunately after using Ext4 with 9.04 I’ve come to realise its probably not the wisest to trust your data with it just yet – unless you get yourself a UPS! Laptop seems to be chugging nicely though.Installation: Like LTS, comes with a Server edition that’s bare bones allowing it to be a minimalistic installation which is always nice! (copy/paste!) Unfortunately picking 9.04 when 9.10 is just around the corner is not going to be ideal, I’ll be stuck with where I am right now in a year or so.

So in case the sudden influx of OpenSolaris posts didnt give you the hint, I decided on OpenSolaris to power the new iZeus 2.0, actually no that sounds lame, zeusy will be the new ZEUS until ZEUS is retired in which case zeusy becomes zeus (confused?).

Why ZFS?

ZFS is one of those file-systems you look at and think, wow! Why didn’t anyone else think of that before?

  • Very simple administration – you only use two commands, zpool and zfs.
  • Highly scalable – 128-bit means we can hold 16 exabytes or 18 Million terabytes worth of data! More porn for you! XFS can no doubt handle the TBs we use for our home boxes now, but no-chance you can get the performance or benefits of ZFS in Ext3/Ext4 or XFS.
  • Data integrity to heal a filesystem (no fsck’ing around!) – 256bit checksuming to protect data, if ZFS detects a problem it will attempt to reconstruct the bad block and continue on its merry way (utilising available redundancy)
  • Compression – you can elect to compress a particular file-system or a hierarchy just by setting one command! I’m thinking things like logs here.
  • No hardware dependency – JBOD on a controller, let ZFS maintain the RAID volumes in software. Checkout Michael Pryc’s crazy adventure with ZFS using USB thumb drives and Constantin’s original voyage with USB drives! RAID-Z is essentially RAID-5 without the write-hole problems has plagued it if power is lost during a write, it can also survive a loss of a drive (with RAIDZ-2 you can loose two drives).
  • Happy snaps for free! Snapshot (a live) file-system as many times as you like, again one easy command. Its like that tendency to hit {CTRL+S} when your working in Windows from back in the days of Windows 9x, snapshot regularly!

So ZFS sounds much like marketing spiel right now, best thing since sliced bread, cooler than a cucumber, and you’d be right it is cool and the best thing since filesystems came to being. Over the coming days I’ll post some more on my musings with ZFS – keeping in mind that I’m still learning these things. It helps to have lots of hardware to play with, but even if you don’t, you can knock up a virtual version of OpenSolaris in VirtualBox, create some virtual disks and try it out.

There are a few caveats that I’ve come across though using ZFS, one is memory! ZFS will try and cache as much data as it can in RAM, so if you have 8Gb of RAM (as I have in this box) it will happily use as much of it as it can afford. Rightfully so, I was getting ~96MB/s transfering a 16Gb MPEG from one box to the other over our Gig link (thats from one end of the house to the other!) mind you this was just a test configuration using 2x 74Gb Western Digital Raptors (WD740ADFD) in a RAID-0 style hitting a single 150Gb Western Digital Raptor (WD1500ADFD). They could have gone much higher, but I was happy with that.

There are also (as of writing) no recovery tools for ZFS, but these are slated to arrive soon (Q4 2009) which is quite scary after you read this post about a guy loosing 10Tb worth of data, however a possible revert to an older uberblock may fix some problems.

Virtualisation

Initially I wanted to concentrate quite a bit on Virtualisation, I tried Xen on OpenSolaris. It was quite easy to setup a Xen Dom0 in OpenSolaris but with the 2009.06 release you had to tweak the Xen setup a bit. I wasn’t too enthusiastic about using Xen after seeing the performance lag in Windows in my musings. Instead I’m opting for my crush, VirtualBox.

So why use VirtualBox when you can get a bare-metal hypervisor? Firstly, performance seems to be sluggish with Xen for me (I didn’t investigate this too much), secondly I want to be able to run the latest and greatest OS’s out without worrying about upgrading Xen (I’m a sucker for OS’s!). VirtualBox development has accelerated at a feverish pace, I started with VirtualBox 1.3 in 2007 and its come an insanely long way since then. When a new release comes along, its as easy as updating VirtualBox and getting all the benefits. Plus with SunOracle‘s backing of VirtualBox you know things are going to work well on OpenSolaris, the Extras repository of VirtualBox makes it as easy as doing a pkg update.

I’m still quite intrigued by the way KVM is heading and how it will pan out, but for the future zeus, it will be VirtualBox.

{lang: 'en-GB'}
Share