Sun Logical Domains

We got a Sun T1000 server in last week to give Solaris Logical Domains a whirl. The idea is that if we can reduce the number of physical boxes we need to run Solaris 10 testing, we’ll save on power and heat. (And have cheaper machines, too.)

My first impression of Logical Domains is that it’s very, very beta. They may have released it, but they’re still working the kinks out. The bulk of the release notes document is made up of known bugs, two of which so far have bit me. Despite this, though, I still have this crazy, unnatural fondness for Solaris. I’m writing this not as a Solaris sysadmin (which I am definitely not) but as a technology professional who could be accused of being chronically lazy and wanting a laymans explanation for a lot of things.

The Sun hypervisor seems to be this crazy hybrid of a hardware and software based hypervisor. It only runs on boxes that have the SPARC T1 processor, which includes the T1000, T2000, Enterprise versions of the T1000 and T2000, Netra 2000, Netra CP3060 Blade, and the T6300 Blade Module. I’m not going into any great in-depth discussions about how to install it, since the documentation does a pretty good job of explaining it - up until you need to load the OS.

Anyway, in my case, I took a T1000 and did a standard Solaris 10 (11/06) install. I upgraded the firmware of the box (more on this in a second), and installed some Solaris patches to add support for  LDom’s. The firmware upgrade is necessary as the hypervisor itself runs out of hardware, as opposed to VMWare which runs in software. Additionally, all of the configs for VM’s live in hardware memory, and not as files on the OS. You then install the Solaris security hardening software, and the Logical Domains software. Presumably the security software prevents you from “breaking” the host OS to the ok> prompt and thereby halting your VM’s. It also flashes an ominous MOTD when you log in.

Once all that is done, you create a primary domain. This is your host OS, which will handle your disk I/O, network I/O, and generally handle all of your resources.  You assign it a number of virtual CPUs, memory, and configure it’s networking. Like VMWare ESX, you configure virtual switches, which is just a fancy name for bridging interfaces like you do in VMWare workstation and server. In the case of my environment, I have bge0 (the first nic in the box) and bge1 (the second) configured. They both become vsw0 and vsw1 (virtual switch). Configuring the number of virtual cpu’s and memory sets a limit to how much the domain can eat up and frees up other resources for the virtual machines. Normally, the OS would consume all of the resources in the box — setting these limits puts a hard stop on how much it will use. The primary domain also acts as a terminal server - it opens various ports for telnetting into virtual consoles for the guests, as well as acts as a virtual disk server.

Now you create your guest domain. Like the primary domain, you assign virtual CPU (at a rate of 4 per core), RAM, disk, and network. You then bind the domain to the server, which then allows you to telnet to the port that it gets assigned. You’re then looking at a ok> prompt just as if you were connected to any other Sun box.

The only way to install Solaris (and, by the way, it can only be Solaris 10) on a guest domain, is to Jumpstart it over the network. Normally you would issue a command such as:

ok> boot net:dhcp - install

But in ldom’s it doesn’t work. The virtual nvram doesn’t have an entry for “net” so you have to make one. I found that out in a PPT marked “Sun Confidential” I found on the web. With that down, I was able to install Solaris 10.

In my next blog entry, I will complain about what has gone wrong since I installed Solaris, including why you probably don’t want to add more physical RAM to your Sun box after you have this set up.