About two years ago the zones team sat down and began to create the
solaris8 brand
for zones. This brand allows you to run your existing
Solaris 8 system images inside of a branded zone on Solaris 10. One
of the key goals for this project was to easily enable migration of Solaris 8
based systems into a zone on Solaris 10. To accomplish this, as part of the project we
built support for a “physical to virtual” capability, or p2v for short.
The idea with p2v is that you can create an image of an existing system
using a flash archive, cpio archive, a UFS dump, or even just a file system image
that is accessible over NFS, then install the zone using that image.
There is no explicit p2v tool you have to run; behind the scenes the zone installation process
does all of the work to make sure the Solaris 8 image runs correctly inside
of the zone.

Once we finished the solaris8 brand we followed that with the
solaris9 brand
which has this same p2v capability. Of course, while we were doing
this work, we understood that having a similar feature for native zones would
be useful as well. This would greatly simplify consolidation using zones,
since you could deploy onto bare metal, then later consolidate that application
stack into a zone with very little work.

The problem for p2v with native zones is that there is no brand module
that mediates between the user-level code running in the zone and the
kernel code, as we have with the solaris8 and solaris9 brands. Thus, the
native zones must be running user-level code that is in sync with the kernel.
This includes things like libc, which has a close relationship with the kernel.
Every time a patch is applied which impacts both kernel code and user-level
library code, all of the native zones must be kept in sync or unpredictable
bugs will occur.

Just doing native p2v, as we did for the solaris8 and solaris9 brands, doesn’t make
sense since the odds that the system image you want
to install in the zone will be exactly in sync with the kernel are pretty low.
Most deployed systems are at different patch levels or even running different minor releases (e.g.
Solaris 10 05/08 vs. 11/08), so there is no clean way to reliably p2v those images.

We really felt that native p2v was important, but we couldn’t make any progress
until we solved the problem of syncing up the system image to match the
global zone. Fortunately I was able to find some time to add this capability,
which we call
update on attach.
This was added into our zone migration
subcommands, ‘detach’ and ‘attach’, which can be used to move zones from
one system to another. Since zone migration has a similar problem as p2v,
where the source and target systems can be out of sync, we do a lot of
validation to make sure that the new host can properly run the zone. Of
course this validation made zone migration pretty restrictive. Now that we
have “update on attach”, we can automatically update the zone software when you
move it to the new host.

While “update on attach” is a valuable feature in its own right, we also built this
with an eye on p2v, since it is the enabling capability needed for p2v. In addition,
we leveraged all of the work
Dan Price
did on the installers for the solaris8 and solaris9 brands and were able to reuse much of that.
As with the solaris8 and solaris9 brands, the native brand installer accepts a variety of
image inputs; flar, cpio, compressed cpio, pax xustar,
UFS dump or a directly accessible root image (e.g. over NFS). It was also
enhanced to accept a pre-existing image in the zone root path. This is
useful if you use ZFS send and receive to set up the zone root and want
to then p2v that as a fully installed zone.

I integrated the
native p2v
feature into NV build 109 this morning. The
webrev
from the code review is still available if anyone is interested in seeing the scope of the
changes. At over 2000 lines of new code this is a pretty substantial addition
to zones which should greatly improve future zone consolidation projects.

A comment on my last post noted that there were no
sample chapters available for the book, however I just
noticed that Wiley has posted some samples on the
book’s
webpage.

The samples include
chapter one,
the
index,
and the
detailed table of contents.

The index and TOC are probably the best sections for getting a feel for the
material in the book. This is actually the first time I’ve seen the index myself,
since it was produced after we finished writing and the final pages were nailed
down. I haven’t reviewed it closely yet, but at first glance it looks to be
pretty comprehensive at 35 pages. I’ve always thought that the index was
critical for a book like this. The detailed TOC is also useful for getting a sense of
the topics covered in each chapter.

2008 was a busy year for me since I spent most of my free
time co-authoring a book on OpenSolaris; the
OpenSolaris Bible.

Having never written a book before, this was a new experience for me.
Nick originally had
the idea for writing a book on OpenSolaris and he’d already published
Professional C++ with Wiley,
so he had an agent and a relationship with a publisher. In December 2007 he contacted
me about being a co-author and after thinking it through, I agreed. I had
always thought writing a book was something I wanted to do, so I was
excited to give this a try. Luckily,
Dave agreed to be the
third author on the book, so we had our writing team in place. After
some early discussions, Wiley decided our material fit best into their
“Bible” series, hence the title.

In early January 2008 the three of us worked on the outline and decided which chapters
each of us would write. We actually started writing in early
February of 2008. Given the publishing schedule we had with Wiley, we had
to complete each chapter in about 3 weeks, so there wasn’t a lot of time to
waste. Also, because this project was not part of our normal work for
Sun, we had to ensure that we only worked on the book on our own time, that is evenings and
weekends. In the end it turned out that we each wrote
exactly a third of the book, based on the page counts.
Since the book came out at around 1000 pages, with approximately
950 pages of written material, not counting front matter or the index,
we each wrote over 300 pages of content. Over the course of the project we were
also fortunate that many
of our friends and colleagues who work on OpenSolaris were willing to review
our early work and provide much useful feedback.

We finished the first draft at the end of August 2008 and worked on the revisions
to each chapter through early December 2008. Of course the
OpenSolaris 2008.11
release came out right at the end of our revision process, so we had to scramble
to be sure that everything in the book was up-to-date with respect to the new
release.

From a personal perspective, this was a particularly difficult year because we
also moved to a “new” house in April of 2008. Our new house is actually about
85 years old and hadn’t been very well maintained for a while, so it needs some
work. The first week we moved in, we had the boiler go out, the sewer back up
into the basement, the toilet and the shower wouldn’t stop running, the
electrical work for our office took longer than expected, our DSL wasn’t hooked
up right, and about a million other things all seemed to go wrong. Somehow we
managed to cope with all of that, keep working for our real jobs, plus I was able
to finish my chapters for the book on schedule. I’m pretty sure
Sarah
wasn’t expecting anything like this when I talked to her about working on the book
the previous December.
Needless to say, we’re looking forward to a less hectic 2009.

If you are at all interested in OpenSolaris, then I hope you’ll find something in our
book that is worthwhile, even if you already know a lot about the OS. The book is
targeted primarily at end-users and system administrators. It has
a lot of breadth and we tried to include a balanced mix of introductory material as well as advanced
techniques. Here’s the table of contents so you can get a feel for whats in the book.

I. Introduction to OpenSolaris.
1. What Is OpenSolaris?
2. Installing OpenSolaris.
3. OpenSolaris Crash Course.
II. Using OpenSolaris
4. The Desktop.
5. Printers and Peripherals.
6. Software Management.
III. OpenSolaris File Systems, Networking, and Security.
7. Disks,  Local File Systems, and the Volume Manager.
8. ZFS.
9. Networking.
10. Network File Systems and Directory Services.
11. Security.
IV. OpenSolaris Reliability, Availability, and Serviceability.
12. Fault Management.
13. Service Management.
14. Monitoring and Observability.
15. DTrace.
16. Clustering for High Availability.
V. OpenSolaris Virtualization.
17. Virtualization Overview.
18. Resource Management.
19. Zones.
20. xVM Hypervisor.
21. Logical Domains (LDoms).
22. VirtualBox.
VI. Developing and Deploying on OpenSolaris.
23. Deploying a Web Stack on OpenSolaris.
24. Developing on OpenSolaris.

If this looks interesting, you can pre-order a copy from Amazon
here. It comes out early next month, February 2009, and
I’m excited to hear peoples reaction once they’ve actually had a chance to look
it over.

In my
last post
I talked a bit about the new way that software and dataset management works for
zones on the
2008.11
release.

One of the features that is still under development is to provide
a way to automatically keep the non-global zones in sync with
the global zone when you do a ‘pkg image-update’. The
IPS
project still needs some additional enhancements to be
able to describe the software dependencies between the
global and non-global zones. In the meantime, you must
manually ensure that you update the non-global zones after
you do an image-update and reboot the global zone. Doing
this will create new ZFS datasets for each zone which you can
then manually update so that they match the global zone software
release.

The easiest way to update the zones is to use the new detach/attach
capabilities we added to the 2008.11 release. You can simply detach
the zone, then re-attach it. We provide some support for the zone
update on attach
option for ipkg-branded zones, so you can use ‘attach -u’ to simply update
the zone.

The following shows an example of this.

# zoneadm -z jj1 detach
# zoneadm -z jj1 attach -u
Global zone version: pkg:/entire@0.5.11,5.11-0.101:20081119T235706Z
Non-Global zone version: pkg:/entire@0.5.11,5.11-0.98:20080917T010824Z
Updating non-global zone: Output follows
Cache: Using /var/pkg/download.
PHASE                                          ITEMS
Indexing Packages                        54/54
DOWNLOAD                                    PKGS           FILES       XFER (MB)
Completed                                     54/54   2491/2491   52.76/52.76
PHASE                                        ACTIONS
Removal Phase                            1253/1253
Install Phase                                 1440/1440
Update Phase                               3759/3759
Reading Existing Index                            9/9
Indexing Packages                               54/54
pkg:/entire@0.5.11,5.11-0.98:20080917T010824Z

Here you can see how the zone is updated when it is re-attached to the
system. This updates the software in the currently active dataset associated with
the global zone BE. If you roll-back to an earlier image, the dataset associated
with the zone and the earlier BE will be used instead of this newly updated dataset.
We’ve also enhanced the IPS code so it can use the pkg cache from the global
zone, thus the zone update is very quick.

Because the zone attach feature is implemented as a brand-specific capability,
each brand provides its own options for how zones can be attached. In addition
to the -u option, the ipkg brand supports a -a or -r option. The -a option allows
you to take an archive (cpio, bzip2, gzip, or USTAR tar) of a zone from another
system and attach it. The -r option allows you to receive the output of a ‘zfs send’
into the zone’s dataset. Either of these options can be combined with -u to
enable zone migration from one OpenSolaris system to another. An additional
option, which didn’t make it into 2008.11, but is in the development release, is
the -d option, which allows you to specify an existing dataset to be used for the
attach. The attach operation will take that dataset and add all of the properties
needed to make it usable on the current global zone BE.

If you used zones on 2008.11, you might have noticed that the zone’s dataset
is not mounted when the zone is halted. This is something we might change in
the future, but in the meantime, one final feature related to zone detach is that it
leaves the zone’s dataset mounted. This provides and easy way to access the zone’s
data. Simply detach the zone, then you can access the zone’s mounted file system,
then re-attach the zone.