news aggregator

I’ll be heading in to Austin tomorrow to check in and get my badge before the crazy lines roll in Friday morning. Tomorrow night I’ll be at the Social Media BBQ at Emo’s. Friday I’ll start off in the Blogger’s Lounge because that’s where all the cool kids hang out. Depending on how my retrospective call at work goes (it’s the end of our agile development cycle) I hope to attend a few UX panels. You can find all the panels I’ve selected to attend on my SXSW schedule.

Saturday night the company I work for is cosponsoring a party with Tumblr, FourSquare, SoundCloud, and KickStarter at Emo’s. They are also cosponsoring the Houston@SXSW party Monday night at Hudson on Fifth. You can RSVP for that party here.

Sadly there doesn’t seem to be any Austin Barcamp activities going on. If there is something going on that I don’t know about yet, please inform me! I’d like to start a discussion on the need for good designers in free and open source software development and to brainstorm ways to get designers more interested in participating!

If you can’t find me at the panels I’ve posted, the Blogger’s Lounge, or at these parties - you should be able to spot me at The Planet’s booths (303 & 305) in the tradeshow sometime Sat & Sun 12 - 6 and Monday 12 - 4. I’m told there will be sweeeeeet giveaways so I hope to see you there!

The release of 2010.Q1 for Sun Storage 7000 Appliances brings a wide variety of new features to the appliance family, including an increased number of ways appliances are integrated directly into storage ecosystems. In addition to the Sun Storage 7000 Management Plug-in for Oracle Enterprise Manager 10g Grid Controller 1.0, Q1 2010 brings integration with Microsoft Volume Shadow Copy Service (VSS) through the Sun Storage 7000 Provider for VSS Software 1.0.

For those that have a Microsoft Windows operating environment and are not familiar with Microsoft VSS, it is basically a framework that facilitates creating fast and consistent snapshots of volumes through the coordination of the applications doing backup-type operations (readers, also known as a requestor) and the applications that are writing to volumes (writers). In layman's terms, this facility built into the Microsoft Windows Platforms gives the ability for an application like Symantec NetBackup Advanced Client to take a snapshot of a volume that is being actively used by an application like Oracle Database or Microsoft Exchange. The reader notifies the framework that it would like to take a snapshot. The framework notifies applications that they need to coalesce. The applications complete their coalescing and the framework tells the reader to go. When complete, the framework releases the application to continue writing. An abstract illustration of the environment is shown here:

The whole process of coalescing and taking a snapshot of a volume can take only a few seconds (more depending on how complex the coalesce and snapshot operations are).

For efficiency, storage appliances (like Sun Storage 7000 Appliances) create a "Hardware Provider", also known as a "VSS Block Target Provider". The Hardware Provider takes over the snapshot and clone operations based on the GUID of a SCSI LUN (in our case for Release 1.0, this is iSCSI only, Fibre Channel is not supported).

Installation of the Sun Storage 7000 Provider for VSS is through the InstallShield package downloaded from the Oracle Corp. Download Site (this site contains the Sun domain ID if you notice those types of things). The installation process should be straightforward and results in a new folder accessible from the "Start" menu (Start -> All Programs -> Sun Microsystems, Inc. -> Sun Storage 7000 Provider for VSS). In this folder are two entries: a README.txt (required reading ... seriously) and the Sun Storage 7000 Provider for VSS ConfigUtil. This latter entry (the ConfigUtil) is also placed on the desktop for quick access.

To verify the hardware provider is registered with the Microsoft VSS Framework, find a command prompt and type "vssadmin list providers". As shown in the following screen capture, this results in a printout of registered providers that will include entry for the Sun Storage 7000 Provider for VSS. The Version number you will see in an original installation is 1.0.10.

This means that backup application, such as Symantec NetBackup Advanced Client, can set up policies that leverage the hardware provider for fast snapshots. There is one more setup operation that must be completed before the policy will successfully complete. The Sun Storage 7000 Provider for VSS uses a combination of "Management Path" and "Data Path" operations to achieve snapshot and clones of iSCSI Volumes.

The "Management Path" credentials to any Sun Storage 7000 Appliances supplying iSCSI LUNs to the Microsoft Windows client must be entered through the "Sun Storage 7000 Provider for VSS ConfigUtil". From the Microsoft Windows platform that is consuming the iSCSI LUNs and has the Hardware Provider installed, open the ConfigUtil (form the Start Menu or the Desktop). Use the DNS names of Sun Storage 7000 Appliances or IP Addresses (I prefer IP Addresses), enter each Sun Storage 7000 Appliance Management Path along with User ID and Passwords given to you by the storage administrator for each appliance. Tap the shoulder of the Storage Administrator and remind them you could use a non-root User ID as guided by the README.txt in the download. Most access rights can be removed from the User ID though they do need to have a role that facilitates snapshot creation and clone creation for the shares that you access. Use the template in the README.txt for a starting point.

This screen capture shows the entry of credentials:

Once the credentials are entered properly, access from applications is seamless.

Look for specific documentation on using Hardware Providers with the specific applications that you use on the Microsoft Windows platform. Because VSS is a framework, you may have products that utilize the framework that we did not specifically test in our labs. The README.txt in the download contains a list of applications that we have run using the provider.

Conclusion
The Sun Storage 7000 Provider for VSS Software 1.0 is a Hardware Provider that plugs into the Microsoft VSS Framework on Microsoft Windows 2003 and 2008, 64-bit or 32-bit variations. Using the provider, backup applications and other requestors can make snapshots and clones directly on Sun Storage 7000 Appliances for iSCSI LUNs consumed by the Microsoft Windows Client on the system that the Hardware Provider is installed on. The installation of the provider is quick and you should verify that it was registered with the Microsoft VSS Framework. You must then enter User ID and Password information for each target Sun Storage 7000 Appliance. No further intervention with the Microsoft VSS Framework is necessary from that point forward and the primary work you'll do is configure your Backup Applications to make use of the Hardware Provider through Backup Policies.

For additional reading, use the following resources:

• Sun Storage 7000 Appliance 2010.Q1 Release Information, start at the entry Turning the Corner
  
• The README.txt included in the Sun Storage 7000 Provider for VSS Software 1.0 Download
  
• The Overview for Microsoft Volume Shadow Copy Service (VSS)
  

Over the past few months I've blogged on various monitoring and alerting topics for Sun Storage 7000 Appliances. Besides my favorite of the blogs (Tweeting your Sun Storage 7000 Appliance Alerts), the culmination of this monitoring work is now available as the Sun Storage 7000 Management Plug-in for Oracle Enterprise Manager 10g Grid Controller 1.0, for use with the just shipped 2010.Q1 software release for the Sun Storage 7000 Appliance Family. Phew, that's a bit of a mouthful for a title but I'll just refer to it as the SS7000MPfOEMGC, does that help? Well, maybe not

Sun Storage 7000 Management Plug-in for Oracle Enterprise Manager 10g Grid Controller creates a coupling between the enterprise-wide monitoring provided by Oracle Grid Control and the monitoring and analytics provided by Sun Storage 7000 Appliances. If you are not familiar with Oracle Grid Control, there is a nice write-up within the Installation and Configuration Guide for Oracle Grid Control. In a nutshell, Oracle Grid Control aids in monitoring your vertical data center rather than simply being an aggregation of horizontal health information. The documentation describes it as software and the infrastructure it runs on but I would simply call it a "Vertical Data Center Monitoring Environment".

The goal of the plug-in to Oracle Grid Control is to facilitate a Database Administrator in their use of Sun Storage 7000 Appliances without attempting to reproduce the world-class analytics available within Sun Storage 7000 Appliances. In other words, the goal is to create a bridge between the world of Database Administration and the world of Storage Administration with just enough information so the two worlds can have dialog about the environment. Specifically, the Plug-in for Sun Storage 7000 Appliances is targeted at the following tasks:

• Connecting Database deployments with Sun Storage 7000 resources that provide storage infrastructure to the database
  
• Understanding the performance metrics of a Database from the perspective of the Appliance (what cache resources are being used for a database, what network resources and the performance being delivered, and how various storage abstractions are being used by the database)
  
• Providing a Federated view of Sun Storage 7000 Appliances deployed in the environment (including storage profiles and capacities, network information and general accounting information about the appliances)
  
• Providing detailed performance metrics for use in initial service delivery diagnostics (these metrics are used to have more detailed conversations with the Storage Administrator when additional diagnostics are required)
  

Let's take a look at one of the more interesting scenarios as a simple way of showing the plug-in at work rather than reproducing the entire Installation Guide in blog-form.

Download the Plug-in for Sun Storage 7000 Appliances, Unzip the downloaded file, and read the Installation Guide included with the plug-in.

Follow the instructions for installing, deploying the plug-in to agents and adding instances of Sun Storage 7000 Appliances to the environment for monitoring. Each instance added takes about 60 minutes to fully populate with information (this is simply the nature of this being a polling environment and the plug-in is set-up to monitor data sets that don't change often less frequently ... 60 minutes ... than data sets that do change frequently ... 10 minute intervals).

Once data is funneling in, all of the standard appliance-centric views of the information are available (including the individual metrics that the plug-in collects) as well as a view of some of the important high-level information presented on the home page for an instance (provided you are using Oracle Grid Control 10.2.0.5). Here is a view of a single appliance instance's home page:

Looking into the Metrics collected for an appliance brings you to a standard displays of single metrics (as shown below) or tables of related metrics (all standard navigation in Oracle Grid Controller for plug-in components).

Included in the plug-in for Sun Storage 7000 Appliances are 5 reports. Of these reports, 3 run against a single instance of a Sun Storage 7000 Appliance and are available from both the context of the single instance and the Oracle Grid Control Reports Tab while 2 run against all monitored instances of Sun Storage 7000 Appliances and are only available from the Reports Tab. Among the 5 Reports are 2 that combine information about Databases deployed against NFS mount points and Sun Storage 7000 Appliances that export those NFS mount points. The two reports are:

• Database to Appliance Mapping Report - Viewable from a single target instance or the Reports Tab, this report shows databases deployed against NFS shares from a single Sun Storage 7000 Target Instance
  
• Federated Database to Appliance Mapping Report - Viewable only from the Reports Tab, this report shows databases deployed against NFS shares from all monitored Sun Storage 7000 Appliances
  

Looking at the "Master" (top-level) Database to Appliance Mapping Report (shown below) you will see a "Filter" (allowing you to scope the information in the table to a single Database SID) and a table that correlates the filtered Database SID to Network File Systems shared by specific appliances along with the Storage IP Address that the share is accessed through, the appliance's Storage Network Interface and the name that the appliance is referred to as throughout this Grid Control instance.

From the Master report, 4 additional links are provided to more detailed information that is filtered to the appliance abstraction that is used by the Database SID. The links in the columns navigate in the following way:

• Database Link - This link takes the viewer to a drill-down table that shows all of the files deployed on the shares identified in the first table. With this detail report, and administrator could see exactly what files are deployed where. The table also contains the three links identified next.
  
• Network File System - Takes the viewer down to a detailed report showing metadata about the share created on the appliance, how the cache is used (ARC and L2ARC) for this share and general capacity information for the share.
  
• Storage IP Address - Takes the viewer to the Metric Details that relate to the appliance configuration (serial number, model, etc...).
  
• Storage Network Interface - Takes the viewer to metadata about the network interface as well as reports on the Network Interface KB/sec and NFS Operations Per Second (combined with the NFS Operations Per Second that are allocated to serving the share that the database resides on)
  

The detail reports for the Network File System and Storage Network Interface (both of which are not directly accessible from the Reports Tab) use a combination of current metrics and graphical time-axis data, as shown in the following report:

Wherever applicable, the Detail Reports drill further into Metric Details (that could also be accessed through an appliance instance target home page).

It is important to note that several of these reports combine a substantial amount of data into a single page. This approach can create rather lengthy report generation times (in worst case scenarios up to 5 minutes). It is always possible to view individual metrics through the monitoring home page. As metric navigation is much more focused and relates to a single metric, metric navigation always performs faster and is preferred unless the viewer is looking for a more complex assembly of information. With the reports, an administrator can view network performance and storage performance side by side which may be more helpful in diagnosing service delivery issues than navigating through single metric data points.

In addition to a substantial number of collected metrics there are several alerts that are generated on various appliance thresholds that can occur throughout the operation of target appliances.

Conclusion
Oracle Grid Control gives a fully integrated view of the "Vertical" data center, combining software infrastructure with hardware infrastructure (including storage appliances). Sun Storage 7000 Management Plug-in for Oracle Enterprise Manager 10g Grid Controller 1.0 presents Sun Storage 7000 Appliances within the vertical context and presents metrics and reports tailored specifically towards Sun Storage 7000 Appliances as viewed by a Database Administrator. For more information on the plug-in and software discussed in this entry:

• Sun Storage 7000 Appliance 2010.Q1 Release Information, start at the entry Turning the Corner
  
• Sun Storage 7000 Oracle Enterprise 10g Grid Controller Plug-in Installation Guide - Extract from the Downloaded Plug-in .zip File
  
• Oracle® Enterprise Manager Grid Control Installation and Configuration Guide 10g Release 5 (10.2.0.5.0)
  

I've found a couple issues when using NetBeans to generate a WSDL file for an EE 6 web service, and then customizing that WSDL file. Some other users have reported them as well, so I thought I'd share them with the community.

• When I set a new wsdlLocation attribute in the web service class, this is ignored and the default, autogenerated WSDL is used instead. It appears that GlassFish v3 doesn't recognize the wsdlLocation attribute. A bug against GlassFish has been reported (11437), there's a fix in the nightly GF builds, and the fix will be there in 3.0.1. In the meantime, it's probably worth downloading and installing the nightly GF build and registering this server in the IDE.
• When you use NetBeans to generate the WSDL for an EE 6 JAX-WS service, generation fails because wsgen "Could not create declaration for annotation type javax.ejb.Stateless" or javax.ejb.EJB. Workaround is simply to comment out those annotations, generate WSDL, then uncomment them. Problem I am told is with wsgen in JAX-WS, not an NB issue. I posted this as issue #837 in jax-ws, but the reply was that this was a problem with apt not wsgen. The workaround is to "Pass the Java EE API jar that contains @Stateless in the wsgen classpath." However, I don't know if the user can do or must do that manually or if we can fix NetBeans to do it.

As part of the existing OpenSolaris constitution, we (the OpenSolaris Governing Board) are required to hold an annual "meeting" before the election in order for the election to be valid.  While, generally, this involves a fetch a rock exercise of core contributors (aka "members") logging into the forum, announcing themselves, then logging off, we do occasionally have useful and interesting conversations here. (and before you comment how silly that requirement is, please note that we have a new proposed constitution at this year's election that removes the annual meeting requirement).

Peter Tribble invited Dan Roberts to our virtual meeting the day after it started, and he joined and was very forthcoming about Oracle and their thoughts on OpenSolaris and Solaris:

"Oracle is investing more in Solaris than Sun did prior to the acquisition, and will continue to contribute technologies to OpenSolaris, as Oracle already does for many other open source projects."

While not all questions could be answered at that time, I was very pleased to see the community being engaged and concerns listened to.

In addition to the last posting here a new one:

[Original by Johnny Vulcan via flicke under cc-licence; edited by Henrik Heigl]

[Original by Velo Steve via flickr, edited by Henrik Heigl]

[Original by Abraxas3d via flickr, edited by Henrik Heigl]

It's a little hard to believe that it's been only fifteen months since we shipped our first product. It's been a hell of a ride; there is nothing as exhilarating nor as exhausting as having a newly developed product that is both intricate and wildly popular. Especially in the domain of enterprise storage -- where perfection is not just the standard but (entirely reasonably) the expectation -- this makes for some seriously spiked punch.

For my own part, I have had my head down for the last six months as the Technical Lead for our latest software release, 2010.Q1, which is publicly available as of today. In my experience, I have found that in software (if not in life), one may only ever pick two of quality, features and schedule -- and for 2010.Q1, we very much picked quality and features. (As for schedule, let it be only said that this release was once known as "2009.Q4"...)

2010.Q1 Quality

You don't often see enterprise storage vendors touting quality improvements for a very simple reason: if the product was perfect when you sold it to me, why are you talking about how much you've improved it? So I'm going to break a little bit with established tradition and acknowledge that the product has not been perfect, though not without good reason. With our initial development of the product, we were pushing many new technologies very aggressively: not only did we seek to build enterprise-grade storage on commodity components (a deceptively daunting challenge in its own right), we were also building on entirely new elements like flash -- and then topped it all off with an ambitious, from-scratch management stack. What were we possibly thinking by making so many bets at once? We made these bets not out of recklessness, but rather because they were essential elements of our Big Bet: that customers were sick of paying monopoly rents for enterprise storage, and that we could deliver a quantum leap in price-performance. (And if nothing else, let it be said that we got that one very, very right -- seemingly too right, at times.) As for the specific technology bets, some have proven to be unblemished winners, while others have been more of a struggle. Sometimes the struggle was because the problem was hard, sometimes it was because the software was immature, and sometimes it was because a component that was assumed to have known failure modes had several (or many) unanticipated (or byzantine) failure modes. And in the worst cases, of course, it was all three...

I'm pleased to report that in 2010.Q1, we turned the corner on all fronts: in addition to just fixing a boatload of bugs in key areas like clustering and networking, we engaged in fundamental work like Dave's rearchitecture of remote replication, adapted to new device failure modes as with Greg's rearchitecture around resilience to HBA logic failure, and -- perhaps most importantly -- integrated critical firmware upgrades to each of the essential components of the I/O path (HBAs, SIM cards and disks). Also in 2010.Q1, we changed the way the way that we run the evaluation of the software, opening the door to many in our rapidly growing customer base. As a result, this release is already running on more customer production systems than any of its predecessors were at the time that they shipped -- and on many more eval and production machines within our own walls.

2010.Q1 Features

But as important as quality is to this release, it's not the full story: the release is also packed with major features like deduplication, iSER/SRP support, Kerberized NFS support and Fibre Channel support. Of these, the last is of particular interest to me because, in addition to my role as the Technical Lead for 2010.Q1, I was also responsible for the integration of FC support into the product. There was a lot of hard work here, but much of it was born by John Forte and his COMSTAR team, who did a terrific job not only on the SCSI Target Management facility (STMF) but also on the base ALUA support necessary to allow proper FC operation in a cluster. As for my role, it was fun to cut the code to make all of this stuff work. Thanks to some great design work by Todd Patrick, along with some helpful feedback from field-facing colleagues like Ryan Matthews, I think we came up with a clean, functional interface. And working closely with both John and our test team, we have developed a rock-solid FC product. But of course (and as one might imagine), for me personally, the really gratifying bit was adding FC support to analytics. With just a pinch of DTrace and a bit of glue code, we now have visibility into FC operations by LUN, by project, by target, by initiator, by operation, by SCSI command, by size, by offset and by latency -- and by any combination thereof.

As I was developing FC analytics, I would use as my source of load a silly disk benchmark I wrote back in the day when Adam and I were evaluating SSDs. Here for example, is that benchmark running against a LUN that I named "thicktail-bench":

The initiator here is the machine "thicktail"; it's interesting to break down by initiator and see the paths by which thicktail is accessing the LUN:

(These names are human readable because I have added aliases for each of thicktail's two HBA ports. Had I not added those aliases, we would see WWNs here.) The above shows us that thicktail is accessing the LUN through both of its paths, which is what we would expect (but good to visually confirm). Let's see how it's accessing the LUN in terms of operations:

Nothing too surprising here -- this is the write phase of the benchmark and we have no log devices on this system, so we fully expect this. But let's break down by offset:

The first time I saw this, I was surprised. Not because of what it shows -- I wrote this benchmark, and I know what it does -- but rather because it was so eye-popping to really see its behavior for the first time. In particular, this captures an odd phase I added to this benchmark: it does random writes across an increasing large range. I did this because we had discovered that some SSDs did fine when the writes were confined to a small logical region, but broke down -- badly -- when the writes were over a larger reason. And no, I don't know why this was the case (presumably the firmware was in fragmented/wear-leveling/cache-busting hell); all I know is that we rejected any further exploration once the writes to the SSD were of a higher latency than that of my first hard drive: the IBM PC XT's 10 MB ST-412, which had roughly 95 ms writes! (We felt that expecting an SSD to have better write latency than a hard drive from the first Reagan Administration was tough but fair...)

What now?

As part of our ongoing maturity as a product, we have developed a new role here at Fishworks: starting in 2010.Q1, the Technical Lead for the release will, as the release ships, transition to become the full-time Support Lead for that release in the field. This means many things for the way we support the product, but for our customers, it means that if and when you do have an issue on 2010.Q1, you should know that the buck on your support call will ultimately stop with me. We are establishing an unprecedented level of engineering integration with our support teams, and we believe that it will show in the support experience. So welcome to 2010.Q1 -- and happy upgrading!

One of the areas I'm keen to improve in the next version of JAX-RS is link creation. JAX-RS already offers UriBuilder but I think an annotation driven approach could save a lot of repetitive coding.

I've been experimenting with a couple of annotations that I think would be useful and I just checked in an experimental extension that partially implements what I have in mind. Suppose you have a resource like this:

and you want to include a URI to this resource in a representation. Using the new extension you can just annotate a field in your representation class like this:

and then Jersey will build the appropriate URI and inject it into the representation before the representation is serialized by a message body writer.

If the URI template contains parameters, their values are obtained by looking for a bean property or field by the same name in the representation. E.g. consider the following representation class:

After processing by the extension, if the getWidgetId method returned "abc123", the value of the link field would be /context/widgets/abc123 where context is the deployment context.

The optional style property of the @Link annotation can be used to select between absolute URIs, absolute path and relative path depending on requirements.

In order to try out the extension you have to:

• Declare a dependency on the new jersey-server-linking module which is available in Jersey trunk under the experimental directory.
• Install the response filter in your application using an init-param in the web.xml like this: <servlet> <servlet-name>Jersey Web Application</servlet-name> <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class> <init-param> <param-name>com.sun.jersey.config.property.packages</param-name> <param-value>your application packages here</param-value> </init-param> <init-param> <param-name>com.sun.jersey.spi.container.ContainerResponseFilters</param-name> <param-value>com.sun.jersey.server.linking.LinkFilter</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet>

The Javadoc of the @Link annotation describes a couple of things that I plan to implement next:

• Support for the @Binding annotation already sketched out in the code. This will allow the values of template parameters to be pulled from alternate sources including differently named properties of the representation and resource.
• Support for EL expressions in link templates. This will allow a little more expressivity for templates that don't already exist as values of @Path annotations.

I’m totally jazzed about the upcoming Marketing FAD which kicks off this weekend.  The Marketeers will be meeting in Raleigh, Friday (late-late Friday, probably starting at zee Waffle House!) and ending Tuesday; I’ll be headed out of town Wednesday.  We have a boatload of deliverables we plan on knocking out, and on top of that, well, FUN STUFF.  We’re going to a hockey game, for one – and, coincidentally, It will be the Carolina Hurricanes vs. Phoenix Coyotes.

So for background: Ryan Rix and I are both traveling to RDU from… you guessed it, Phoenix.  I am not exactly the world’s biggest sports nut, but of course I have to root for my hometown, so I’ve been cracking jokes about how I’m going to be decked out in my full Coyotes gear (which, of course, I don’t actually have).  Here’s a fun clip from the marketing meeting yesterday on IRC, where we were discussing any last-minute items we needed to wrap up before our FAD:

In any case: I’m somewhat packed, I’m only bringing 2 pairs of shoes (zomg, I know, right?!) because I have to make room for…

COOKIES!!

I’m up to my ears in promises. ke4qqq may be going home from the FAD with 10 boxes of girl scout cookies because of his limesurvey awesomeness (plz see my ongoing tale of luv for survey-goodness on the fp.o market research wiki page).

But seriously: I had a point to this blog post. And I’m getting to it…. right… now.

I am looking forward to doing a quick video conference call next week with the guys in the Moscow OpenSolaris User Group. It will be at 2:30 in the morning for me, so I think I will make it a quick chat. Meeting details here. MOSUG info here. Special thans to Vladimir Legeza for the invite. Hopefully, some day I will be able to get to Russia. Never been. Always wanted to go.

I am looking forward to doing a quick video conference call next week with the guys in the Moscow OpenSolaris User Group. It will be at 2:30 in the morning for me, so I think I will make it a quick chat. Meeting details here. MOSUG info here. Special thans to Vladimir Legeza for the invite. Hopefully, some day I will be able to get to Russia. Never been. Always wanted to go.